What’s good in the hood, folks? I’ll tell you what. Today my guest is Edu Duque – Founder and creator of Sonoteller.ai

Creative technologist and expert in music, AI, and advertising, Edu Duque has spent over 20 years leading product, technology, digital advertising, and audiovisual production teams. That includes more than a decade at Google, Utopia Music, and his own startup, Musikame. He is currently at the front of Sonoteller, a new music analysis AI that summarizes song lyrics and identifies genres, emotions, or instruments from the audio. Its goal is to enrich data for music catalogs, recommendations, audience analysis, search engines and a wide variety of use cases for music lovers and industry professionals.

Chris: Hi there! What inspired you to create Sonoteller?

Edu: Hello, since the emergence of music analysis engines, all of them have relied on analyzing the music when classifying it, which is ideal when you think of characteristics like genre, instrument identification, and even voice timbre. However, when it comes to emotions, music and instrumentation can lead to misunderstandings. We all know songs where the music is upbeat, but the lyrics are sad, even depressing. When classifying music, especially when making recommendations for creating playlists or associating it with moods, the content of the lyrics and what they aim to convey is crucial. The lyrics are the soul of a song. And without considering that “soul,” associating emotions with a musical piece loses all its meaning.

Artificial intelligence has evolved significantly in the last two years, and today we can process the audio of a song to identify its musical features, as well as understand what a song is about, what it aims to convey, whether the content is suitable for all audiences, and much more.

Imagine a creative agency working alongside a music curator when searching for the perfect song for a client’s advertising campaign. The music, its energy, and style are crucial to enhancing the creative brief and the audiovisual piece, but what about the lyrics? The connection to the brand’s values and the message it wants to convey are 100% linked to the song’s lyrics, and of course, to more sensitive issues like brand safety and associating an appropriate message with a brand.

On the other hand, there are many ways to approach musical analysis through algorithms and AI models, and Sonoteller complements well-known techniques with others that have only recently become possible, such as the use of LLMs, which offer many more possibilities beyond traditional analysis.

Chris: How does Sonoteller compare to other AI-powered music analysis tools on the market?

Edu: When it comes to capabilities, Sonoteller has two clear competitive advantages over traditional music analysis engines:

  • On one hand, as I mentioned, it identifies features of the song’s lyrics, which can range from a summary to scoring emotions, themes, language identification, etc. This is critical for the right understanding and tagging for songs, and it also overcomes the challenges for many catalog owners to get this information (having access to the original lyrics of a song is not always easy). Sonoteller analyzes lyrics from the audio, so there’s no need to get access to the lyrics themselves to understand and analyze the song.

  • On the other hand, unlike other analysis engines, Sonoteller is very flexible when it comes to categorizing aspects such as genres, subgenres, or emotions. DSPs (Spotify, YouTube Music, Deezer, Apple Music, etc.) use their own taxonomies when categorizing and processing music. Sonoteller allows for adapting these categories into different taxonomies when classifying music, which optimizes and simplifies the process of managing music catalogs for a wide variety of use cases.

Chris: What were the key technical challenges that Sonoteller needed to overcome in order to be a commercially viable product?

Edu: As you can imagine, designing a new product with a different approach than the traditional one was filled with challenges. We experimented with various AI models for audio processing, the creation of artifacts to identify the characteristics of the lyrics, choosing an efficient and scalable infrastructure (especially managing GPUs), blending traditional models with LLMs… and so on. Development and testing have been a continuous learning process, but at the same time, very enriching.

Chris: How do you see Sonoteller impacting the music industry?

Edu: On one hand, I trust that Sonoteller will help improve the management of musical catalogs, but overall, the development of new capabilities and use cases is what excites me the most. Especially those focused on branding and audiovisual production – I come from a long path in advertising so I always loved this creative duet of advertising and music. There are new use cases in co-creation with our partners that I’m sure will have a significant impact on the industry.

Chris: Can Sonoteller be used to identify plagiarism in music?

Edu: It’s not a use case I’ve considered yet, but it certainly could help to address that issue. The similarity of harmonies, lyrics, and the musical structure of a song are areas that could be explored using Sonoteller as a foundation. That’s an interesting one, indeed.

Chris: What are some of the challenges you’ve faced in developing and launching Sonoteller?

Edu: Beyond the technical challenges we discussed, I believe the key (like with any other product or service) lies in problem-solving for our clients and contributing to their business objectives. The Sonoteller website and its public API showcase some of the capabilities of the AI engine (like lyric analysis, music, or identifying the “golden minute” of a song). But the capabilities extend far beyond those when working with clients and partners, addressing their specific use cases and developing custom versions of Sonoteller. Without a doubt, one of the most relevant and enriching challenges.

It’s also great to discover new music through the website’s users (over 10,000 so far), music lovers who analyze songs in every conceivable language, which I never thought I’d understand. Now we can do it.

Chris: Thanks for being with me, any last words? Where can our readers follow you?

Edu: Thank you, Chris, for your interesting questions and time. It’s been a pleasure. You can follow me on LinkedIn, or feel free to share your ideas, use cases and feedback via email ([email protected]). I invite you to try Sonoteller, and I wish everyone great music!