Xavier ‘X’ Jernigan, the voice of Spotify’s DJ, explains what it’s like to be an AI

In March, Spotify launched its first AI-powered function with the debut of AI DJ, a sensible audio information with convincingly life like sound. This AI character was truly primarily based on an actual particular person – Xavier “X” Jernigan, Spotify’s head of Cultural Partnerships, who had the distinction of being the primary sound mannequin for the AI ​​function.

TechCrunch teamed up with Jernigan to be taught extra in regards to the AI ​​coaching course of and Spotify’s future plans for AI DJ efforts.

The brand new AI DJ customizes the music listening expertise for listeners by tailoring a number of music primarily based on their pursuits. He additionally commented on every music like an actual radio host.

Along with Jernigan’s main position at Spotify, he additionally hosts a number of Spotify podcasts, together with “The Window”, “Showstopper” and the now defunct “The Get Up” podcast. That is why he is used to creating his voice heard to hundreds of thousands of listeners. Nonetheless, having his voice known as an AI is a novel expertise.

Spotify selected Jernigan as its first sound mannequin as a result of “his voice and persona have already resonated with a lot of our listeners,” Jernigan advised TechCrunch. “[The company was] I am fairly certain it is going to resonate that manner too.

Spotify’s Morning Present “The Get Up” garnered practically 6 million listeners and was among the many high 10 podcasts on Spotify earlier than ending abruptly in 2022, demonstrating the ability of Jernigan.

Nonetheless, the podcast host admitted that being a sound mannequin for a DJ was troublesome to ponder at first.

“I acquired a suggestion to be this sound mannequin for the DJ and when it was introduced to me I used to be blown away,” Jernigan mentioned. “If that is the primary time you have heard of it, you have acquired nothing to have a look at and I am like, ‘Wait, what? It is going to be me however not me and it’ll appear like textual content and voice however me and synthetic intelligence?”

“Working with AI on this manner was a brand new expertise for me. I simply handed out,” he added.

Spotify says the AI ​​DJ was created utilizing each Sonantic and OpenAI applied sciences.

Sonantic is an AI startup that Spotify acquired final yr. The corporate’s expertise was accountable for creating life like AI-based sounds, together with the one used for Val Kilmer’s voice in “High Gun: Maverick.”

Previous to the acquisition, Jernigan famous that Spotify spent a number of years researching AI-powered expertise and was working “some iterations” on the DJ function. He declined to share precisely how lengthy the method took, however mentioned integrating Sonantic expertise “actually gears up.”

Jernigan defined the AI ​​coaching course of, which requires strolling right into a studio, studying a script, and talking with numerous cadences and intonations to convey completely different feelings. He fed the AI ​​with sure phrases that he solely used to make it really feel as life like as doable.

“We use the phrases I say… I do not say ‘melody’ for songs. “I do not discuss like that,” he mentioned. “I say ‘hits’ or ‘hits.’ “We even made a strategy of tips on how to say ‘hey’, tips on how to say ‘howdy’. I carried a pocket book with me and would write down these completely different sentences that had been one thing to say.

He added that the Spotify staff made certain to carry their pure pauses and breaths in order that the AI ​​voice sounded really human.

Even Jernigan’s mom stamped approval on the outcomes.

“[DJ] Mother handed her take a look at. Earlier than she got here out, I performed for her, I defined to her, and I am attempting to get her to come back to her senses,” she mentioned. “He listened to all of my podcasts, so he was used to listening to my voice recorded and performed earlier than, and he mentioned, ‘This appears precisely such as you.’ My mother mentioned it seemed like me, so I knew it was spot on.

Whereas life like AI sounds are already accessible, we would argue that Spotify’s DJ is the calmest and creepiest in comparison with every other we have heard. Whereas Google’s Duplex tech could sound life like, it could not sound nice to hearken to when attempting so as to add pleasure to your summer time jam playlist.

“For me, once I was doing the efficiency from a vocal standpoint, my aim was to attach with individuals, chat with individuals, and take into consideration one particular person. So once I was coaching the AI, once I was within the studio, I imagined an individual speaking to them and being their good friend,” he added.

Along with making the AI ​​sound pleasant to the viewers, the DJ’s design can also be made to really feel approachable.

The animated inexperienced circle that customers see whereas listening to the DJ is a nod to the Spotify emblem and strikes like a mouth when the AI ​​speaks.

“In terms of design, we thought of how we might customise the entire expertise – the way it labored, the way it sounded, the way it seemed, and the way we might personalize it for every consumer,” mentioned Emily Galloway, Spotify head of Personalization Product Design. TechCrunch. “For the visible facet, we explored early on some choices that felt extra technical (think about issues like sound waves). Nonetheless, that simply did not really feel proper since we needed to humanize the AI…”

“We needed it to appear and feel distinctive. Actually, it was so distinctive that it patented a design,” Galloway added.

Apart from recording his voice, Jernigan additionally contributed to the DJ in different methods.

In order that AI can present knowledgeable commentary on music, Spotify has created a writers’ room of curators, cultural consultants, and music consultants.

Jernigan has an in depth musical background, so he is additionally featured within the author’s room. Beforehand, he has labored for high artists like Diddy, Amy Winehouse and a pair of Chainz amongst others.

Whereas Jernigan is the DJ’s first sound mannequin, listeners have the potential to listen to extra sounds sooner or later.

TechCrunch requested Jernigan if the corporate plans to rent voice fashions that talk different languages.

“Keep tuned,” he implied.

AI DJ is presently solely accessible in English for Premium subscribers within the US and Canada. As of February, the DJ function remains to be in beta testing.

“We have got plenty of actually cool new options popping out total,” Jernigan mentioned. “There are some actually nice issues popping out.”