YouTube to allow Shorts creators make music with AI vocals of prominent artistes

Dream Track is powered by Google DeepMind’s most advanced music generation model to date, Lyria

YouTube is allowing select Shorts creators in the United States to make original music snippets with artificial intelligence (AI)-generated vocals from prominent artistes as part of a new experimental feature called Dream Track, the video platform's latest move to attract creators.

This feature comes after YouTube introduced a policy that requires creators to disclose if they have uploaded altered or synthetic content that "appear realistic", including videos using AI tools. The policy is expected to be rolled out in the coming months.

Dream Track is powered by Lyria, which, the video platform, said was Google DeepMind’s most advanced music generation model. Nine artistes including Charlie Puth, Charli XCX, Demi Lovato, John Legend, Sia, T-Pain, Troye Sivan, Papoose, and Alec Benjamin are part of the initiative.

Creators can simply type an idea into the creation prompt and select a participating artiste that appears in the carousel to create unique soundtracks of up to 30 seconds, which they can use in their short videos.

A mockup of Dream Track Experiment in YouTube Shorts (Image Source: YouTube)

"At this initial phase, the experiment is designed to help explore how the technology could be used to create deeper connections between artists and creators, and ultimately, their fans" the company said in a blogpost.

Dream Track is part of YouTube's first set of AI-related music experiments that comes months after the Google-owned video platform announced plans to work closely with its music partners to safeguard the creative works and copyrights of artistes and ensure fair compensation in the era of generative AI.

In August, YouTube introduced a Music AI incubator with the world's largest music firm Universal Music Group as its first partner.

As part of this initiative, Universal Music artists, songwriters and producers will explore, experiment and offer feedback on YouTube's AI-related music tools and product experiments, the company said. YouTube had also stated plans to invest in evolving its rights management technology Content ID to adapt to the new wave of AI-generated content.

New music AI tools

In the blogpost on November 16, YouTube said artistes, songwriters and producers in the Music AI Incubator are helping the service "test, learn, gain feedback and hear ideas" to develop the best experiences possible for its community of artistes, viewers and fans.

"It was clear early on that this initial group of participants were intensely curious about AI tools that could push the limits of what they thought possible. They also sought out tools that could bolster their creative process. As a result, those early sessions led us to iterate on a set of music AI tools that experiment with those concepts" YouTube's global music head Lyor Cohen and vice president, emerging experiences & community products Toni Reid said in the blogpost.

These music AI tools are expected to help artistes augment their creative process and their music creation journey. For instance, creating a new guitar riff just by humming it or converting a song from one genre to another — say a pop track to reggaetón. "We’re developing prospective tools that could bring these possibilities to life and Music AI Incubator participants will be able to test them out later this year" the executives said.

YouTube said it was still in the "early stages of this work" and expects that its approach will continue to evolve in the future.

As part of its latest policy that was announced on November 14, YouTube had introduced the ability for its music partners to request the removal of AI-generated music content that mimics an artist’s unique singing or rapping voice.

"In determining whether to grant a removal request, we’ll consider factors such as whether content is the subject of news reporting, analysis or critique of the synthetic vocals," the firm said in the blogpost.

The feature will be available to labels or distributors who represent artists participating in the video service’s early AI music experiments and it will be expanded to additional labels and distributors over the coming months.

"We’re committed to collaborating with them as we enter this new era, critically exploring together new opportunities and developing sensible and sustainable controls, monetization and attribution frameworks" the blogpost read.

Generative AI push

YouTube is also gearing up to introduce its own AI products. In September, it unveiled a suite of generative AI products and tools targeted at creators.

This includes the ability to create AI-generated video or image backgrounds for short videos by just typing an idea into a prompt, helping creators come up with new video ideas, find music tracks for their videos and reach new audiences through features such as automatic dubbing.

Content created from these tools will be automatically labelled as altered or synthetic, the company said on November 14.