Join leaders in Boston on March 27 for an exclusive night of networking, insights, and conversation. Request an invite here.
The pace at which AI video is spreading across the entertainment industry is quickening. This week alone came the news that buzzed-about 1970s period horror film Late Night With the Devil is using AI title cards to create its retro graphics (to the chagrin of some viewers), as well as the release of Stable Video 3D a 3D AI asset creation tool, and a new partnership between AI video startup Runway and ad agency Media.Monks, wherein the latter’s team members and clients would have access to the former’s text-and-image-to-video AI software.
But as it turns out, that wasn’t the only partnership Runway had in store: today, the New York City-based startup is also announcing a new partnership exclusively through VentureBeat, teaming-up with successful Italian music tech startup Musixmatch, which powers the on-device, live-synced lyrics to songs on Spotify, Apple Music, Amazon Music, Facebook, Instagram, Tidal, Vevo, and more.
The partnership will give the more than a million musicians and artists using Musixmatch access to Gen-2, Runway’s latest and greatest AI video generating model, which has been steadily upgraded with features such as a Multi Motion Brush and region selector.
The idea is to “allow creators to bring their songs to life through dynamic video content at scale and aligned with their creative visions,” according to a press release Runway provided to VentureBeat via email.
In other words: musicians will now be able to generate AI videos with Gen-2 that will then be synced to the lyrics of their songs using Musixmatch’s platform, potentially speeding up the creation of “lyrics videos,” popular videos that display a song’s lyrics in-time with music over ambient visuals — as opposed to full music videos, which often feature the artist, band, or characters in a narrative.
Already, some artists have been given Gen-2 access through Musixmatch to create lyrics videos, including Atlanta trip-hop outfit Earthgang, which created the following video for their song “Osmosis” with Runway’s AI and Musixmatch’s lyrics syncing software:
A match made in musical heaven?
“We are thrilled to be able to put Runway’s models and creative experience directly into the hands of Musixmatch’s unique community of artists and musicians,” says Runway CEO Cristobal Valenzuela in the press release. “Runway’s models and tools will help these artists expand how they express their creative vision through video, and will help them forge deeper connections with their listeners and audiences around the world.”
“With AI we are witnessing a new form of creativity that simply put ‘thinks different’,” said Marco Paglia, Musixmach Chief of Product, alluding to the famed Apple slogan. “Today, our incredibly talented songwriters and musicians have a new assistant with whom to collaborate when making videos, powered by Musixmatch’s unique ability to extract the meaning from lyrics and Runway’s best-in-class generative video models. This will enable more music creators to make videos and drive continued innovation in this space. If this is just day one, I can’t wait to see what the future holds.”
Big name artists are already embracing AI to varied results
Already, a number of famous artists including A$AP Rocky, Madonna, Ye (formerly known as Kanye West), and Jared Leto’s rock band 30 Seconds to Mars have all used AI video generation programs — in some cases, Runway itself — to generate music videos or concert backing video.
Separately but relatedly, rapper Nicki Minaj got into a back and forth with users on X, the social network formerly known as Twitter, over her use of AI generated still images to promote her album Pink Friday 2.
Clearly, a number of high-profile names in the music business are already embracing AI to provide visuals to match their music, and Runway is smartly moving to make that operation even easier with this latest partnership.
Part of Runway’s steady expansion — even as it faces down intensifying competition
The company has also partnered with other software providers such as Canva to bake-in its AI video generation models to their programs, further exposing the tools to a wider audience.
Runway allows a limited number of free video generations, but after that, it charges a monthly subscription starting at $12 per month and rising through various tiers. The startup is funded by Google, among other investors.
For much of its existence and certainly following the release of Gen-2 in June 2023, Runway has been widely viewed by the burgeoning AI video and creative community as the gold standard in AI video generation.
Yet OpenAI stole Runway’s thunder and possibly its crown last month by demoing even more photorealistic, smoother, and faster moving videos made through its own model, Sora, though that model is only accessible to OpenAI employees and designated creative partners as of now.
Meanwhile, another rival AI video generation startup, Pika, has been steadily adding new audio generation features such as a “Lip Sync” feature powered by AI audio startup ElevenLabs that allows characters within a Pika-generated video to move their mouths in time with uploaded audio or text-to-audio AI clips. PIka further added a sound effects generation model of its own earlier this month.
And former Runway partner Stability AI, with which it co-created the open source Stable Diffusion AI model widely used for generating still images, also offers an open source video generation model, Stable Video Diffusion.
So Runway is now playing defense against all these insurgents and competitors, in a rapidly progressing space. Will its partnerships allow it to maintain a competitive edge, and possibly get even more data to introduce new features for its video generating AI software? Stay tuned.
