Hollywood's pivot to AI video has a prompting problem

The Verge

3 days ago

It has become almost impossible to browse the internet without having an AI-generated video thrust upon you. Open basically any social media platform, and it won't be long until an uncanny-looking clip of a fake natural disaster or animals doing impossible things slides across your screen. Most of the videos look absolutely terrible. But they're almost always accompanied by hundreds, if not thousands, of likes and comments from people insisting that AI-generated content is a new art form that's going to change the world.
That has been especially true of AI clips that are meant to appear realistic. No matter how strange or aesthetically inconsistent the footage may be, there is usually someone proclaiming that it's something the entertainment industry should be afraid of. The idea that AI-generated video is both the future of filmmaking and an existential threat to Hollywood has caught on like wildfire among boosters for the relatively new technology.
The thought of major studios embracing this technology as is feels dubious when you consider that, oftentimes, AI models' output simply isn't the kind of stuff that could be fashioned into a quality movie or series. That's an impression that filmmaker Bryn Mooser wants to change with Asteria, a new production house he launched last year, as well as a forthcoming AI-generated feature film from Natasha Lyonne (also Mooser's partner and an advisor at Late Night Labs, a studio focused on generative AI that Mooser's film and TV company XTR acquired last year).
Asteria's big selling point is that, unlike most other AI outfits, the generative model it built with research company Moonvalley is 'ethical,' meaning it has only been trained on properly licensed material. Especially in the wake of Disney and Universal suing Midjourney for copyright infringement, the concept of ethical generative AI may become an important part of how AI is more widely adopted throughout the entertainment industry. However, during a recent chat, Mooser stresses to me that the company's clear understanding of what generative AI is and what it isn't helps set Asteria apart from other players in the AI space.
'As we started to think about building Asteria, it was obvious to us as filmmakers that there were big problems with the way that AI was being presented to Hollywood,' Mooser says. 'It was obvious that the tools weren't being built by anybody who'd ever made a film before. The text-to-video form factor, where you say 'make me a new Star Wars movie' and out it comes, is a thing that Silicon Valley thought people wanted and actually believed was possible.'
In Mooser's view, part of the reason some enthusiasts have been quick to call generative video models a threat to traditional film workflows boils down to people assuming that footage created from prompts can replicate the real thing as effectively as what we've seen with imitative, AI-generated music. It has been easy for people to replicate singers' voices with generative AI and produce passable songs. But Mooser thinks that, in its rush to normalize gen AI, the tech industry conflated audio and visual output in a way that's at odds with what actually makes for good films.
'You can't go and say to Christopher Nolan, 'Use this tool and text your way to The Odyssey,'' Mooser says. 'As people in Hollywood got access to these tools, there were a couple things that were really clear — one being that the form factor can't work because the amount of control that a filmmaker needs comes down to the pixel level in a lot of cases.'
To give its filmmaking partners more of that granular control, Asteria uses its core generative model, Marey, to create new, project-specific models trained on original visual material. This would, for example, allow an artist to build a model that could generate a variety of assets in their distinct style, and then use it to populate a world full of different characters and objects that adhere to a unique aesthetic. That was the workflow Asteria used in its production of musician Cuco's animated short 'A Love Letter to LA.' By training Asteria's model on 60 original illustrations drawn by artist Paul Flores, the studio could generate new 2D assets and convert them into 3D models used to build the video's fictional town. The short is impressive, but its heavy stylization speaks to the way projects with generative AI at their core often have to work within the technology's visual limitations. It doesn't feel like this workflow offers control down to the pixel level just yet.
Mooser says that, depending on the financial arrangement between Asteria and its clients, filmmakers can retain partial ownership of the models after they're completed. In addition to the original licensing fees Asteria pays the creators of the material its core model is trained on, the studio is 'exploring' the possibility of a revenue sharing system, too. But for now, Mooser is more focused on winning artists over with the promise of lower initial development and production costs.
'If you're doing a Pixar animated film, you might be coming on as a director or a writer, but it's not often that you'll have any ownership of what you're making, residuals, or cut of what the studio makes when they sell a lunchbox,' Mooser tells me. 'But if you can use this technology to bring the cost down and make it independently financeable, then you have a world where you can have a new financing model that makes real ownership possible.'
Asteria plans to test many of Mooser's beliefs in generative AI's transformative potential with Uncanny Valley, a feature film to be co-written and directed by Lyonne. The live-action film centers on a teenage girl whose shaky perception of reality causes her to start seeing the world as being more video game-like. Many of Uncanny Valley's fantastical, Matrix-like visual elements will be created with Asteria's in-house models. That detail in particular makes Uncanny Valley sound like a project designed to present the hallucinatory inconsistencies that generative AI has become known for as clever aesthetic features rather than bugs. But Mooser tells me that he hopes 'nobody ever thinks about the AI part of it at all' because 'everything is going to have the director's human touch on it.'
'It's not like you're just texting, 'then they go into a video game,' and watch what happens, because nobody wants to see that,' Mooser says. 'That was very clear as we were thinking about this. I don't think anybody wants to just see what computers dream up.'
Like many generative AI advocates, Mooser sees the technology as a 'democratizing' tool that can make the creation of art more accessible. He also stresses that, under the right circumstances, generative AI could make it easier to produce a movie for around $10–20 million rather than $150 million. Still, securing that kind of capital is a challenge for most younger, up-and-coming filmmakers.
One of Asteria's big selling points that Mooser repeatedly mentions to me is generative AI's potential to produce finished works faster and with smaller teams. He framed that aspect of an AI production workflow as a positive that would allow writers and directors to work more closely with key collaborators like art and VFX supervisors without needing to spend so much time going back and forth on revisions — something that tends to be more likely when a project has a lot of people working on it. But, by definition, smaller teams translates to fewer jobs, which raises the issue of AI's potential to put people out of work. When I bring this up with Mooser, he points to the recent closure of VFX house Technicolor Group as an example of the entertainment industry's ongoing upheaval that began leaving workers unemployed before the generative AI hype came to its current fever pitch.
Mooser was careful not to downplay that these concerns about generative AI were a big part of what plunged Hollywood into a double strike back in 2023. But he is resolute in his belief that many of the industry's workers will be able to pivot laterally into new careers built around generative AI if they are open to embracing the technology.
'There are filmmakers and VFX artists who are adaptable and want to lean into this moment the same way people were able to switch from editing on film to editing on Avid,' Mooser says. 'People who are real technicians — art directors, cinematographers, writers, directors, and actors — have an opportunity with this technology. What's really important is that we as an industry know what's good about this and what's bad about this, what is helpful for us in trying to tell our stories, and what is actually going to be dangerous.'
What seems rather dangerous about Hollywood's interest in generative AI isn't the 'death' of the larger studio system, but rather this technology's potential to make it easier for studios to work with fewer actual people. That's literally one of Asteria's big selling points, and if its workflows became the industry norm, it is hard to imagine it scaling in a way that could accommodate today's entertainment workforce transitioning into new careers. As for what's good about it, Mooser knows the right talking points. Now he has to show that his tech — and all the changes it entails — can work.