Seedance 2.0 may be the next great hope of AI video generation, but it’s still shoddy

Seedance 2.0 may be the next great hope of AI video generation, but it’s still shoddy

When Irish filmmaker Ruairi Robinson started uploading a series of short clips made with Seedance 2.0 – the latest video generation model from TikTok developer ByteDance – it was hard to deny that the footage was far more impressive than what we’ve seen from other generational AI outfits. The star of the clips (a digital duplicate of Tom Cruise) looked a lot like the real thing as he fought Brad Pitt, humanoid robots and zombies. And the characters moved with a complex fluidity that almost passed for choreography and was enhanced by kinetic ‘camera work’.

Gen AI enthusiasts like to proclaim that the traditionally produced entertainment industry is done, and some of Hollywood’s biggest studios seemed alarmed by Seedance’s recent capabilities as its ersatz Cruise videos continued to gain views online. Association for Film Images, Disney, largestAnd Netflix all sent ByteDance letters about copyright infringement claims. And in response to this said ByteDance that it would take steps “to strengthen current safeguards as we work to prevent the unauthorized use of intellectual property and likeness by users.” ByteDance has yet to officially release a version of Seedance that prevents users from generating footage that the company doesn’t have the rights to create.

Everything about the Seedance 2.0 rollout felt like a viral stunt, especially when studios have already made it clear they’re willing to sue when AI companies steal their IP. It’s true that the videos made by Seedance look much better than a lot of the stuff we’ve seen with Sora, Veo, Runway and others. But the fact that producing highly polished rip-offs is the new model’s main claim makes Seedance 2.0 just another slop generator – albeit a fancier one.

When we call AI video generation “slop,” we’re usually commenting on aesthetics and presentation. But the way AI images are created is a crucial part of the equation. Unlike traditionally produced films, shows and online videos – which are possible sloppy manufactured – things made with AI are “slop” because they are the product of workflows without any direct authorial or artistic intent. Unlike a team of human filmmakers, a generational AI video model can’t always follow the beats of a story or a character’s motivations, but it can parse simple inputs and generate outputs that corpses informed by a story (if you squint), because the program is trained on massive amounts of visual data.

In essence, Seedance is not that different from its peers

Being able to mimic the real (read: human-made) thing is the whole point of projects like Seedance 2.0, but the models can’t do that unless they’re first given an ample amount of source material to iterate on programmatically. And by allowing such blatant infringement of intellectual property rights, ByteDance has told us that Seedance – aside from its sharper action visuals and stronger sound design – is essentially not all that different from its peers. It was easy to recognize Seedance 2.0 as a slop generator when you look at the most viral clips created with the program, which usually feature A-list celebrities and, of course, copyrighted fictional characters. But the trick to it all is much harder to fathom when you look at that of Chinese director Jia Zhangke Jia Zhangke’s dancea Seedance 2.0 generated short film featuring Zhangke having a debate about the nature of creativity with an AI version of himself.

Jia Zhangke’s dance goes meta as the two characters discuss whether films made with AI should be seen as souped-up copies of human-made works or as a new kind of art form. After one of the Jias reveals himself as an AI copy of the other, the short film follows them both on a Matrix-like journey through various environments meant to showcase AI’s ability to conjure up whatever images a prompter can imagine. Jia Zhangke’s dance unfolds with a fluidity and narrative coherence that you’d struggle to find scrolling through OpenAI’s Sora app. But if you look closely at what’s happening in the background of the short’s busier scenes involving background characters, it’s not hard to see that Seedance 2.0 makes some of the same continuity errors that plague all video generators.

Jia Zhangke’s dance is a shining example of how filmmakers can make things passable with generational AI, provided they’re skilled enough to know how to work around the technology’s limitations. Although the film’s shots are very short, like most AI-generated videos, they are edited together in a way that gives the illusion that they are part of longer takes. And while characters in the distance occasionally clip in and out of view, you can tell that Seedance 2.0 is trying to cover up these errors by covering them with moving objects in the foreground.

Filmmakers can make passable things with gen AI if they know how to work around the technology’s limitations

If there is anything, Jia Zhangke’s dance shows us the extent to which many AI enthusiasts haven’t tried particularly hard to make their creations look like the kind of art that would kick butts in theaters or get people to sign up for a streaming service. ByteDance’s engineers deserve at least some credit for building a model that can mimic real people’s faces with such accuracy. But it appears that that power has to do with the model’s ill-gotten training data, which has gotten ByteDance into so much trouble that the company has halted its plans to release Seedance 2.0’s API to the public.

The only way AI-generated video will look better than it does now is if the companies behind it can prove that their models can create things without having to steal the work of others. Studios like Asteria and companies including Adobe are trying to address that second problem with “IP-safe” models built with appropriately licensed data. But until we see quality work coming out of this new wave of AI programs, it’s going to fizzle out completely.

#Seedance #great #hope #video #generation #shoddy

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *