My experience with GEN AI on a project.
I'm an ex-VFX artist with 15 years in the industry, and I recently dipped back in. I saw some posts on here about Gen AI and people worried about their jobs. I just did a project—I'm not posting it yet as I don't know if it'll be crossing the rules here—but we basically had a production pipeline with about five people, and Gen AI fit in where it did.
It was very interesting to see what it can and can't do. In my opinion, it's going to be added to the pipeline. It has its place, but there were definitely certain things where we think its role is mainly as a renderer. There are certainly things that it could not do, and it needed the help of traditional visual effects tools. They had to be used in concert to get the job done.
Text-to-video absolutely cannot work. It won't work. It's just absolutely impossible. I don't even know why the Gen AI guys actually thought that medium of text-to-video would succeed. But image-to-video and video-to-video? That's definitely where it's at. text to video is so beyond frustrating it is wild. The only reason i can think of it is a thing because it causes multiple retries to get what you want which is more tokens spent for the providers. No determinisim. Which you need 100% in vfx work.
I could later, if there's interest and I get the clear, post the project. It is high quality. Five people, money was spent on it, and I would say there was about 100x cost savings in what we were able to accomplish. But we learned a lot about what the tools can and can't do. When I was working, the project we did would cost between 5-7m to get done. I didn't spend over 60K on this.
Certainly, actors are going to be okay, and of course animatiors, riggers. Voice actors are absolutely going to be okay. I don't even know what they're worrying about. The audio Gen AI stuff is so far off, it is wild. You just cant get nuance direction out of the tools. IMO, all the preproduction jobs are okay.
From my career in visual effects, Uncanny Valley was a big problem. To cross it took billions of dollars and tens of years, decades even, to get across that. Gen AI has huge Uncanny Valley problems, as I think we all realize. So we made sure out project was 'animated' stylistically. We took that animated approach because, you know, we're professionals and we've been through that circus before. We're a team of very experienced professionals.
That is essentially my thoughts on it: it's definitely going to be a renderer, another tool in the toolbox. Throughout my career, I've used many. I'd say like every couple of years I was switching renderers, doing a different renderer. I think that's the same thing here. Gen AI is going to be a renderer.
So Yeah, looking forward to your thoughts. I'd like to share my project because it was a really great learning experience of what was possible and all the different techniques that had to be used to get a shot done. Again, a lot of software had to be used, just like doing visual effects to get something done.
Budget notes:
3 mins. 5 people. VA and composer, storyboard artist, concept artist that could do previz, and gen ai technical artist.
49 500 USD of which 2660 was gen ai credits across various image and video models.