I really don't like the word fix, the movie is not really broken and people can enjoy that look. I think reskin fits better. Ai reskins The polar Express.
Polar Express looked the way it did because it was stylized based on the look of the kid's book, written and illustrated by Chris Van Allsburg. Now one might argue the film didn't perfectly mimic the illustration style, but that's what they were going for.
When the movie came out, it was heavily critized for diving too deep into the uncanny valley, by deliberately choosing CGI to depict real humans. I'd argue that shot significantly over and beside the style the book has.
You're part of that 65% on Rotten Tomatoes, aren't you? haha- I do like the movie overall-- just with the majority who thinks the CGI looks pretty rough around the edges. Both at release, and today.
I saw it when it was released early from Hallmark. My wife worked for Hallmark at the time and we got to see an early screening of it. Love the movie and the nostalgia. Even though the ai looks good, it wouldn't be the same to me. I have a fondness for the look of it and the music is second to none.
@@TheoreticallyMedia why don't you use a framer ai which is open source . It works like taking 2 pictures first frame and end frame and filling it up using the using ai ?
And overdub different languages and adjust the lip sync in the video so that you won't be able to tell whether any movie was originally filmed in your native language.
The problem here is that you are ending up with HUMAN looking actors - not "animated" looking at all. Clearly there was a reason "Polar Express" was made as an animated film and not as live action. I suspect we need something in-between - something that looks far more animated than "real".
@@BopDiva Oh, sure, look cool but, again, not really the point. This doesn't "fix" the problem, it just kind of adds to it by changing the entire character of the film. Sort of like "let's use AI to make the animated Beauty and the Beast to have real people in it". Not sure ANYONE wants that - Disney already did a pretty good job of screwing that one up anyway.
This sort of thing is going to be great for lost media. There are over 90 missing episodes of the original Doctor Who. Because the BBC used to destroy old shows rather than archive them. But over the years they have managed to get copies of the audio of those episodes from fans. If they can use the remaining episodes as reference they might be able to create period-appropriate visuals synced up to the original audio. Right now there are occasional animated recreations which are good but feel different. Providing they can get the relevant clearances.
As a Who fan of many decades, I'd like to see this to....as you say, with permission of course. The versions with stills and sketches with audio are better than nothing but aren't great. The animations are better but the parts I've seen were poorly animated. Using these techniques could be like the animations but much better and as close to the originals as possible.
That would be really interesting. So the audio is out there? Might be a really amazing fan project. Again, there's the clearance issues, but at the same time, if it isn't for profit and just fan created/circulated, I don't think it would be a huge deal. If it gets big enough, it might even rattle the BBC to actually do it. Should talk to @uncanny_harry about this! He's got some BBC connections!
The entire Ray Bradbury Theater series from the late 1980s would be one I'd love to see. What's online to watch is just so-so, looks like a film, to U-matic, to Betacam transfer then turned into a compressed mp4 and uploaded to UA-cam.
@@TheoreticallyMedia Yes, for the missing episodes there is audio for most, if not all of it. For any missing audio, scripts would be available so that could be done with AI too in most cases.
@@TheoreticallyMedia I have to point out an important fact about the missing Doctor Who episodes. Out of the 97 missing episodes, almost all (a whole EIGHTY of them) have what are called "Telesnaps" - these were high quality photos taken of the screen (approximately every 20 seconds). This lets us see EXACTLY what every set and shot looked like. This makes it WAY EASIER for AI to accurately recreate the episode.
I mean this is kinda cool to look at... but it's not a fix. The AI completely misses the little details and overall lacks any real context. I think the only thing that needs fixed is the face's themselves, in my opinion
I LOVE and watch that movie every Christmas and now my younger nieces do the same. I feel like the animation was just fine as it's goal was to match the artistry and tone of the beloved original illustrations!
oh, 100%! I kinda ended up cutting it from this video, but I did have a bit in there where I mentioned that I'm not a fan of Xmas movies, but this one isn't terrible. At least it has some teeth to it. Agreed on the part about going painterly...I was shooting for a photorealistic look here, but I know that wasn't their original intent.
every time i hear someone describe The Polar Express as "uncanny valley" i become more convinced that people don't understand the uncanny valley. it's not "that cartoon looks weird"; it's "that human is... *off* somehow, and i can't figure out exactly why."
Although I'm amazed by what AI can do, in this case it's just replacing one kind of creepiness with another. In 20 more years, today's AI technology will be viewed exactly as Polar Express is today.
I was working on "reskinning" a movie the other day but the lack of character consistency makes it impossible to retain coherency across scenes. There would be so much insanely tedious work around fixing every single cut in a movie to get it to retain the same characters. We're close tho
I'm going to try a few locking seeds tricks-- I mean, you can get close-- but it still takes a BUNCH of re-rolls. I stand by my prediction though: Winter of 26 and this will be a non-issue. Likely earlier.
This seems to be a tooling issue. Some are getting better generating multiple layers, and keeping the seed stable for each layer. It still needs progress, but I don't think we need better AIs, we just need better tools that apply AI more clever. But I am very confident we will see progress there soon.
True-- TECHNICALLY, you could prompt in that direction as well. I know the photorealistic stuff is just a lot more digestable as an example, but-- I mean you could do the whole thing in any style you wanted to. Anime Polar Express? I guess!
There is a movie that I would like to see AI revamp, that is the Last Starfighter, and someday maybe a sequel could be done from it. The movie itself sets up for a sequel that never happened. The actors are too old now and some are deceased, but AI could still do that movie, what do you think?
This is the first video of yours I've seen, and I really like what I see! As a former film student, graduating shortly after the advent of consumer-available generative ai and seeing the shock and fear and vitriol over it, I was never afraid of it, and always saw it as something that could enable far more creativity than ever before. I think it's exciting, and I love seeing explorations of its capabilities like in your video! I have a couple other channels, might play a tad more with ai on those than on this channel, but we'll see. BTW, I love Polar Express. Saw it in theaters as a kid when I was like 9, only got more cynical about it in later years. Yes, the CGI isn't great (but is impressive for what it was trying to do), and the main music theme plays way too often for it to be effective, but even today, it feels like a decent little holiday movie, and it always tended to scratch that Christmas adventure itch. I almost want to make my own Christmas adventure movie. We'll see.
Hey! Congrats on graduating! Former Film Student Kid here (a long time ago, as you can tell by the grey hairs...)-- and yeah, the best advice I can give is to dive into this technology. When I was coming up, it was at the crossover to from Analog to Digital. And all the same arguments we hear today were the same: "It's not ready yet. It's not broadcast quality. It makes everything too easy." I can remember arguments that rendering times for video were too slow, and that you could cut faster on two analog decks. And to some degree-- sure, those old NLE stations could take hours to render out a 30 minute video. But. technology always improves. And what we're looking at here is 100% going to be a big part of the production pipeline. Mind you, not the WHOLE process-- but a big part of it. So, honestly: jump in feet first. You're in a unique position to take this technology and shape it for the future. I'm doing what I can from my side of the street-- taking all my experience on sets and in post, and applying it to these new tools. But, YOU are the generation that will actively take it and make something groundbreaking with it. Just keep an eye on that Skibidi generation right behind you. I don't trust them. And tell them to get off my lawn!
@@TheoreticallyMedia Hahaha, love the comment. The Skibidis will be a challenge to satisfy, for sure. But the movies I grew up with -- Lord of the Rings, Spider-Man, Indiana Jones, Back to the Future -- that's the kind of stuff I want to see more of. The real limitations of AI can be seen in some of the stranger products I've seen like that new AI Coca-Cola ad, but those seams are more noticeable to people like you and me, not the average customer that'll be satisfied and fooled perfectly fine by the illusion. It's like how subtle edits and cuts in films being a millisecond faster than it should be, bothers us but slides unnoticed by most viewers. The things with AI, though, is that NOW is the worse it'll be, and it's already very impressive. It will only get better. So I would rather ride the wave than get crushed by it.
possibly less! I honestly think it'll be doable by next winter, it might just take some elbow grease and a decent machine to pull off. A year after that? It'll be an app. Crazy.
I think it would be really cool to use this tech for restoration. I’m sure AI upscaling will continue to improve, but I could see a multi step process where you upscale old animations from low quality/damaged sources, then “reskin” it in the original art style to clean up artifacts - throw in some frame generation to fix stutter and blending, and you could potentially even remaster shows that only have very low quality copies available
I think the movie would have looked less uncanny if the characters were portrayed by more actors, that's why they all look so weird, they all have the same mocap 😂
If Polar Express had been live action, it would be a classic Christmas Film (even more so than it is for the people that like it). It has this mix of eerie surrealism and whimsy that you find in fairy tales and other classics like Willy Wonka and the Chocolate Factory. The CGI gets more uncanny with each passing year- seeing a remastered version would be a real treat- especially if Zemeckis was actually involved and they put some real time into it.
You have to give credit to Robert Zemeckis for being the pioneer of these mocapped, photoreal movies like Scrooge, Polar Express and Beowulf. The uncanny valley effect is there, but you have to start somewhere
oh, 100%. I think it got cut in the edit, but I had a bit in there where I was basically giving credit where it's due. Polar and Beowulf were stepping stones to where we currently are with MoCap. For better or worse, you don't don't get to Marvel without The Polar Express.
Big props to the animator actually being intrigued by the "What If..."s of using AI with this movie. Indeed I also hope we can make it look more like a living painting than just live action, but the live action is VERY neat.
I'm looking forward to seeing what studios can do with this. This can help polish the final product if they start with a render that's already very good.
AI is amazing and holding many promises for future filming, i can imagine playing a recreated 4K movie from 70th and gives you options to change actors along with their original voices
i think it's a bit of a problem when people will just steal more brand stuff and try and rehash it vs creating something original... Would it be getting the same attention if it didnt use a brand already known?
I kind of hit a similar point towards the end of the video. I’m personally much more interested in folks creating their own worlds and stories with this tech. I mostly view this as an experiment to show what can currently be done. But I think with some elbow grease, you can take your phone, shoot some video, and use this method to come up with a cool short film.
A.I. is like we're seeing images from an alternate dimension. It even has a dream like haze to it. With that in mind, I think the movie that needs this A.I. touch up the most is Tron Legacy.
One thing this is missing that I have not even heard of yet is generating images to match with audio, IE lip syncing during a generation with an audio imput. Or at the very least putting audio over an existing video clip and then just changing the lips of the characters in scene to match the audio. Once you have such powerful tools they would become mainstream in studios.
The big studios already have those tools (they use that to get lipsyncs on digi doubles but we would require much higher quality ai 3D generators that can do it consistently for thousands of frames.
@@kuromiLayfe If they have them now then they have not been employed in any project yet, probably due to lack of consistency. Its why your not seeing any major AI projects being made, it lacks control.
@@kuromiLayfe Ahh your talking about older tech thats been around for years now. Yeah that still requires manual control to get the right effect. but because its all manual you can get the specific effect your going for. What I'm talking about is the ability to create the generations from a single image and audio file. no previous model or animation required. When you can create the dialogue from scratch and the lip sync works. that's the game changer I'm talking about.
I'm not going to say the animation is great, it's not IMO but I do like the film as a whole. It's a nice Xmas film. As another comment says, I'd love to see these techniques used fill in for missing or damaged footage, rather than using it on footage that has no issues. I don't count dead eyes as an issue, it was the film they made and it should stay that way.
I feel like you could set up a comfy UI workflow that takes a list of still images that are just 1:1 frames from every frame of a movie, and then retexture each frame, and then utilize that newly created frame on the workflow as a style reference for the next generation. As long as you go scene by scene, it should be able to keep a pretty consistent theme across them all. Then, run them through an animation workflow to stitch them back together. I'm pretty sure this would melt my PC if I tried it, but with some tinkering I'd bet a decent rig could output a solid amount, maybe even something like a start up that focuses on budget effects for indie movies.
But who will provide the content used to train and update the models? Unless we think popular culture is now frozen and we just regurgitate the past forever? I know Hollywood has a tendency to do that already but should it be the *only* way?
@@echopeakbicycling85 yeah, not like the hundreds, if not a thousand workers who bring movies to life for you to reskin? You'll need a source...even if it'll end up entirely text-generated- copyright laws will eventually catch up to this and put a kibosh on 'your own version of the matrix', 'reskin of Polar Express" etc.
Sounds shit. It's like people taking old animation and adding interpolated frames to make it 60fps because ",mah video games are 60. " It always looks bad and ignores why it was animated the way it was originally. Just a lot of modern day Ted Turners.
I can't understand how people prefer minimax to kling.... minimax is such a wild card, sometimes you'll get a decent shot, most of the times, your subject will morph into an old lady with walker. kling is definitely the best at realistic results.
Was that Roto? I thought that was all full 3d? Man, it's been ages since I saw that-- But you're like, the 4th comment to bring up the FF movie-- I think it's time. for a rewatch!
Fascinating breakdown! It's amazing to see how AI can breathe new life into classic CGI films, especially with tech like Midjourney and Runway. Watching the process made me wonder: could this approach become standard for updating old CGI movies someday?
Very cool idea. I'm looking forward to when we actually start doing a lot more film appropriation. Scenes and exposition can be done very different in all films, and it would change so much about what we like about films. Looking forward to more, thank you Tim.
It looks worse. By a lot. Way more uncanny than the already kind of uncanny Polar Express. No charm or depth either. Polar Express without it's golden charm is like a PBJ without the BPJ- just two bland slices of bread on a plate.
Finally, people are making positive videos and using AI. In a productive manner, instead of just looking for the worst examples to prove some point that will become irrelevant soon...
as someone who grew up watching the polar express when i was young, the uncanny valley style is purely subjective and ive never had any problems with the animation, the ai reskins here take away all the soul from the movie and removed basically anything i liked about it, its completely nonsensical, nothing is consistant, nothing is set in the right time period (the movie is supposed to take place in 1957) it is a beautiful movie which other than the occasionally really weird character design it is an absolute classic and has so much soul and care poured into it. you cant "fix" the polar express, it only needs minor tweaks like the weird animations in some scenes, using ai just removes anything special about the movie.
One major problem with MiniMax is they use copyrighted videos to train their models. A few of my video tests have the CW logo in the lower right hand corner. Can you discuss this?
Im sorry, but, to me, the best way to do it its by finding the original motion caption videos and do motion caption videos for the rest, of course, there is a way to change a character motion with IA too, but i just only know 1 tool for it and its not for video, but, yes, this is cinema, we are really close to see the first IA filtered at least movies, personally, i love superman, and what if we could remake effects for old movies, the smallville series, clark and lois or the new adventures or superman, i recently saw some things and thats why i have it on my mind, but, videos for music songs! and im currently doing my first song with a video, everything with IA. The proccess was this: i asked chatgpt to create a song with something in mind, i changed it to make it better, i mean, the lyrics, its a personal thing, because i believe IAs are good for lyrics creation but for a base idea, after that i changed it the way i wanted, because IAs tend to do the song lyrics too goofy, from my point of view and really, really for 100% of the public, its like a childs song, even if you ask something dark it will give you some of it but with to much possitivity or correctness, english is not my native lenguage neither but, of course i can correct it. After that i start to create songs, for example, if i want to create a song like some "exact song" from the beatles i ask a IA, what are the instruments? whats the tempo? chorus, voices? and maybe i add something like the time from where the song is, and that is how you make the prompt of the song. Once i have a good song, because there can be a lot of songs that are not good enough because those can have errors or the lyrics are too long so the song its too fast, much more than a normal song, or maybe, once you hear it, you understand that certain words need to be changed. After that i maded a portrait with image generators, whatever i imagine and i used that portrait as a base for some IA generated videos, like a commercial song of a guy singing, im currently editing a song because i have like 30 videos of 4-5 seconds each and im trying to match every piece, because, i especified to the video generator that the guy was singing. Ok, for the blank spaces i could add some videos based on the scenario behind of the guy who is playing the piano, its a sad song with hope. After all of this i believe im glad that im just doing a music video because is only 2 minutes long but its to much work, it will take me like 2 days, cheers!
Thanks for the great videos and info! I just started learning about AI recently, and always learn something new from your vids. AI video generation is fun, (and frustrating), stuff. Liked and already subscribed!
Kling’s new lip sync using the original audio to drive it would have been a great addition as part of this work flow, for me it beats live portrait and act 1 for realism and and it works on moving video
Ah man, its been ages since I've seen it-- I should go back-- or perhaps just let it live in perfect FX memory in my head! Sometimes that's for the best!
Very interesting experiment. I wonder what's a wise thing to do while waiting for some new A.I. video features. Like you said, in one or two years time a lot more can be done, or more easily. So what should we do in the meantime preparing for that. Like developing our own characters and stories (cinematic universes, lol.) and scripts... to have all that ready for the moment when technology catches up.
I think short films are the way to go right now. Build up a lot of the production reps and workflow that are more traditionally based, like editing/sound etc. The tools and tech are moving so fast that anything you start working on has a chance of being “outdated” (from a tools perspective) in 3 weeks. So, instead, work on a short that will take a week. Something in the 2 to 3 minute range that you can accomplish, or team up with someone to work with. By the time it “gets there” (and you know that goalpost will move) you’ll have built up a solid audience and portfolio, plus you’ll have established all the process workflows to efficiently manage a larger scale production.
@@TheoreticallyMedia Awesome answer! Thanks Tim. I'm working on a virtual rockstar. Not all pieces are there yet to create what I want, but it's getting closer every day. I totally agree on the short media being the best way to go... music videos in my case. Can't wait to see your next video experiments!
I really don't like the word fix, the movie is not really broken and people can enjoy that look. I think reskin fits better. Ai reskins The polar Express.
True story, but...y'know, the internet.
Accurate titles rarely play well.
Parts of it ARE broken ... like the elves... Creepiest things ever. Unless it was really a Halloween book and the author was going for that.
Sure yet he wont change it cuz of views. Blame the game I guess and not the player
@@TheoreticallyMedia yeah but you're screwing the minds of young people watching this. they will think old CGI=terrible, AI=the solution to fix them
@@davidaustin6962Imperfections are part of filmmaking. AI is not filmmaking
Polar Express looked the way it did because it was stylized based on the look of the kid's book, written and illustrated by Chris Van Allsburg. Now one might argue the film didn't perfectly mimic the illustration style, but that's what they were going for.
When the movie came out, it was heavily critized for diving too deep into the uncanny valley, by deliberately choosing CGI to depict real humans. I'd argue that shot significantly over and beside the style the book has.
Yes but this story deserves animation as good as The Adventures of TinTin
I think I’d prefer the studio just re-render it, keeping the animated look with better eyes.
What you mean, Polar express is a banger movie and the best christmas movie of all time
You're part of that 65% on Rotten Tomatoes, aren't you? haha- I do like the movie overall-- just with the majority who thinks the CGI looks pretty rough around the edges. Both at release, and today.
@ the CGi has the charm the movies these days dont have
@ has the best ost of all time also
I saw it when it was released early from Hallmark. My wife worked for Hallmark at the time and we got to see an early screening of it. Love the movie and the nostalgia. Even though the ai looks good, it wouldn't be the same to me. I have a fondness for the look of it and the music is second to none.
@@TheoreticallyMediaI'd like to see what AI can do with the effects in the Spawn movie
Give it a few years, will be able to reskin shows and movies completely
I'm saying by Winter 26. Probably sooner.
(and actually, that isn't that far!)
@@TheoreticallyMedia why don't you use a framer ai which is open source . It works like taking 2 pictures first frame and end frame and filling it up using the using ai ?
hopefully sooner
I dont even wanna see it happening, its scary
And overdub different languages and adjust the lip sync in the video so that you won't be able to tell whether any movie was originally filmed in your native language.
The problem here is that you are ending up with HUMAN looking actors - not "animated" looking at all. Clearly there was a reason "Polar Express" was made as an animated film and not as live action. I suspect we need something in-between - something that looks far more animated than "real".
I agree with this, but you gotta admit that the realistically looking human characters are absolutely amazing.
@@BopDiva Oh, sure, look cool but, again, not really the point. This doesn't "fix" the problem, it just kind of adds to it by changing the entire character of the film. Sort of like "let's use AI to make the animated Beauty and the Beast to have real people in it". Not sure ANYONE wants that - Disney already did a pretty good job of screwing that one up anyway.
This sort of thing is going to be great for lost media. There are over 90 missing episodes of the original Doctor Who. Because the BBC used to destroy old shows rather than archive them. But over the years they have managed to get copies of the audio of those episodes from fans. If they can use the remaining episodes as reference they might be able to create period-appropriate visuals synced up to the original audio. Right now there are occasional animated recreations which are good but feel different. Providing they can get the relevant clearances.
As a Who fan of many decades, I'd like to see this to....as you say, with permission of course. The versions with stills and sketches with audio are better than nothing but aren't great. The animations are better but the parts I've seen were poorly animated. Using these techniques could be like the animations but much better and as close to the originals as possible.
That would be really interesting. So the audio is out there? Might be a really amazing fan project. Again, there's the clearance issues, but at the same time, if it isn't for profit and just fan created/circulated, I don't think it would be a huge deal. If it gets big enough, it might even rattle the BBC to actually do it.
Should talk to @uncanny_harry about this! He's got some BBC connections!
The entire Ray Bradbury Theater series from the late 1980s would be one I'd love to see. What's online to watch is just so-so, looks like a film, to U-matic, to Betacam transfer then turned into a compressed mp4 and uploaded to UA-cam.
@@TheoreticallyMedia Yes, for the missing episodes there is audio for most, if not all of it. For any missing audio, scripts would be available so that could be done with AI too in most cases.
@@TheoreticallyMedia I have to point out an important fact about the missing Doctor Who episodes. Out of the 97 missing episodes, almost all (a whole EIGHTY of them) have what are called "Telesnaps" - these were high quality photos taken of the screen (approximately every 20 seconds). This lets us see EXACTLY what every set and shot looked like. This makes it WAY EASIER for AI to accurately recreate the episode.
Why would anyone assume they can "fix" someone elses art?!
Tone deaf AI bros do this all the time. + it makes a good UA-cam clickbait
The Beast Christ lady took an ordinary picture of jesus and made it go viral
I mean this is kinda cool to look at... but it's not a fix. The AI completely misses the little details and overall lacks any real context. I think the only thing that needs fixed is the face's themselves, in my opinion
@@DoomguyIsGrinningAtYou.It's so lifeless compared to the original and it an outright spit to the face of the people behind the film
I love the North Pole scenes. They’re eerie, nostalgic, and seemingly endless. Like the backrooms.
I LOVE and watch that movie every Christmas and now my younger nieces do the same. I feel like the animation was just fine as it's goal was to match the artistry and tone of the beloved original illustrations!
oh, 100%! I kinda ended up cutting it from this video, but I did have a bit in there where I mentioned that I'm not a fan of Xmas movies, but this one isn't terrible. At least it has some teeth to it.
Agreed on the part about going painterly...I was shooting for a photorealistic look here, but I know that wasn't their original intent.
@@TheoreticallyMedia TIL Tim is the Grinch that stole Christmas. Bummer. 😕
❄🎄❄🎅
every time i hear someone describe The Polar Express as "uncanny valley" i become more convinced that people don't understand the uncanny valley.
it's not "that cartoon looks weird"; it's "that human is... *off* somehow, and i can't figure out exactly why."
Although I'm amazed by what AI can do, in this case it's just replacing one kind of creepiness with another. In 20 more years, today's AI technology will be viewed exactly as Polar Express is today.
I was working on "reskinning" a movie the other day but the lack of character consistency makes it impossible to retain coherency across scenes. There would be so much insanely tedious work around fixing every single cut in a movie to get it to retain the same characters. We're close tho
I'm going to try a few locking seeds tricks-- I mean, you can get close-- but it still takes a BUNCH of re-rolls. I stand by my prediction though: Winter of 26 and this will be a non-issue.
Likely earlier.
@@TheoreticallyMedia Oh I believe it's coming for sure and I have so many movies I wanna redo
@@iamisobe I'd love to see those.
This seems to be a tooling issue. Some are getting better generating multiple layers, and keeping the seed stable for each layer. It still needs progress, but I don't think we need better AIs, we just need better tools that apply AI more clever.
But I am very confident we will see progress there soon.
I’ll say that the Polar Express doesn’t need a reboot. It is perfect for the way it was made years ago. Let the art live!
when i were a kid the characters in the movie looked very realistic to me i almost thought its real people
I never had a problem with the slightly uncanny style... It works in the films favor especially in the scenes with the ghost.
It looks good as is but I would love to see something that looks like a live version of the whole movie.
I'd say-- 2 years? Probably less...
Same I would love a live action version of polar express and they could maybe also do it with the Rankin bass specials and nightmare before Christmas.
OMG! This was awesome, Tim!!!
Haha, this was 24 hours of obsession. I never want to see The Polar Express again!
"How AI ruined the polar express". There I fixed the title for ya
but it lost that painterly colorful vibe of the book that the original had.
True-- TECHNICALLY, you could prompt in that direction as well. I know the photorealistic stuff is just a lot more digestable as an example, but-- I mean you could do the whole thing in any style you wanted to. Anime Polar Express? I guess!
@@TheoreticallyMediaagreed. AI learning how to do the animation as if it was done with chalk is just a prompt change away
There is a movie that I would like to see AI revamp, that is the Last Starfighter, and someday maybe a sequel could be done from it. The movie itself sets up for a sequel that never happened. The actors are too old now and some are deceased, but AI could still do that movie, what do you think?
Yeah the actors were too old 25 years ago.
@@davidaustin6962 Yup, even the younger brother was old enough to buy Playboy.
@davidaustin6962 how old does one have to in order to be a fighter pilot?
@thomaskalbfus2005 In the galactic armada? Depends on your species.
1:00 Wouldn't the first longest mo-cap movie necessarily just be the first one?
This is the first video of yours I've seen, and I really like what I see! As a former film student, graduating shortly after the advent of consumer-available generative ai and seeing the shock and fear and vitriol over it, I was never afraid of it, and always saw it as something that could enable far more creativity than ever before. I think it's exciting, and I love seeing explorations of its capabilities like in your video! I have a couple other channels, might play a tad more with ai on those than on this channel, but we'll see.
BTW, I love Polar Express. Saw it in theaters as a kid when I was like 9, only got more cynical about it in later years. Yes, the CGI isn't great (but is impressive for what it was trying to do), and the main music theme plays way too often for it to be effective, but even today, it feels like a decent little holiday movie, and it always tended to scratch that Christmas adventure itch. I almost want to make my own Christmas adventure movie. We'll see.
Hey! Congrats on graduating! Former Film Student Kid here (a long time ago, as you can tell by the grey hairs...)-- and yeah, the best advice I can give is to dive into this technology. When I was coming up, it was at the crossover to from Analog to Digital. And all the same arguments we hear today were the same: "It's not ready yet. It's not broadcast quality. It makes everything too easy."
I can remember arguments that rendering times for video were too slow, and that you could cut faster on two analog decks. And to some degree-- sure, those old NLE stations could take hours to render out a 30 minute video.
But. technology always improves. And what we're looking at here is 100% going to be a big part of the production pipeline. Mind you, not the WHOLE process-- but a big part of it.
So, honestly: jump in feet first. You're in a unique position to take this technology and shape it for the future. I'm doing what I can from my side of the street-- taking all my experience on sets and in post, and applying it to these new tools. But, YOU are the generation that will actively take it and make something groundbreaking with it.
Just keep an eye on that Skibidi generation right behind you. I don't trust them. And tell them to get off my lawn!
@@TheoreticallyMedia Hahaha, love the comment. The Skibidis will be a challenge to satisfy, for sure. But the movies I grew up with -- Lord of the Rings, Spider-Man, Indiana Jones, Back to the Future -- that's the kind of stuff I want to see more of.
The real limitations of AI can be seen in some of the stranger products I've seen like that new AI Coca-Cola ad, but those seams are more noticeable to people like you and me, not the average customer that'll be satisfied and fooled perfectly fine by the illusion. It's like how subtle edits and cuts in films being a millisecond faster than it should be, bothers us but slides unnoticed by most viewers.
The things with AI, though, is that NOW is the worse it'll be, and it's already very impressive. It will only get better. So I would rather ride the wave than get crushed by it.
This has taken it beyond the animated look into more realism. I think the animated look is what they were going for.
Correct, it's supposed to look like the book illustrations which had a sort of slightly haunted look. The AI just strips it of all personality.
In 2 years, anyone can reskin/reanimate/retexture any movie, show, or even phone videos for that matter. Crazy times!
possibly less! I honestly think it'll be doable by next winter, it might just take some elbow grease and a decent machine to pull off. A year after that? It'll be an app.
Crazy.
@TheoreticallyMedia Agreed. 2 years might allow kids in rural towns in developing countries make their own movies😆
I think it would be really cool to use this tech for restoration. I’m sure AI upscaling will continue to improve, but I could see a multi step process where you upscale old animations from low quality/damaged sources, then “reskin” it in the original art style to clean up artifacts - throw in some frame generation to fix stutter and blending, and you could potentially even remaster shows that only have very low quality copies available
I think the movie would have looked less uncanny if the characters were portrayed by more actors, that's why they all look so weird, they all have the same mocap 😂
The Polar Express doesn’t even look that bad. I think people exaggerate it too much
This was super interesting! Just subscribed!
From pretty weird and crappy to pretty crappy and weird
haha, snake eating its tail!
1:52 i love how awesome he looks in the reskin ngl
1:43 “all right. Render it. That’s a sweet girl and definitely not nightmare fuel.”
No because the engine will look ugly
If Polar Express had been live action, it would be a classic Christmas Film (even more so than it is for the people that like it). It has this mix of eerie surrealism and whimsy that you find in fairy tales and other classics like Willy Wonka and the Chocolate Factory. The CGI gets more uncanny with each passing year- seeing a remastered version would be a real treat- especially if Zemeckis was actually involved and they put some real time into it.
This is awesome - Retexture is such a powerful tool! Thanks for the feature and shoutout too! 🙌
You have to give credit to Robert Zemeckis for being the pioneer of these mocapped, photoreal movies like Scrooge, Polar Express and Beowulf. The uncanny valley effect is there, but you have to start somewhere
oh, 100%. I think it got cut in the edit, but I had a bit in there where I was basically giving credit where it's due. Polar and Beowulf were stepping stones to where we currently are with MoCap. For better or worse, you don't don't get to Marvel without The Polar Express.
Big props to the animator actually being intrigued by the "What If..."s of using AI with this movie. Indeed I also hope we can make it look more like a living painting than just live action, but the live action is VERY neat.
I think the "magic" of polar express _is_ this style. When I was young, i wanted to see a CGI-movie which looks like CGI.
I'm looking forward to seeing what studios can do with this. This can help polish the final product if they start with a render that's already very good.
I'd like to see new films made with this, too. But, I still can't help imagining old classic films renewed with AI. Incredible.
Ugh. The dollied footage characters look terrible, like surrealistic New Englanders and not like people from Grand Rapids.
Haha, that is an ODDLY specific dig at New Englanders...I'll take it!
I like the polar express art style to be honest
found you recently and this has been great info
AI is amazing and holding many promises for future filming, i can imagine playing a recreated 4K movie from 70th and gives you options to change actors along with their original voices
i think it's a bit of a problem when people will just steal more brand stuff and try and rehash it vs creating something original... Would it be getting the same attention if it didnt use a brand already known?
I kind of hit a similar point towards the end of the video. I’m personally much more interested in folks creating their own worlds and stories with this tech.
I mostly view this as an experiment to show what can currently be done. But I think with some elbow grease, you can take your phone, shoot some video, and use this method to come up with a cool short film.
You could also use Kling to do begining and end frame.
Ahhhh...THAT might be interesting! Hmmm, I'll give that a shot!
runway has input and last frame too
"How to make a good movie look way worse."
There. I fixed your video's title.
AI ruins everything I love 😔
A.I. is like we're seeing images from an alternate dimension. It even has a dream like haze to it. With that in mind, I think the movie that needs this A.I. touch up the most is Tron Legacy.
Would you do the Polar Express : Glacier's Gulch Pitch 1.05x AI Please.
One thing this is missing that I have not even heard of yet is generating images to match with audio, IE lip syncing during a generation with an audio imput. Or at the very least putting audio over an existing video clip and then just changing the lips of the characters in scene to match the audio. Once you have such powerful tools they would become mainstream in studios.
The big studios already have those tools (they use that to get lipsyncs on digi doubles but we would require much higher quality ai 3D generators that can do it consistently for thousands of frames.
@@kuromiLayfe If they have them now then they have not been employed in any project yet, probably due to lack of consistency. Its why your not seeing any major AI projects being made, it lacks control.
@ It is used to sync up speech lipsync on digi-doubles but yea it is usually a handfull of frames per scene
@@kuromiLayfe Ahh your talking about older tech thats been around for years now. Yeah that still requires manual control to get the right effect. but because its all manual you can get the specific effect your going for. What I'm talking about is the ability to create the generations from a single image and audio file. no previous model or animation required. When you can create the dialogue from scratch and the lip sync works. that's the game changer I'm talking about.
I'm not going to say the animation is great, it's not IMO but I do like the film as a whole. It's a nice Xmas film.
As another comment says, I'd love to see these techniques used fill in for missing or damaged footage, rather than using it on footage that has no issues. I don't count dead eyes as an issue, it was the film they made and it should stay that way.
we need this to become a reality when the tech is better. Or just re-make it into a live action all together.
I feel like you could set up a comfy UI workflow that takes a list of still images that are just 1:1 frames from every frame of a movie, and then retexture each frame, and then utilize that newly created frame on the workflow as a style reference for the next generation. As long as you go scene by scene, it should be able to keep a pretty consistent theme across them all. Then, run them through an animation workflow to stitch them back together. I'm pretty sure this would melt my PC if I tried it, but with some tinkering I'd bet a decent rig could output a solid amount, maybe even something like a start up that focuses on budget effects for indie movies.
On the subject of Tom Hanks: Toy Story
The characters' linear acceleration and limited points of movement make it a prime candidate for a remaster.
We're entering an era of personalized entertainment. Quite exciting.
I like "democratized" entertainment. No studios or producers needed.
But who will provide the content used to train and update the models? Unless we think popular culture is now frozen and we just regurgitate the past forever? I know Hollywood has a tendency to do that already but should it be the *only* way?
@@darmok072 you
@@echopeakbicycling85 yeah, not like the hundreds, if not a thousand workers who bring movies to life for you to reskin? You'll need a source...even if it'll end up entirely text-generated- copyright laws will eventually catch up to this and put a kibosh on 'your own version of the matrix', 'reskin of Polar Express" etc.
Sounds shit. It's like people taking old animation and adding interpolated frames to make it 60fps because ",mah video games are 60. " It always looks bad and ignores why it was animated the way it was originally.
Just a lot of modern day Ted Turners.
Streaming services doing this would be a nightmare.
I can't understand how people prefer minimax to kling.... minimax is such a wild card, sometimes you'll get a decent shot, most of the times, your subject will morph into an old lady with walker. kling is definitely the best at realistic results.
absolutely great for the old grainy classics that are pretty much unwatchable on modern screens.
😮 incredible technology
Starship Troopers: Roughnecks, didn't hold up when it was made
Ngl, it might exist a lot better in my mind’s eye. Haven’t gone back to rewatch it, and maybe given your comment, it’s better that way?
How is The Polar Express the first CG movie to be fully rotoscoped? What about Final Fantasy: The Spirits Within?
Was that Roto? I thought that was all full 3d? Man, it's been ages since I saw that-- But you're like, the 4th comment to bring up the FF movie-- I think it's time. for a rewatch!
Saying that AI can “fix” a movie is kinda silly, mainly because there isn’t any real talent and there isn’t any soul.
Fascinating breakdown! It's amazing to see how AI can breathe new life into classic CGI films, especially with tech like Midjourney and Runway. Watching the process made me wonder: could this approach become standard for updating old CGI movies someday?
Getting very close to restoring old/lost Doctor Who episodes.
I just noticed the YES album up on your shelf... nice.
The conductor went from serious mode(original) to happy mode(Artificial Intelligence) 😂
Dragonheart really deserves this treatment, the story is excellent but the CGI wasn't good even when the movie released
If we can recreate the polar express than we also can recreate the goth father or Miami vice. Etc it’s gonna be fun to watch these again.
I’m not sure if you meant to say the Goth-father instead of the Godfather, but now I know what I want to do with AI!
@@TheoreticallyMedia yes that was a typo sorry ! I mean the godfather yes or any other great classic redone with AI
Polar Express ran so Monster House, Christmas Carol, Avatar, and Ready Player One could also run
and Tintin
Great experiment. I think at this stage, MiniMax knows if they add a great lipsync, they have all the AI dudes by the cajones.
Very cool idea. I'm looking forward to when we actually start doing a lot more film appropriation. Scenes and exposition can be done very different in all films, and it would change so much about what we like about films. Looking forward to more, thank you Tim.
Interesting, but I’d say The Polar Express is my favourite Christmas movie still, AI or Not, I still love it to this day!
It looks worse. By a lot.
Way more uncanny than the already kind of uncanny Polar Express. No charm or depth either. Polar Express without it's golden charm is like a PBJ without the BPJ- just two bland slices of bread on a plate.
I would like to see some clever people use AI to extend or explore new angles of scenes from famous movie scenes.
It’s a great movie. It’s a little off looking, but that Hot Chovolate scene was awesome. Makes me want hot chocolate every Christmas.
I'd like to use this technique to make animations of realistic movies!
I feel very lucky that I was never bothered by the animation of this movie.
Finally, people are making positive videos and using AI. In a productive manner, instead of just looking for the worst examples to prove some point that will become irrelevant soon...
whole channel is pretty much dedicated to just that!!
Why not use stills from the movie, use Leonardo's Content Reference, Character reference, etc to create alternative styling and then use Ebsynth?
What about using mid journey to restyle and using tooncrafter to animate between key frames?
I haven't played with Tooncrafter in a bit! I'll swing back over and give it a shot!
as someone who grew up watching the polar express when i was young, the uncanny valley style is purely subjective and ive never had any problems with the animation, the ai reskins here take away all the soul from the movie and removed basically anything i liked about it, its completely nonsensical, nothing is consistant, nothing is set in the right time period (the movie is supposed to take place in 1957) it is a beautiful movie which other than the occasionally really weird character design it is an absolute classic and has so much soul and care poured into it. you cant "fix" the polar express, it only needs minor tweaks like the weird animations in some scenes, using ai just removes anything special about the movie.
I want AI To do the same thing with CGI Fireman Sam, Foodfight and Where The Dead Go To Die.
I don't know, I think it's in the eye of the beholder. I love the polar Express just the way it is. And yes I do use dark mode, on everything...
One major problem with MiniMax is they use copyrighted videos to train their models. A few of my video tests have the CW logo in the lower right hand corner. Can you discuss this?
AI has a ways to go, but so far, it's good for concept design and variations. The retexturing feature seems pretty cool for 3D art as well!
The original animation is great! We can't judge its visual quality by todays standards
Im sorry, but, to me, the best way to do it its by finding the original motion caption videos and do motion caption videos for the rest, of course, there is a way to change a character motion with IA too, but i just only know 1 tool for it and its not for video, but, yes, this is cinema, we are really close to see the first IA filtered at least movies, personally, i love superman, and what if we could remake effects for old movies, the smallville series, clark and lois or the new adventures or superman, i recently saw some things and thats why i have it on my mind, but, videos for music songs! and im currently doing my first song with a video, everything with IA.
The proccess was this: i asked chatgpt to create a song with something in mind, i changed it to make it better, i mean, the lyrics, its a personal thing, because i believe IAs are good for lyrics creation but for a base idea, after that i changed it the way i wanted, because IAs tend to do the song lyrics too goofy, from my point of view and really, really for 100% of the public, its like a childs song, even if you ask something dark it will give you some of it but with to much possitivity or correctness, english is not my native lenguage neither but, of course i can correct it.
After that i start to create songs, for example, if i want to create a song like some "exact song" from the beatles i ask a IA, what are the instruments? whats the tempo? chorus, voices? and maybe i add something like the time from where the song is, and that is how you make the prompt of the song.
Once i have a good song, because there can be a lot of songs that are not good enough because those can have errors or the lyrics are too long so the song its too fast, much more than a normal song, or maybe, once you hear it, you understand that certain words need to be changed.
After that i maded a portrait with image generators, whatever i imagine and i used that portrait as a base for some IA generated videos, like a commercial song of a guy singing, im currently editing a song because i have like 30 videos of 4-5 seconds each and im trying to match every piece, because, i especified to the video generator that the guy was singing. Ok, for the blank spaces i could add some videos based on the scenario behind of the guy who is playing the piano, its a sad song with hope.
After all of this i believe im glad that im just doing a music video because is only 2 minutes long but its to much work, it will take me like 2 days, cheers!
"Fixes" - not even close
They need to do this with Beowulf.
Props for mentioning Roughnecks: Starship Troopers Chronicles ❤🔥💥💥💥
I recently upscaled Roughnecks" Anime to 1080p It looks gorgeous now using Gigapixel.
"This movie looks terrible, we'll show them how to do it right!"
Oh shit that's not very good either.
Lol 😂😂😂😂
Thanks for the great videos and info! I just started learning about AI recently, and always learn something new from your vids. AI video generation is fun, (and frustrating), stuff.
Liked and already subscribed!
We need a full remake of this movie. It really is a good movie its just too bad it looks like it does
Kling’s new lip sync using the original audio to drive it would have been a great addition as part of this work flow, for me it beats live portrait and act 1 for realism and and it works on moving video
I'm hoping for an update to Jumanji. That movie really looks dated.
Ah man, its been ages since I've seen it-- I should go back-- or perhaps just let it live in perfect FX memory in my head! Sometimes that's for the best!
Let´s use this technology to restore incomplete movies, like "Game of Death" from Bruce Lee.
Should probably have AI enhance all of Peter Jackson's LoTR films too, because the CGI of yester year doesn't stack up to today's standards.
Very interesting experiment. I wonder what's a wise thing to do while waiting for some new A.I. video features. Like you said, in one or two years time a lot more can be done, or more easily. So what should we do in the meantime preparing for that. Like developing our own characters and stories (cinematic universes, lol.) and scripts... to have all that ready for the moment when technology catches up.
I think short films are the way to go right now. Build up a lot of the production reps and workflow that are more traditionally based, like editing/sound etc.
The tools and tech are moving so fast that anything you start working on has a chance of being “outdated” (from a tools perspective) in 3 weeks. So, instead, work on a short that will take a week.
Something in the 2 to 3 minute range that you can accomplish, or team up with someone to work with.
By the time it “gets there” (and you know that goalpost will move) you’ll have built up a solid audience and portfolio, plus you’ll have established all the process workflows to efficiently manage a larger scale production.
@@TheoreticallyMedia Awesome answer! Thanks Tim. I'm working on a virtual rockstar. Not all pieces are there yet to create what I want, but it's getting closer every day. I totally agree on the short media being the best way to go... music videos in my case. Can't wait to see your next video experiments!
pretty sure a good portion of the population would say this makes it look worse. there is just a charm about the movie that no robot can replicate.
Fascinating. Is there any way AI can be used to generate a background to replace a green screen with tracker markers?