I Can't Tell it's AI! - AI Animator BLOWS my mind!
Вставка
- Опубліковано 2 чер 2024
- Toon Crafter is the beginning of AI based animation moving forward. These demos are AMAZING!
▼ Link(s) From Today’s Video:
Demo Page: doubiiu.github.io/projects/To...
Tooncrafter Github: github.com/ToonCrafter/ToonCr...
Try ToonCrafter: huggingface.co/spaces/Doubiiu...
Proper prompter's tweet: / 1797255628202918208
► MattVidPro Discord: / discord
► Follow Me on Twitter: / mattvidpro
► Buy me a Coffee! buymeacoffee.com/mattvidpro
-------------------------------------------------
▼ Extra Links of Interest:
AI LINKS MASTER LIST: www.futurepedia.io/
General AI Playlist: • General MattVidPro AI ...
AI I use to edit videos: www.descript.com/?lmref=nA4fDg
Instagram: mattvidpro
Tiktok: tiktok.com/@mattvidpro
Second Channel: / @matt_pie
Let's work together!
- For brand & sponsorship inquiries: tally.so/r/3xdz4E
- For all other business inquiries: mattvidpro@smoothmedia.co
Thanks for watching Matt Video Productions! I make all sorts of videos here on UA-cam! Technology, Tutorials, and Reviews! Enjoy Your stay here, and subscribe!
All Suggestions, Thoughts And Comments Are Greatly Appreciated… Because I Actually Read Them. - Наука та технологія
The example with paint is what sold me completely. This could be a renaissance of indie amateurish web animation ala "flash animations" and Newgrounds, where it's not about quality of drawings and animation but just the content, the ideas, people doing something creative and a lot of entertainment many people can feel encouraged to try their hands at
^^^You got it! A Golden Age is coming. We've been locked out from actual art by the corps, now we get exactly what you are describing I think...I just couldn't put my finger on it but you did pretty well with the comparison.
Ideas are the plates for the spaghetti.
I am not going to watch them.
"I like rusty spoons" IYKYK
@@artyshreds6274 ?
I hope this AI could advance enough to be able to assist in finishing unfinished scenes from the thief and the cobbler
Or berserk
That'd be pretty cool actually
When high fidelity enough. Also, Watership down, as intended.
Gotta throw in some cameos from Aladdin.
I tried it ,tweaked the settings, sadly it doesn't understand the complexity (masterpiece) of the images. But it works great with coloring
Miyazaki would die from heart attack hearing about this tool
There is a very intense video of him shaming some Ghibli artists, more of an attack on AI, for the wunky souless results they showed him. He's very disappointed.
@@empire4627I mean it could be useful. I dont think it can ever replace humans. But maybe for small things like simple headturns. The problem is A.I always lacks character in animation as they usually go from A to B and ignore a lot of principals
@@CantHandleThis6 "Can't ever replace humans"? You don't know about Moore's Law and the rate at which AI research is progressing? Can't now doesn't mean it can't in the future.
@@isekaidev7803 Even if A.I can get that good it can never produce intent. It would need an actual personality and consciousness to do that. At that point its just human. You ever listen to a song or watch a show and feel the passion for a project through the screen. A.I can never truly replicate a Demon Slayer Tanjiro vs Rui moment, or a Deku vs Muscular moment animation wise (granted the latter was more a performance thing) the inherent passion will be missing. It might be able to make cool shit but it will always be better to give scenes like that to humans and maybe just let A.I reduce some workload.
@@CantHandleThis6 That is exactly what is happening now. Many companies have fired a huge number of employees and making the rest use AI. The articles don't say what AI, but I assume it's things like ChatGPT. AI isn't going to replace humans as it's going to be doing all the work without any humans. It's going to be replacing 1/10, then 1/6, then 1/5, then 1/4 th etc people and the rest will use AI to do more work per person using AI.
Just putting it here, doraemon had a gadget that turned comic book to anime episodes, so this is yet another 22nd century gadget coming to life, can't wait to meet real life doraemon
Couple more papers down the line and we'll be generating our own anime. The next few years are gonna be wild.
at first it'll shave 90% of the time off of creating an episode. then in a few years it'll just spit out an infinite amount of episodes of anything you want. hard to imagine. but its happening.
@@Chris_Hruskabetter copyright if you have a good idea 😂
@@Chris_Hruska What would you make?
@@imanigordon6803 You can't copyright an idea, only the execution of that idea. But I think copyright will eventually be kicked into the dustbin of history by AI anyway, and good riddance. Its primary outcome these days is maintaining corporate strangleholds over arrays of IPs they bought out, none of which have any relation to any of the people involved, which is a total perversion of its original purpose (to protect creators).
The hardest part will still be writing good content, AI can help with that too but can't do it alone.
This is amazing for the indie animation industry
Imagine how many fewer frames they’ll have to draw
Imagine how many artists they will no longer need
@@ghost-user559 technology replaces jobs, this has been a thing for centuries
@@neomicryo Sure, that’s very true. But what do you think happens when we have passed the point where humans are needed any longer for anything? Do you think the people in charge, who say the world is over populated will need this many?
@@ghost-user559dude why are thinking all this now? When it happen then you have to show concerns it's gonna take centuries to make all things automated, specially passionless job slavery
@@ghost-user559 you got it.
This means that with a good enough workflow, an entire 20-minutes anime could be made in 11 hours and be fully automated, using GPT-4, Controlnet, and this tool
No, GPT is too nerfed for anime
As an animator myself, no this wouldn't actually speed up the workflow too much, research animation principles such as follow through, lines of action, squash and stretch. An AI can anticipate certain actions which is an animation principle in itself, but it's very rudimentary. So what this would be good for is speeding up the in-between process which will help key animators. Even SORA AI can't come close to actual animation at the moment despite creating fluid movement.
@@AbyssalSodagive it a few more months
@@MarcoCholo-iz9js I've been hearing "give it a few months" for over a year now. It's still going to be a while before it can do half decent animation
Not a chance. Unless you want to give away creative control in “make a cool 20 min animation “
People don’t get how much time gets into planning scripting etc.
For more like family guy and such yes maybe it could be cut down a lot tho
The MAPPA employees will finally be able to go home and sleep!
Nah, they just fire the other employees and keep two guys to do all the extra work
they will out of work and no longer afford a home.
as if
they will just make the animation even more incredible with all bells and whistles at all time - when this can be done in 1080p that is
they will just make more anime but still work the same time
This is legit amazing. Video title is definitely accurate for this one. Thanks for showing us these demos!
This is incredible. I’d love to try this. The “Caustic” effects with the water is amazing
This is pretty impressive. I hope AI video evolves quickly. I'm still frustrated that I can't even prompt SVD yet.
I've tried this, and it works ok. I just don't know how to get the sketch in. What do you upload as the sketch? A video, or a animated gif?
The colorising animation is the most useful part of this at this point. I'd like to see it on random not anime start to finish tho
this current version is absolutely amazing for animation clean up and pacing..
seen many high budget animations where they could have done certain scenes way better if they interpolated the frames better instead if just cutting away.
If you have your old flipbooks, you might try putting sequences from them into it.
Anyway, pretty interesting. Many years ago, maybe 1985 or 86, one of the guys on the Amiga Compuserve forum was working hard to build a tweener. Of course, it wasn't neural net based, just morphing by connecting points in one image to points in another and linearly pulling them together over time, but it stayed with me how very much that guy really wanted something like this. Don't know what happened to him, but he would certainly be blown away.
Now I really want it to be the program creator. Haha
finally some advancements in ai video, moving at a similar pace to image generation at the start
Why do all the example images come from an already existing anime?
Ikr?
because it's trained on existing data....no originality
Because the people who want to use this tool don't draw
Looking good Matt!
This along with the news that SD3 is to be released 12th of this month made my day! I hope this works well on 3D art too, I'd love trying it on mine, it's a lot of work animating all the pretty details like hair, clothing and vegetation.
Thanks Matt,I was wondering what I was going to do this weekend.
Can't wait till I am able to remake one punch man season 2 in season 1 quality
what was wrong with season 2? the animation wasn't as good but still pretty high quality
@@thomblueart8448 Certain scenes were alright, but the majority was subpar. Lack of fluidity, incorrect proportions, still frames.
@@chineseducksauce9085 I might be a bit biased just for Garou being in it I guess!
@@thomblueart8448 happens
@@thomblueart8448Season 2 trash
17:45 that is pippo from doraemon, i miss her to this day, she sactificed her race to save earth 😭😭😭
Finally someone can fix "when they fire all the animators" Twinkle Nora
Finally, but is basic
I imagine if you fed two frames of Twinkle Nora to this thing the entire server would burst into flames.
Well, if this AI can animate Sailor Mercury, then I’ll be worried for my studio. Until then, it’s just another semi-useful tool.
Someone needs to do that xD
@@Wonders_of_Reality it seems it's only good at interpolation. That is just a part of animation... not the whole. Animators not getting fired yet 😂 .
But on the bright side, just take 10 friends animators + ai and make your own company. It depends how you look at it... On one side you get fired and depressed, on the other side you become business owner and rich.
Lol what most people dont realize it how good your input has to be , means you better be an artist of some form. You can see even on the glowing piece flick how inbetween frames would be necessary. For all true artists and new artists this tool is dope. For the rest no its not sora
just use ai to generate a good image
You just have to combine this with existing AIs
@@jaredf6205 ok create me 2 coherent pics that would result in a clean animation like a character where the arm bends around an elbow in 90 degree , just wit ai gen tools
I agree.
Is this really that much better than animatediff in stable diffusion and morphing between a number of frames using ComfyUI?
I couldn't use the rough drawings to guide the animation, did anyone figure out how to do this?
Hey Matt, is there a good way to convert footage to sketches to use this tech effectively by starting with a green screen video filmed on anything? would love to see you try to connect these technologies end to end!
Cant wait till No Game No Life s2 is animated by AI
Haha, this!
All of my favourite miscellaneous models have always been covered 1st by you.
This is why this is the best channel about ai.
i have the feeling the training data they used is the exact animation that they used as examples. still impressive that it’s so much better than the others but i want to see how it does with frames it hasn’t been trained on.
Tested it on some kaiyu 8 frames, it was pretty good
@@Samuel-wl4fwoh that’s sick. i didn’t finish the whole video to see it was open source and he even did his own example which turned out pretty decent 😂
really cool. could see this theoretically making animators' lives easier. (e.g. only animating 1/3rd as many frames.) but not sure it can work with their workflow (e.g. what if they need fine edits). But we're getting real close to useful stuff.
Welp rest in peace in-between frame animators ✌️
Nice one Matt! Thank you ✨🙏✨
Most interesting. May I ask if this (or another tool) may be capable of creating and maintaining the very same set of character’s skills/proportions/look and use them in different environments? Meaning: can I use the characters I’ve created (both with drawing and digital art) for creating an entire illustrated/animated story? Thanks for sharing this video
I've been waiting for this, finally no more drawing for long durations of time
definitely cool stuff, can't wait for it to get better.
The main issue im noticing with this is the timing of things, it seems very robotic. Dont get me wrong, extremely impressive stuff, but once again, the computer isnt accounting for the 12 principles of animation.
10:19 it is caustics, not causatics, Matt :) Keep up the great job, mate.
maybe it'll save an anime studio some time animating between keyframes, but I'm not sure how helpful this would be, you kinda have to first be able to draw 2 high quality anime frames
The quality of animation is better than in some modern anime, so it should be useful.
Also, how will this work if you are using to speed things up on the workload but the end product cannot be licenced and distributed internationally due to indivdiual copyright laws - AI generated content cannot be copyright for example?
I think this is only the case if it's only generated using AI.
The coloring frames option is not out there..
I'd like to see somebody this with consistent characters and controlnet to try and do the whole animation process
What's amazing is the coherence through all the frames
Okay, I'm sold totally installing this later tonight!
it's not there yet, but inbetweening might get processed by AI at some point.. it's already a terribly paid job anyway with terrible conditions..
Kinda feels like 2D animation will somehow be like 3D one if this keeps up, just drawing key poses and let AI do the rest.. interesting technology honestly, but don't expect to make a whole anime or something without skill or talent, this tool won't change that
So what if you add more frames? Or is that not possible?
You can input how many frames you want in between the keyframes you draw
@@jajankenanims5409 oh hell yea thank you. 🙏🏾. I currently don't have my computer to test it. But man I can't wait to get back to work.
I waited my entire life for this kind of technology and it's finally here!
I feel bad for Japanese animators that earn a living interpolating anime. They were already underpaid and exploited, soon they're gonna be out of a job entirely
Well this tool will help them not kill them lmao
@@MilkGlue-xg5vj no it wont it will make most of them not get hired, and human creativity will continue to decline more and more
On one hand you people complain about those animators being exploited with low wage, now you complain about them being out of jobs. You keep blaming AI, instead of blaming the real cause here, the greedy corporations ? :D So blind and ignorant.
@@mine.moment no one thinks the concept and technology of ai is bad but the entire gen ai system is based off of an illegal data scraping of artists work, and it was all done by greedy corporations, and people are feeding money to these literal criminals by paying for AI. AI should assist artists not replace them, and so far, the industry is just laying off animators and artists. Its the fact that companies will not use AI to make life easier for artists, but instead just pay them even less, or lay off most of them, while keeping one or two to continue to work at an exploitative wage just so they can squeeze more money out in a shorter time.
@@mine.moment well of course i would complain about the low wage and i would complain about the AI cuz their both two different problems and both bad. And the greedy corporations are bad, and so is AI.
😂bruh every time I come to ur page…I leave feeling like I got new super powers 😂
I knew this was coming but damn. P.S. since midjourney has character stuff now. you could concievably use MJ to generate frames, and this to generate animation. If need be you could maybe repose with SD with openpose.
now this is what I like to see
The guys over at Corridor Crew are going to plotz when they see this.
Hah! So I'm not the only one who thought this, huh? 😄
Idk, last time I saw them talking about AI it was very, very negative.
@@mycelia_owIn what sense?
@@mycelia_ow What video was this?
As an animation film producer, I am thrilled by this. Because of the hefty price of good animation, I keep waiting for do it yourself animation like this to get better. Its moving fast and getting closer to being able to make it cheaper to do a full animated film.
4:12 The person actually always keeps their left foot in front. That's an issue that also happens in Sora.
Damn, i tried the demo and it worked great !
Finally One punch man seasons can come out once a year instead of once every 5 years...
Lol
@@NoahtheGameplayerSeason 2 is a non canon event
It's exactly what I would expect - mind not blown
Definitely a tool that will help save a lot of time and money for in-betweening work in anime studios. It could definitely help cut cost, and perhaps unfortunately, a few in between animators might lose their jobs.
nice animation tutorial 😊
When done with real video (in a year or so) then one can take video scene beginning and end and just regenerate badly recorded moments. Or do every even and have a new movie. Incredible stuff being released.
I want to use this to go and make my manga have "still animations" like in a harry potter photo... I think that would be cool. While also colorizing the pages.
This is legit amazing
I actually see this as a good thing for anime companies as it could help artists be more efficient once this technology becomes better and better.
Being more efficient means they need less animators…
@@Silks-no, same number of animators but ton lot of project handled by each of them 💀
@@andra9694 wat
@@Silks-they can use this as a tool
@@Silks-It means that the businesses will either replace them or expect more of them and overwork them twice as much 😂
I really dont want these creative ai stuff what I want is a medical renaissance thats what the world needs cures for cancer tinnitus foreskin regeneration all that stuff thats what I want so badly and pray to all the universe we see that soon
16:04 I agree with you that this tool would be so much better if we could input more images. Just using 2 images is not enough, because the animated clip turns out too short.
I see a lot of potential with this not for finished products, but rather auto-generated in-betweens and rough drafts to speed up the animation process. Weirdness doesn't matter then because you can just erase it or draw over it like they were guidelines.
I don't think this AI will be able to replace animators.Interpolation is already done in the 3d software.
Colorisarion looks really great
It’s about the fifth video I see showing tooncrafter and what bugs me is that so far only the samples provided on the project website are actually impressive. I tried it, and I’m also underwhelmed by the results. They’re not bad and much better than any other kind of in betweening frame interpolation I’ve seen, but still they have the same flaw that every AI tool I know have : giving you ALMOST what you want, but not quite, which makes you have to do a lot of post processing for being really usable. Sometimes it’s not that much and all in all it’s faster, but sometimes the time you gain is not significant. And the problem is that there’s no way to tell in advance if it’s going to do a good job or totally mess up, it’s all trial and error, which is not a really sustainable workflow.
I’ll be able to make my own Tokyo Ghoul remake if we don’t get one from a big studio. 🥳
So much excitement for a potential threat to people's dream jobs. It's unreal.
It's something I seriously want to get into but it needs a huge system though. maybe 64gb of ram minimum and maybe 24gb video card minimum. It's seriously system resource-heavy.
I'm going to try it with midjourney photo realistic images and see what I get.
You would think that animation would be the most successful as it can absorb the uncanny valley of A.I. video generations far easier and the bonus of being able to digitally correct the animations generated. Getting closer.
Holy shit. We are about to step into an age of cinema
Age of slop which nobody cared enough about to even make?
@@paulsheldon8838 nah, so many passion projects are ruined or made not as hood because the person can’t animate and has to come to a middle ground with the people who make it. Part of the reason people say mangas and books are better. We also will have way more of them made so by default there will be more good ones. And when the visuals are all really good for all the animes/shows people will have to really focus on the story and characters to make their show worthwhile
Oh hey, what about that one anime with like 5 frames total. Nora I think? Maybe it can fully animate it.
Make a anime movie will come from really expensive and lots of working hours to some hours....
Finally.
I like that it’s open source.
The future of animation is uncertain.... but cool!
I can't wait to see something like this for Unreal Engine.
just a few years and then I will animate one punch crossover with every piece of fiction ever existed and defeat every villain with one punch
I love that small companies and individuals will be gaining tools to flesh out their projects and make awesome things.
I don’t think more traditional animation will die, but I will lament its fading to a sub-sect of people and companies that appreciate legacy animation. If I want the little guy to have tools to flesh out awesome things, I have to accept the larger companies using them too when they could make something wholly unique and impressive without it.
I have been predicting an Indie Animation Boom eversince SD 1.5 began to be Fine Tuned to hell and back.
Now with SD3 Medium releasing to the public on the 12th and this.
Let's just say the mad lads working with Confy UI are going to have one fun summer ahead of them.
Especially once ControlNet gets updated for the new architecture.
Amazing.
Having more input keyframes should be pretty straightforward: just run it multiple times, pairwise. So run it on frames 0 and 1, then on frames 1 and 2, then 2 and 3, etc.
2D animators getting replaced for the second time in history is crazy
Almost nobody is studying the consequences of all this in the med/long term.
@@nigermant6347
Consequences for whom? People are not one group.
Music for animation: Udio beta
Sound effects: Audiogen V2
Video for animation: Tooncrafter
😅😊
It would be great if it allowed more than one keyframe, but it's still cool!
Fucking finally😂 the world is mine. I cant waot until this is legit
i never expected to see zunda horizon here (17:47)
You could use your pictures and chatgpt and make yourself a toon image of those picture of yourself using chatgpt and then put it back in this program and see what it does.
Incredible! but-
The amount of work it would take to fix that many frames you might as well just do it yourself to begin with. Is it amazing? YES. but we're not 100% there yet. For animatic type workflow and prototyping scenes this is off the charts useful though;
Testing emotional feedback to certain timings etc. without actually having to invest countless hours- crazy
Even though the currently generated results are not fully polished, I expect that AI will replace "in between" animators in the future.
Animation programs already had automatic inbetweening interpolation for years.
Obviously? @@Skitskl33
This isn't that though
@@jumpy2783 It's pretty similar to most interpolation tech we've seen over the years, including what is in modern tvs with motion smoothing.
It also seems to have alot of the same issues like artifacting and deforming the image.
Completely crazy!!
3d practically killed hand drawn animation and most artists are against AI but I think this may be the tool to bring back "traditional" animation back into the spotlight, so disney can go make shitty hand drawn movies instead of another generic pixar flic and 2d animators will be once again employed to do what they spent all those years learning to do.
hope that will help to finish Berserk and animate vagabond
Pls try Abstract motion graphics.
it will kill the magic of hand drawing animation
instant manga to anime mayhaps
Wish that it could also move only 1 image