Bring 2D AI Characters & Scenes To Life with Budget friendly options.
Вставка
- Опубліковано 4 чер 2024
- Find out more about AI Animation, and register as an AI creative for free at aianimation.com
This video is an in-depth tutorial where I take you through all the steps to turn a 2D AI image into an animated scene with a 3D environment and animated character.
With a broad mix of AI tools and some more traditional/digital composition/motion graphic techniques to tie it all together.
------------------- -------------------- ---------------------
Times
0:00 - Awesome-ness
1:06 - Image Gen (Free)
3:39 - Image Gen (Paid)
7:34 - 3D Mesh Gen (Free)
9:03 - Open Mesh in Blender (Free)
12:30 - Open Mesh in After Effects (Paid)
15:07 - Character Animation Overview
16:21 - Sad Talker (Hugging Face) Free/Paid
20:00 - Sad Talker (Discord) Free
22:27 - Lalamu Studio (Free *Demo)
24:54 - Hey Gen (Paid)
26:14 - D-id (Paid)
27:05 - Hey Gen, D-id Comparison
27:41 - Composite, After Effects (Paid)
30:02 - Wrap it up
------------------- -------------------- ---------------------
Tools Used in this Tutorial:
2D Image Generation:
Bing Image Creator (Dall-E 3)
www.bing.com/create
- MidJourney (You could also use Leonardo.ai or Stable Diffusion)
www.midjourney.com/
Adobe Photoshop:
www.adobe.com/uk/creativeclou...
ClipDrop
clipdrop.co/
Adobe Firefly:
www.adobe.com/sensei/generati...
Adobe Express:
www.adobe.com/express/
Zoe Depth:
huggingface.co/spaces/shariqf...
Mesh Magic AI:
Coming soon
- Blender
www.blender.org/
- Adobe After Effects:
www.adobe.com/uk/creativeclou...
- Topaz Labs Video AI: (affiliate link) (for upscaling footage)
topazlabs.com/ref/2271/
- Pika Labs:
www.pika.art/
- Sad Talker (on Hugging Face)
huggingface.co/spaces/vinthon...
- Sad Talker (On Github)
github.com/OpenTalker/SadTalker
- Sad Talker on Discord (Floor33)
/ discord
- Lalamu Studio (demo)
lalamu.studio/
- Hey Gen
app.heygen.com/
- D-id
www.d-id.com/
------------------- -------------------- ---------------------
After Effects Plugins:
None.
------------------- -------------------- --------------------- - Фільми й анімація
I don't say this often. I admire your preparation, practice and professionalism. Thank you!
Thank you very much. I really appreciate the comment. Have an awesome day.
First off - wow on the avatar animation - I never would have noticed!!
there's gotta be a single tool around the corner that does this kinda thing quick and easily out of the box!
Yeah no doubt it's coming. I would probably expect Adobe to do it eventually.
I’ve done frame by frame animation for years. Now blending 2d with 3d I find opens more and more doors for me. I choose traditional animation. Still trying to find a benefit to use AI as a tool to better my skills as an artist/animator. 🎉 I can only support AI if it is a tool to learn from. NOT steal from others.
An amazing time we’re witnessing right now. Great overview video showcasing this magic.
Great video! Also in canvas there is a feature in their new magic tools that can grab the character and you can move it around or even remove it from the frame, quite cool
HOnestly the BEST VIDEO about talking AI videos and also some of them really good quality and REALLY FREE you just SAVED me bro Thank you
Cheers for that. Glad the video helped so much. 👍
Just finished watching. Amazing mate! I’ve been asked to create some talking head scenes that will be placed on screens throughout a ‘winter wonderland visit Santa’ sorta thing over in Ireland and this is gona help me out a lot. So thank you!
Awesome. Honestly really pleased to hear it might be helpful. Good luck with your project.
This is the tutorial i've been looking for for days!! thank you
Cool, glad you like it.
YOURE ROCKING THIS BUDDY! Thank you
My favourite workflow so far! Good job and channel! Keep it up!
Great to hear. Thanks
This is such a great tutorial. Thanks for doing this gonna try this today.
I'm using D-ID and Sad Talker for a while to videos in Portuguese, and in Portuguese the lip-sync is quite a bit different than English, and I'm also having better results using D-ID for sure!! Thanks!!
Thanks for this tutorial! Simple and Amazing....
Actually, DALL·E 3 will give you a 16:9 image if you ask for it in the prompt
Brilliant video, found you via Matt Wolfe video, the Zoe Depth tool is amazing and definitely something we be using to create our videos - Thanks :D
The ZoeDepth is very useful for me. Thanks for sharing that. I think I would only use A.I. generators for environments as I don't like drawing these at all.
Amazing content! Thanks a lot for share the AI knowledge! :D
Omg!! The video of “you” is remarkable!!! I re-watched it so many times, & it’s so spot on! I’m very interested in that once you release it!
That was created using HeyGen, head over to their site and train up your own avatar. It's very cool.
Been waiting for this thank you!!
Thanks, I hope it lives up to the build up.
great video ! thank you for this excellent work !!!
Awesome tutorial! Thank you.
Thank for for this!! Definitely learnt a lot!!
Very useful, thanks for sharing，It is a kind of enjoyment to watch the dance on the dance homepage and create animations over and over again.
Been looking forward to this!!
Cheers, thanks for commenting so so quickly too.
This is such a brilliant tutorial!!!
wow, a great video! I'm so glad I found you
Outstanding ! Thank you very much for sharing with us.
Glad you found it helpful. 👍
This is incredible.
Your videos are very helpful. Good job ! Greetings from Poland.
At 11:22 in blender, instead of correcting the color britness, you can set color rendering in the render tab to simple instead of filmic...
Yep, found that this week. Also connecting it to the emission for the material instead of color solves the brightness issue. Slowly learning my way around Blender (for these videos) after many many years of using Cinema 4D.
@@AIAnimationStudio Welcome into the free world ;)
Amazing. Thanks!
Great breakdown, cant wait to try the Mesh Magic Ai soon :)
Thanks. It's nearly ready for prime time... nearly... ish.
Man, this is hardcore 😮
concise and well presented ..thank you
Thanks very much. Glad you liked the vid 👍
Best video I've seen 🎉🎉
came here because I saw the other one!
damn this is way beyond crazy to me ... love it
Great job
Super impressed with your video 👌👌👌
Thank you sooooo much for this, but also for ALL your works and information as I begin to embark into this community now. Your videos are INCREDIBLE and very inspiring! Much success for Mesh Magic 🥂! One very important question (as I’m sure the first of many to come)…. You mentioned very briefly when it comes to audio 11 labs? I’ve scrolled over and over to see if I missed instructions for that and don’t see it. Have you done a tutorial for that? I need it.
Great workflow - thank you for sharing:)
Cheers, I appreciate the appreciation.👍
OpenAI Sora will change all of this forever.
Yup… it’s all going to get a bit different. 🤯
great thanks ao much!!
Great Tutorial and very useful tips
Cheers!!
Very interesting.. thanks buddy !!
That's super cool
Amazing!
thanks man
master class
Amazing brother
Ai is the new magic of this era , am trying to get in to this new creative world via my channel 🙌
Finally.... well done good job
Ha.. you were quick! ...👏
Hi! I'm really enjoying your tutorials; they're well-crafted, easy to follow, and just the right pace. As a technologist at a museum, I'm adapting your methods for our unique needs. We're using Adobe Character Animator with a puppet rig for flexibility in storytelling, skipping the After Effects final render part. I've tried using a D-ID movie for facial expressions and also creating them in Photoshop, with better results in the latter. Your tutorials have been incredibly helpful, getting me about 70% of the way there. It would be fantastic to see a tutorial on bringing 2D AI characters and scenes to life using Adobe Character Animator. Your expertise would be invaluable for this!
Thanks so much.
Character animator is one I've never spent a tonne of time with, as it never tied in with the kind of client work we do at our studio. Always looked great for character work with live lip sync and things like that.
I have used joysticks n sliders to create similar character rigs in after effects.
Maybe I'll get round to character animator soon.
@@AIAnimationStudio hmmm. Maybe I'm better off learning After Effects ;)
Brilliant. How use poses, generated from a video or taken from a game engine animations, to animate a specific illustrated character? Ideally skeleton plus hands plus face
Thanks for commenting.
A long way to do what you've described would be to create a 3D model of your character illustration using something like CSM (or similar). Then rig and animate with Mixamo. A bit like I did here. ua-cam.com/video/BDWfroMBrZU/v-deo.html
Alternatively there's this approach where bones are used drive the position of a character generated with Stable Diffusion, using OpenPose and Control Net... but I've not explored this yet but looks interesting.
toyxyz.gumroad.com/l/ciojz
Fantastic video. Can you do more workflow videos like these? Other channels mostly focus on a feature and not the end- end product. Thanks so much for such a great explanation.
Thanks for the comment. It's really good to know what aspect of the video is useful and/or different from others. Yeah I'll try and do more full workflows soon. The recent Christmas short breakdown was kinda similar.
Hope to share many more after the holidays now as it's getting busy.. and festive.
@@AIAnimationStudio This may be a silly request but if you could do an AI based workflow to create the visuals and audio for nursery rhymes, that’ll be so cool. It’s the most monetizable content on UA-cam looks like.
Great 😮
Thanks for sharing your incredible content. I'm also committed to educating and inspiring through my channel. Your insights are invaluable.
thanks
Your blender scene is so dark because your only relighting it with ambient light for no reason. You should make the material base color black, plug the color information to the emission color, and turn emission to 1.
ahhh thanks... yes!!.. that makes alot more sense. Just tried it. Every days a school day :-)
Great tip.
Do you happen to know what I have to do to make the camera give me the exact view of the original image?
Right now I have "background grey colored holes" visibile for the areas behind foreground objects. It's okay that these holes in the mesh are there, because the AI couldn't possibly know what's behind those foreground objects, but they shouldn't appear on the render. They should disappear with the right camera parameters for focal length, distance etc. Is there a way to get this right automatically, rather than having to wing it?
@@tsentenari4353I haven’t actually done any stuff like this, I just happen to know Blender. But in theory it’s actually not the focal length, but the position of the camera. Try what happens if you put the camera in the origo (center of the scene). And compensate the crop factor with the focal length.
@@Notacet Thanks! I tried it out, getting the camera to the origin definitely helped. No focal length could make the holes completly disappear, though, so I guess I'll have to render the foregrund objects and the background seperately, and then bring them back together. Thanks for your input!
Hey, @@tsentenari4353 . I think you probably missed the tick box for 'keep occlusion edges' in Zoe Depth, this will fill in the gaps you're seeing by connecting the mesh. 👍 . Here in the video. ua-cam.com/video/7u0FYVPQ5rc/v-deo.htmlsi=67xClmDYNLsm-6CU&t=510
HeyGen generates teeth better i see 😁 🙏👏so cool tnx
It's incredible!
can you create a discord channel?
Not much info, maybe you can suggest groups where I can ask more questions.
Thx u soo many dead ends with other ai video generators
daaamn how are you so good at animation and programming
Ha ha... Thanks.... I think it comes down to just being really really old. 😀
Good morning 3:15
Would love to be able to do this but you lost me at blender. Lol. I wish I had the time to learn this. Fun video and result
:-D ... you can skip the Blender bit if you're familiar with After Effects.
Hi. Thnx for the vid. I'm wondering, do you know of any software that offers good quality talking head avatars for non-human characters?
Just Insane ! Thank to share in final final FREE ALTERNATIVE vs PAID ALTERNATIVE. I would like to know why ain't appear hugging face in your slide ?
And what is your best stack actually for make all of this now you know and masterize all tool. I am curious about that. Thanks
Thanks.
I'm not sure why the duplicated space in hugging face wasn't working for sad talker. Some issue with the code and the version of numPY being used. Hopefully it will be fixed soon. Though the fkoor33 discord approach is actually very good and currently free.
The best stack of tools I'd use would be the paid approach, as I have the tools for work anyway.
So I'd use:
- Midjourney,
- Photoshop,
- Zoe depth (or mesh magic AI :-)
- D-id
- After Effects Beta,
(*But I am enjoying making clips with Pika or Runway ML and then using Lalamu to add lip sync)
Brilliant! I want to create an AI commentator, did you do a video on that?
@AIAnimation can you help build a needed sizzle reel using this AI which will then go to producers at WBTV ?
19:20 SadTalker came up with impressive results if you ask me
Hi from Brazil!
How did you create your Avatar with Heygen?
Can you record a video about this? It was incredible!
Hi over there. It's done using their 'Instant Avatar' on Hey Gen. Once logged in (I think you need a paid account to try it). It lets you create 3 avatars. It's different from simply using a still image. You need to record at least two minutes of yourself talking to the camera, with a pause between sentences and maintain eye contact with the camera at all times. Once done, it takes a little bit of time to train the model, and then you have the avatar where you can either upload your own model or write text, and it will generate audio in the style of your voice.
@@AIAnimationStudio Thanks!
Dall E will do other aspect ratios if prompted to do so
Adobe Express does audio to animation for free.
How do I animate using Blender? Can you do a video that goes over that? Not trying to add another subscription and I would like to learn blender more. Thanks!
Thanks for posting. Yeah I might tackle that more in an upcoming video. I'm definitely more at home in Cinema 4D for 3D work but blender is so good now and price is just right.
Hey, the clips of you talking in video is really made in hey gen? I tried but but i can't get this realistic results and hands moving. Can you please share how did you make it?
Yeah the section of me in the intro is pure Hey Gen. It's the trained avatar option, rather than an animated still portrait photo. It's done by recording at least 2 minutes of video, (filming yourself starring directly into the camera and talking, with pauses. More footage the better I think). You then submit it and it trains the avatar based on the footage it can even generate an AI version of your voice with a text input. The hand movements were pure luck as I just gestured with my hands during the recording.
Let us know when one of these tools includes this whole workflow! This is a lot of steps. I was hoping to make a 30 or 45 minutes animated film, but I don't think the technology (on a budget) is quite there yet, unfortunately.
It'll get there eventually. But yes, right now the array of tools you could use (both paid and open source) are pretty vast. There are various sites and companies trying to be a one-stop shop for everything, i.e. Leonardo AI lets you create images and now videos (via stable Video). RunwayML offers image creation, voices, and video. Adobe will no doubt roll out many more AI tools across its suite of professional apps (and do a good job at it too). Unity, Unreal, Microsoft, Google, Meta and more all in a race to be 'the goto solution'.
I've been playing with RunwayML today and it's pretty useful too. I think it will solve my immediate need for a "virtual storyboard" for a screen play. Thanks.
@@AIAnimationStudio
感谢
谢谢
What type of laptop or desktop are you using and what are the recommended computer specifications?
I'm on a M2 Mac now. But think I was on my M1 mac book when making this video.
@AIAnimationStudio so would you recommend a more graphic intensive system or can you get by with basic system?
Would be nice to be able to do this with-out Adobe...! great video! Less Adobe more AI!
Yea the DID was a bit better, tho heygens mouth looked more natural in animation,
❤
If you had put the final render in at the end, it would have been a good payoff for watching.
Well I'm trying to follow along but I got to be honest it's a bit daunting my question is with programs I keep hearing you have to pay this and pay that how much is the total cost of all these downloads and will my regular computer handle this I was looking at your computer it's a little more higher grade than mine I just have a laptop and it's already saying that it's full will I be able to download all those with my regular little laptop is there classes other than like I know you're teaching online right now kind of but I know it's kind of hard to actually follow along when the print is so small and it may makes me think I may get to get two computers but first of all I can't see the print cuz the print is so small but I do thank you for being English speaking teacher for this AI animation because I have heard others and unfortunately they're either in Arabic or Pakistan or Palestine and it's very hard to follow along with it as well as visual so what would you suggest what kind of degree would I need will I need to get a bachelor's degree how do I go forward with all this information and who do I work with or do I work with it by myself am I a freelancer on my own creations please let me know thank you
Can the same effect be generated in canva instead of after effects ?
Sorry not up to speed with Canva, apart from doing some quick social media graphics with it years ago.
can these things be done in capcut? amazing video 👏🏼🌺
Never had a play around with Capcut. I see it does background removal of video and offers a few AI tools. I'll add it to the list to check it out. 👍
It’s cool that we can animate our characters, but the facial expression and movement feel kind of stiff and unnatural. It lacks life in my opinion. But other than that, it’s definitely interesting. I want to create a character that I can use as an avatar in a “no face” UA-cam channel. I’m camera shy, but I want to create a storytelling channel, and I prefer to use a character to help me with that. I want to reuse the same character in different videos, but I might change the background in each video.
Yeah, definitely agree. It's cool to have these tools but being able to add more life is the goal.
Okay Dear
I followed the blender process and my 3d background came out very distorted how do you fix that in blender?
Not sure without seeing the file/setup as could be a number of things... corrupt file?, viewing angle?. Did it look correct when viewing the output preview in ZoeDepth?..
an you provide us with the image generator prompt
You can try something like.
3D [Describe your character], smiling, sitting at desk, staring directly at camera. In messy bedroom with sci-fi interior. 3D film. cinematic.
If using Midjourney; include:
--ar 16:9
At the end of the prompt so it generates a wide screen image.
thanks alot
@@AIAnimationStudio
I am very interested in your 2d to 3d project. Let me know how I can get involved.
cool. Head over to aianimation.com, ping an email via the contact page info . I think the address is there. Thanks.
Neither D-id nor HeyGen recognize faces of animals. Do you know of any Generative AI tool that support that feature?
I've not see any yet.
You might have some success via Lalamu Studio. As more stylised characters can work. But animal faces might not work.
The only other approach I can think of would be to use a pretty elaborate setup. Animate an Openpose rig and use that to generate new animations. Which is probably lacking the control or quality compared to D-id or Heygen. Plus certainly isn't as straight forward.
any update on Mesh Magic AI
Sorry, it's been put on the back burner while I focused on some other projects. (Other videos, and a Stable Diffusion After Effects Plugin that's going well).
Too many ideas and never enough time.
Mesh Magic is working but I needed to setup a nice smart way to beable to charge for processing, as it was running on Amazons cloud services, and a decent GPU was overly pricey. I'll revisit one day hopefully, as being albe to swap out the texture on a file was handy and the potential for video to 3D mesh was really cool.
WAY SOME OF PEOPLE SAYIN THAT UA-cam IS NOT PAIN FOR KIDS CHANALS ESPECIALY MADE WITH AI
how possıble 10 mınutes anımatıon vıdeo makıng could you show us
Great video man - I really need a animated character like this but I'm too lazy to do it myself lol
Anyone in here like to get paid to design me a charcter like this?
I'm willing to pay good £££ to hire someone that can do something similar to the animation in this video for me. So if anyone is interested in putting their creative/ai knowledge to good use, reply to this comment so we can get in touch and I'll give you more info. FYI - this is not spam and I promise I'm not a bot haha
Nice!!. Hopefully, some users will reach out. You can also find some AI creatives over at aianimation.com . Good luck.
When i import my 3d file into after effects, its just totally black.
Is that with the latest beta? It used to do that for me in an older beta version. Does the file open up ok in blender (to help rule out if it's after effects or the file itself)
@@AIAnimationStudio yes, works great in blender. And yes, im using beta. Maybe my laptop is not powerful enough for beta.
I think though a bigger issue i ran into with this flow so far is that the mesh it created based off my image is very hard to animate in blender without the viewer being able to see distorted mesh. Any tips on that? I guess I need to do less camera movement. I wanted to do a shot through the sky like a birds eye view over a 3d game like cityscape.
Is there any open source project capable of lip sync on 3d character to use it directly on blender or any other 3d software?
Not that I've seen ... But would likewise love to know if there is.
Why not just generate a background and create a character with a green background rather than trying to remove a character from a picture? I know AI tools are way better at removing stuff now, but you don't necessarily need to combine them into 1 image when generating.
The benefit of removing the character from the background, means the lighting (and art style) will match the environment perfectly. Plus trying to prompt a character on a green screen is often a bit hit and miss.
at 6:04 why books are shaking??
Ha... wow, that is weird. Good spot. It was probably a graphics glitch on my computer. Odd though, I've not seen that before.
@@AIAnimationStudio yeah probably. Do you think AI plugins will come to After Effects? lip movement etc.
@@muratyank27 . I expect so. Adobe are doing so much in the AI space and levelling up all their software. It seems like AI voice and lipsync generation must be on the list of future integrations. Plus it'll be interesting to see what MidJourney do when they finally unveil what they're working on in the 3D/Video/Animation space. Bringing all the separate tools together for easy workflows seems inevitable.