AI Image to Animated 3D Model Workflow Using Stable Diffusion, DALLE and Mixamo| Day 260 Blender
Вставка
- Опубліковано 29 сер 2024
- Source Files and Resources Used: [linktr.ee/sara...](linktr.ee/sara...)
Click on Blender| Search for Day in Title
Disclaimer: Haven’t figured out a way to apply the original texture (at least in all views).
00:00
0:12 - 0:34 Generating Image using PlaygroundAI
0:35- 0:39 Playground AI Result
0:37- 1:09 Using DALLE to finish the rest of the image.
1:09 DALLE Result
1:11- 1:29 Using ICON to turn image into 3D Model
1:30-1:42 Uploading OBJ file (3D model) into Blender just to see what it looks like
1:43- 2:10 Using Mixamo to animate the 3D model
2:11-2:24 Mixamo Results
2:25- 2:57 Uploading model into Blender
2:58 Small Blender Project
![Turn your AI images into CG movies [Workflow Video]](http://i.ytimg.com/vi/-rfNOBGIiyk/mqdefault.jpg)
quick and concise, thank you
I've been looking for an image to 3D model using Ai website for a while now glad I finally found a good free one
Righttt. I was looking for something like this too and when I found it I was like finallyyy 🙌😭
Omg this was great. Loved the clarity, pacing and no bs format, reminds me of Miziziziz's tutorials, instant sub!
Aw thank you! I really appreciate it. Glad you liked it :)
I was waiting for your to apply the original texture on the 3d model.
Layering all these AIs together is soooo cool! Great editing btw!
Thank you!
I recommend installing Stable diffusion, I have the SD krita plugin as well. Not only can you get WAYYY more specific with what you want, you can also do a style transfer render (not 100% sure how to do that yet) and end up with a near perfect rotoscope. You want your slightly shitty AI 3d model to rather be an anime girl? Litteraly a few clicks away!
where can i learn more about this workflow
@@JJ-vp3bd ua-cam.com/video/eCSIqJ5cmJs/v-deo.htmlsi=hShhW-MvBHC-2lke
This is for the Krita extension
ua-cam.com/video/JPP6DGL5ykQ/v-deo.htmlsi=zC1ssET-lFUn-_q0
For style transfers
I love you thank u
haha thanks for watching!
I was doing some research in order to achieve a similar workflow applied to videogame asset production. thanks so much for this video!
your'e welcome!! glad you found it useful :)
Amazing and useful!
Thank you!
That was really helpful, thank you.
Subscribed.
Thanks!
Nicely done 🤯
Thank you!!
Cool. Have you tried adding the texture from the AI generated image to the 3D model?
Thank you.
I didn’t do it for this video bc I only know how to do it in a very ‘unfinished’ way but I just posted a video on how to at least get the front texture of the model working in Blender: ua-cam.com/video/sN2fAoMJYnA/v-deo.html
Wow! this is so cool!😁
Thanks!
Amazing!!
Thanks!! 💓
That's very interesting workflow 😱😱😱
thanks!
YOOOO THIS IS SO COOL
hahahah yesss it’s really exciting!!
You have a very lovely voice.
what a great workflow video!! very helpful!! thank u
thanks!!
Very cool, thanks for the video!
Thanks!
Thanks for sharing!
Thanks for watching!
AAAmazing, great video.
Thank you!!!
wow
I'm always thankful. There are clothing photos from various angles. Is there a way to naturally change the clothes of the character in the video?
Pretty cool workflow 👍
Thank you :))
this is so cool, thanks for sharing!
Thanks, glad you liked it!
Great video, thanks 👍
Thank you for watching 😁
Image to 3d model! Fuuuuuuutuuure
Yesss
Wow so good. Please also make and video on 2D image of person to 3D character
Helpful, thanks!
Thank you for watching!
Cool vid. Never heard of the image to 3d model before.
Though, I’m curious how well it’d do on a more unusual image. And I’ll also say that the rigging was less than great. Better to touch it up manually.
Thank you! I've tried it a lot and turns out that I get the best results when the model is (1) facing towards the camera, (2) not having anything obstructing their view like a purse or tables, (3) has a clear separation between skin and cloth and (4) has a clear separation of the body parts (hence the T-pose). The last one is mostly to get it working for Mixamo. If I'm just going to generate the model with ICON then condition 4 is not absolutely necessary. Oh and yes I agree on your point about the rigging, I honestly just hid them for the render haha.
Yo! Use the image to projection map the texture onto the model and you're golden :D
Like this? ua-cam.com/video/sN2fAoMJYnA/v-deo.html
I got the front view to work only tho lol. I’m trying to see if it’s possible to project two different images (like front view and back view) onto the same mesh. I just need to get a back view of the model so working on that.
@@saraimarte Accidental nightmare fuel, but otherwise nicely done :D
@@MarkArandjus hahaha thank you
that website seems to not work anymore, any substitute for it?
Is this still the best method from 2d to 3d in your opinion ?
Not anymore. 1. The authors of the code came up with new versions. 2. I figured out a way to add complete front and back textures (using blender tho) I just haven’t gotten a chance to make a video yet 😭
Hahaha this is good. It will become a legit workflow soon tho.
Hope so!
Wow, are you talking and working in Blender like that at the same time, at that speed without saying uhm? I could not do that, even with post-recording my voice. How did you get your video so fluent?
The voice was post recorded but you’re right. I tend to record everything in one go so when you hear me speaking it’s mostly without edits in between. It took a lot of tries for the rest of the parts but for Blender I had already memorized the things I needed to click bc I do blender a lot lol.
But to actually answer your question, I legit start the whole recording over when I make a mistake and I start over. I make mistakes a lot so by the time you hear the final result I would have memorized what I had to say which may explain why it’s so fast haha. I really dislike editing sound so it’s either I get it 100% or I start from scratch lol. Bad habit I know 😭
@@saraimarte Wow, thanks for sharing your workflow. I'll give that a try, maybe, because I guess I am on the lazy side. Good stuff!
@@saraimarte Yes, in one go sounds much better. I often hear tutorials where they cut a lot, but then it's harder to follow. I think one flow, as you did, is best.
@@Newmediasupply thank you 😊
ousam
thank you :))
Hey great video!. I was wondering what's the link to the ICON website?. This is such a helpful video!. Thank you for sharing!.
Thank you!
Here it is: huggingface.co/spaces/Yuliang/ICON
It is also in the description box. You just have to go to the Blender Log for Day 260. It contains all other resources I used as well!
@@saraimarte Your Welcome!. I'm trying to setup icon with Google colab but I can't connect the two because I'm trying to run the cell to enter my username and password and it's not working. Can you do a tutorial on how you setup ICON with hugging face and google colab?. Thank you.
I'm not sure if I can use CPU with ICON on Google colab. And what are the requirements to do this setup?.
@@saraimarte I have a Intel CPU,can my setup work with hugging face to Google colab to ICON?. I really want to get my 2d characters to 3D with ICON. I have lots of character 2d art on playgroundai and I would really like to use ICON and make them 3D and upload them to mixamo.
Thank you.
@@dproductionsai Hi, you don’t have to use the Google Colab. If you use the hugging face space you can do everything in three clicks.
Here is the link: huggingface.co/spaces/Yuliang/ICON
You don’t need any GPU. Hugging face does all the heavy lifting for you. That’s that good thing about it. I used all web based tools that allow anyone to follow this procedure regardless of the GPU they have.
To find all other resources: maple-lungfish-d35.notion.site/Day-260-49e0a11b481b4e21988ddc736ca9c209
@@dproductionsai Also, I wouldn’t recommend using the Google colab for this because it gave me an error the last time I tried it on there lol.
The only thing I would recommend is doing some audio balancing. You are way too quiet compared to the intro and outro music. This makes it hard to hear you on phones where the volume is capped.
I definitely need to try this workflow out though. Could be a cool way to pick up some basic Blender skills
Yes, thanks. I completely agree with you. I recently learned how to use audio gain in premier but when I do it, it sounds so loud on my desktop yet it sounds low on my phone- and now I realize why lol- bc volume is capped.
I tried going up a little bit more in the one I just posted: ua-cam.com/video/sN2fAoMJYnA/v-deo.html
what is the link to upload the image into 3d?
Here it is: huggingface.co/spaces/Yuliang/ICON
I think the face cuts are not really clear right? I mean we can't make it out who the person is..
Yup I agree with you, its not perfect. Hopefully finer models come out soon. I haven't seen any that could get to this level in terms of the body however. I believe there is a program that just does heads really well (but not bodies) and it can only be used in Blender. An idea would be generating the body with ICON and then cutting off the head and then replacing it with the head from the other program. I don't remember what the program is called from the top of my head but lmk if you're interested and i'll look it up later.
이게뭐야 ㅇㅅㅇ.. 얼굴 이미지로 3d 개체까지 다 만든다는건가
Yes I am, all thanks to these AI’s that make this possible!
@@saraimarte wow 인크레더블
Hi thank you for sharing this. Can you please also share the text description that you used to generate DallE image (photograph of highly detailed girl, looking forward etc....) As I am not able to generate the image such that I can try with mixamo. Thank you. Your work made me subscribe your channel.. keep sharing this.
Yes, ofcourse! (And thank you, I’m glad you enjoyed the video! )
Here it is: photograph of a highly detailed symmetrical full body of a beautiful woman standing in a t-pose, wearing white T-shirt and black leggings, short sleek black hair, global illumination, radiant light, eyes open, eyes looking forward, front view, plain white background, hands visible, face facing the camera, standing straight, arms spread outwards, wearing black shoes
I also added the prompt to the link in the description. You can click on Blender and find Day 260 and all the links to the resources can be found there.
You might benefit from a Remesh. Think the retopo phase of photogrammetry post.
Are you on LinkedIn?
Thanks for the advice, I’ll try it!
And yes I am but I’m not active there.
Thank you so much but what about the textures??? I will be hoping for a new tutorial ASAP
Textures are not possible with ICON. So I’ve been doing them manually in Blender. Up until now I’ve only been able to figure out how to apply the image as the texture for the model but only the front view works as I only have the front view image of the model.
Already posted the tutorial! ua-cam.com/video/sN2fAoMJYnA/v-deo.html
Great video. moving forward please increase the volume of your speaking section and decrease the music. went deff when you stopped talking and your end card music kicked in.
Ty!! and so sorry 😭😭
Hello, I can't use ICON. After registering, he prompts: Runtime error
Space not ready. Reason: Error, exitCode: 1, message: None.
Can you help me solve this problem? I am willing to pay for study
I believe it's down currently down. You can try PIFu: colab.research.google.com/drive/11z58bl3meSzo6kFqkahMa35G5jmh2Wgt#scrollTo=eUEXAvcvkVYV There is a tutorial there if you get stuck but what I do is go to "Runtime" at the top and click "Run all" but before that go to the "Render Result" section and put a hashtag infront of line 1 and remove the hashtag from line 8. It's going to take a while to run but it should work. Let me know if you need me to clarify anything!
@@saraimarte Can you take the time to help me remotely? I can pay for your time. thank you very much... I'm not very good at these websites😂
when i try do this all wrong model is not correct
Try PIFu: colab.research.google.com/drive/11z58bl3meSzo6kFqkahMa35G5jmh2Wgt#scrollTo=eUEXAvcvkVYV. If the result is not what you want then maybe it's the image that's not the best for it.
Thanks a lot. What about the textures?
That would be the next step. I'm trying to find another tool I could use for that since ICON only gives the base model without textures. Blender is an option but only allows me to apply the texture for the front of the model only because that is the only 'view' I have available. I heard substance painter can do it but it costs $$. When I figure it out i''ll post a video.
Hi! I Just posted how to at least apply the texture to the front of the model: ua-cam.com/video/sN2fAoMJYnA/v-deo.html
honey you should normalize your sound channel
Ik lol, working on it haha