FREE-AUTOMATIC Facial MoCap Shapekeys (Blender)
Вставка
- Опубліковано 1 сер 2024
- Creating 52 shape-keys can be a lot of work, but there is a way for you to automatically generate the shape-keys after you've rigged the face - Here is my FREE automatic shape-key workflow-!!
ArtStation Shapekey Generator Link:
www.artstation.com/marketplac...
Facial MoCap Promo Project:
• BTSmash - Butter (3D A...
ArKit Apple iphone Shape-Key Official Documentation:
developer.apple.com/documenta...
Faceit Blender Addon:
• Faceit Process - Jason...
If you enjoyed this video, I have a small 1$ Member perk available for anyone who would like to help us save up for better tools to improve the content here like Hardware, Software, Environments, Animations, and special effects :)
(Just click the "Join" Button next to "Like"!)
/ royalskies
Twitter at: / theroyalskies
Blender RIGGING & ANIMATION SERIES:
• Blender 2.82 : Charact...
Intro To Unity Programming FULL Series:
• Introduction To Game P...
Anime Shader Tutorial Series: • Blender Anime Shading ...
Blender MODELING SERIES: • ENTIRE Low Poly Blende...
If you're a gamer, please check out my new game on steam! It took over 3 years to create and has thousands of hours and heart put into it :)
store.steampowered.com/app/65...
Or if you just wanna play something casual, you can download my phone game to test your reaction time and see if you get into the top 10!
play.google.com/store/apps/de...
As always, thank you so much for watching, please have a fantastic day, and see you around!
- Royal Skies -
#3D#Animation#SmashBros
-------------------------------
Polywink UA-cam Channel:
• 3D Facial Animation Ri...
Copyright Disclaimer: Under Section 107 of the Copyright Act 1976, allowance is made for "fair use" for purposes such as criticism, comment, news reporting, teaching, scholarship, and research. Fair use is a use permitted by copyright statute that might otherwise be infringing. Non-profit, educational or personal use tips the balance in favor of fair use.
![FaceIt Tutorial 1: Facial Mocap & ShapeKeys addon [Blender]](http://i.ytimg.com/vi/KQ32KRYq6RA/mqdefault.jpg)
Free ArtStation download link can be found here:
www.artstation.com/marketplace/p/pkRk3/blender-facial-mocap-blendshape-generator?
Also, sometimes it's easier to just combine everything on the face like the eyes, brows, tongue, etc into a single object > and just apply the shape keys once instead of separately for each mesh. That's actually how I usually work with the face (being a single mesh at the end). And, as I said before, I'm not the best python programmer, but the code does work lol - If anyone has a cleaner or more efficient way to execute the code, definitely feel free to share it down below :)
thanks for this man i really appreciate the work you have done here. im really new to all but thanks to you and a discord im a part of it been making this easier for me. and i want to port tuff in to sfm. i know i mite some back talk about that but its what i know best and prefer to use. but blender it a great to get what i want into make sfm. thanks again for this i hope it can be useful to me in the future if i need human shape keys. i'll give you a tip once i get the money and when i download it. but rn i try to learn how flexes work in sfm.
I'd like to ask permission to send you an unlisted video of a similar workflow that I use that I think you could do a video on. It's to speed up modular armor workflows, does work with facial morphs (but only if the vertex group the shape key is applied to is on the base character) and saves me having to apply shape keys to a hundred different pieces of armor and accessories.
Reason I'd like to send it: I don't like video editing. So if I made a video on the topic myself, it'd be long and rambly. But if I sent it to you, I figure you could boil it down to 60 seconds or less and help a lot of people make modular characters with modular equipment in a lot less time.
If you're okay with this, I'll just email it to the email listed on your account.
PS- My workflow is similar to some I've learned from you, but with an extra step that saves a lot of extra time in the long run.
great
@@DawnBriarDev unreal engine? I used fskeletal mesh merge
how do you combine everything?, for me when I join everything the jaws and eyes freeze?
Mate your hard work is truly hidden from the blender community. I've been trying to promote your videos to other people in discord servers because that's the least I can do. Thank you for all these tutorials and assets
bruh i just set ik for legs, arms in something like 7 mn both with this guy videos
@@S9universe I learned ik from this guy and got it in the first try, never saw any ik tutorials again
@@ferdraun quick and efficient
Yeah i make it a lot too haha always in my list of what channel to watch
true story :0)
just want to thank you for all your work man. I've been here since you made the how to make a run cycle animation loop and since then, I've been dropping by. your tips and tricks have been a monumental help to me over the years!
Wow that's incredibly kind of you to give this code to us for free! Thank you so much for all the work you put into helping others.
Years later and your still doing great things for the community, I'll never forget how much u helped me when I first started my journey!
hey on the question how do you download the addon? I drag ged the zip file but nothing seemed to happen.
unpack the zip :D
@@jesusaulloa1653
You are way too kind and the community is awesome.
Much gratitude for this and all your superb "get to the point" content. I always look for your fast and informative tuts when searching for blender info. Respect and appreciation to you my friend!
You are the undisputed champion of the 2 minute master class. ✊✊✊ Thanks 🙏🍷🎩🎩🎩
You're so awesome. The accessibility will save so much time for everyone looking into this.
I use FaceIt for all my animations and find it really fast and easy once you have figured it out. Plus the developer is very responsive.
You are really awesome, i found your chanel like 2 months ago and you help me a lot with your videos, i'm just a teenager and i have no job or other thing to gain money, but when i have the money i will support you because you are the greatest
That's the beauty of free software, you receive what u plant.. thx a lot royal
I love your work and videos so god damn much. Always referencing them and sharing them. So fricken helpful.
Thank you for all these tutorials and assets!! When my situation improves I will make the deserved contribution.
You always have the answer that I'm looking for. Thanks man!
this is super useful and generous of you. Thanks!
Thank you a whole lot! This is too sweet I genuinely appreciate it with all my heart
Big shout-out to you for this great piece of work I ill try it soon and test .. Keep doing great stuffs mate ❤️
So cool! So nice of you the Squad!
Absolute Legend
Excellent! Thanks matey!
Thanks for sharing! You're a legend!
Damn you really did it! Thank you!
UR a Life-saver, Royal!
you've earned yourself a sub, thank you so much you actual angel on earth
Wow!!! Thanks a lot, mate!
WOW Thanks so much for sharing this!!!
Thanks for the tutorial!
Animating the shapekeys was always very hard for me, but now I'm using the ReveRig Addon and is Amazing 😍
(I've tried Faceit but it's too expensive and the results were not satisfactory)
With ReveRig I can simply transfer my animation recorded from the iPhone and improve it to make it more cartoonish.
Really suggested 😉
Tested: This still works for 3.0, but not 3.5 or 3.4
Thank you, RS!
Wow thank you all the things you doooooo.
This is so FANTASTC, I might get a FANTASTIC t-shirt!
You are the best!!!
Dude...thank you so much!!!!!
wow , thanks for sharing this you are a legend
big fan of (noooot aa praaaaablem)
Ive said it before, I'll say it again, Sir, you are a Legend
WOW THIS IS A W!!!!!
Thanks man❤❤❤
You're awesome bro. Have a fantastic day.
White people literally copy EVERYTHING we do, except for being last rate citizens around the globe ...when you have been lynched, castrated , baby's cut out of woman's belly's , being ripped apart by two horses , a dozen of us , tied to a tree , and burned a live , with a crowd cheering on, ....then you can also call each other bro ... but if you all new what respect was,, you wouldn't dare to tell us that it was such a ling time ago !
Much much appreciated.👊👊
You are a wonder. Thank you very much.
This is awesome ✌✌✌
wow. . always top ...thx
you guys are cooler than blender guru
I LOVE THISSSSSSS
sori late been busy but not busy enough to not leave a like a watch the ads till the end
This man need more views
Yes
Thank you, Sir, 🙏🌺😇
Wowww
Goddamn dude you’re a gem! Gonna try this thing out now!
cool vid I always wonder
You definitely need to spend some more time with FaceIt. It blew me off my seat and I use it all the time now.
That said, free addons that do something very similar are very much appreciated ;)
Daaaaaaaaang
Support squad are heroes~
wow
Sir, may you be known as friend to both man and beast.
I'm confused after this process how do you get the shape keys back on your rigged model in the video you make a new copy of the head with al the shape keys "and your done but how do i mix body capture data with facial capture data if the head is separate and now has no bones?
i expanded your code into a plugin, it creates a action/animation on the selected rig then adds pose markers to the frames to be filled out, when you've done that, select all the objects you wana generate shape keys for.............nnnnnnnnn your done?
i left a github link on my profile if anyone wants to try it or just look at the py file
OoooOOOhhh... Sounds legit...
nice
work
I tried it and it only generates the last tongue out shape key and not the others
Edited* I know the likelihood that this comment will get a response is small, but I am going to try anyway! I have been searching for a way for over a month at month to transfer multiple object Shape key data from Blender to unreal Engine. It seems like you are only able to transfer transfer the shape key data for a mesh If all of the shape key data Happens within a single object in the match. For example, I am not able to transfer transfer the shapekey data from an eyelid end and I lash at the same time or at all For the same mesh. I would gladly pay somebody a substantial amount of money to help me figure this out. Sorry for the weird text, This is a voice text. - Update - I think I have figured this out and now I just want to tell you that you are a beautiful person and I have learned so much from you. Thank you Royal Skies!!
Update - I am not sure still that I have solved it. Damnit. So close @-@
Unfortunately, most of my experience is in Unity, so I've never experienced this problem and have no useful advice about fixing it :( I do plan to invest time into learning Unreal 5 later on in the year (Probably for work purposes) - And, I have a feeling I will encounter this problem when that time comes, but by then you'll probably know more about it than I do. Have you tried going on to some forums?? There have to be other people who have had the same problem I presume - All I can say, is when I do move to Unreal, you can be sure I'll share everything I learn here on the channel, in the meanwhile, I wish you the best of luck -
@@TheRoyalSkies I have watched so many of your tutorials I completely read this comment in your voice! I think I just figured it out. I would be so happy to share what I know with you as well if you have any questions and want to put some of these things in your videos. Lmk! I think this is a particularly poorly documented Blender to UE4 process - when I say I've been exploring this for tutorials for months, I literally mean months! I think I just cracked it - I would be so happy to share something with you especially because I would not be where I am without you! Thank you again so much, and let me know if you run into this issue and are ready for the solution - I would be so happy and thrilled to help and see this FINALLY covered in a tutorial and I think there are so many Blender users banging there heads out there over this! Really appreciate the reply - thank you so much for everything that you do.
bro you have to merge them together
That's great. So how do we rig the face?
One day, I'll watch a vid from this channel 1 sec after upload
One Gay ...we know ... prinzie
You should make a git repo for your code so people can fork & optimize code! Id love to help if you do
I could not figure out what mistake i am doing. Whenever i run the script in text editor, it says Indentation Error: unexpected incident (line 1)
If I understand it right, this script only rearrange the manually posed keyframes into shapekeys. if I pose keyframe 1 to shape "tongue out", it would be a problem as it should be "eyeBlinkLeft". it does not automate the shape posing portion which usually is the most time consuming part.
This is true - It does assume you have correctly posed the face each frame. But, the most important thing about automating this part of the workflow is it eliminates the potential mistakes someone can make misspelling or creating the shape keys in the wrong order. I can't tell you how many times I accidentally missed 1 or 2 keys, or had 1 key slightly misspelled and not work. Trust me, you want to automate that part of the job away if possible -
@@TheRoyalSkies Only thing you need to do is what I do, ...something probably nobody has ever heard of .... get the list of Apple or from where ever, and ... here it is : Copy Paste .... I gave away my secret .... I hope all your ass crawlers are not going to call me a legend now !
My god, for a second, I thought u were Ian Hubert. You have exactly the same voice. :D
Which Plugin do u use in maya for shape key u have press that white outline head icon then shape keys created pls tell me
Hi, have you seen the new shape key addon by the blender foundation? I does some if the things you showed here, and would probably work better.
Any chance you’ve got a link to that?
Woow
Soo, the shapekey generation you showing in Maya, any way to get my hands on this heavenly piece of zeroes and ones? 🥵
I hope you are reading this: every time I run the script, it make all the shapes keys in the new mesh be whatever the shape key is on the frame that was selected when the script was run, i.e. if it was one frame 1, all 52 shape keys produce the shape of eyeBlinkLeft, and if I'm on frame 4 they all have the shape of eyeLookOutLeft, and so on. (btw, all the names of the shape keys are correct, just not the shape). Have you run into this problem?
I have that problem too
@@fergadelics be sure to be on frame 1 or add bpy.context.scene.frame_set(1) to the beginning of the script
Okay, so thoughts:
I'm doing a lot with vrm's at the moment. Might not be your use case. But vrm's are consumed through programs like VseeFace and Vup. There are others, but those two are the most reliable for my use case, which happens to be vtubing.... until you start layering shape keys from different layers with the same name.
Of course, from here, any sensible person would tell me, "Lynn, but this is why we have blendshape avatars in Unity," and you would be correct.
But, Vup in particular flattens your shape keys, assuming that they're all individually named and unique.
Got two EyeWideLeft where one of the layers is the eyelash only? Well, that's great, because the system will just... randomly pick the first one it sees, which is done alphabetically. So Brow.EyeWideLeft is going to be seen before Face.EyeWideLeft, and your model is toast.
There's a language barrier when you're getting support from these programs, and something this specific will not translate for someone who's google translating your discord messages into Chinese. Trust me, I've been there. Nothing against anyone, but the consumer level language translation tools we have today SUCK.
Anyway, I figured I needed a workaround, because learning Chinese is hard.
And my workaround involved prepending the names of each shape key with the area of the face they exist in, or the name of the object, and the ones with the main names are not actually the individual components, but in fact mixes. Ta da! You're done. No need to learn Chinese to explain why the software you want to use doesn't work to the people that wrote it. You're done.
But wait. See, because you're a reasonable person, you're sitting here and saying, "But Lynn, it's just one program that works this way! You crazy girl, you." And, if you didn't know the specifics of this, you would be right. But, you would be wrong because the source of the problem is a unity library someone wrote for Unity and posted on Github that translates FBX files and reads the shape keys as blendshapes, which I still contend are and should be considered different things (I'm writing a tretis on mixes, lol).
Anyway, that library is used all over the place, it's in a lot of mocap software, not just VUP. And, as far as I can tell, it's one of only a handful of Unity libraries that cover the topic.
If you're using Unity-based software to do your vtubing... you've got anywhere between a 30% and 50% chance of seeing it again!
So, the solution is explicitly named shape keys for the layers they're on. Prefixed. Mixed to make the final ones, which you DO name this way. And bingo bango, you're compatible with everything. Everywhere. There's no program on earth that won't read your shape keys properly and shower them with love. And you can actually get some sleep at night, rather than finding ever crazier ways to rig your models (like I have :) ).
Hope this helps somebody
Oh, also, if the blendshapes for blender plugin worked the way the maya plugin does, I would happily give you $50 for it. I'm still a hobbyist, but I spend enough time in blendshape hell, that any respit that actually works, so I can get back to the fun part, is welcome
Also, seriously, I can help with the python part. I'm offering
@@PriestessOfDadaLearned something new here : .... ''Bingo Bango'' ! You are on point with your remarks, I feel that it's not a coincidence that all those ''great'' Mocap software programs work all on their individual little Islands! By the time we (me) get a reasonable animation sequence going , the artistic feel of it, is completely gone by the technical side of it , converters, OTT priced half working solutions, and loads of tutorials that show only part of the add ons workflow, expecting you (me) to be smart, LMAO.... Why doesn't he respond , when you offer to help ???
I believe your hope that you've helped has borne fruit for me :D thank you so much, this comment taught me more than the last week of researching/learning about blendshapes.
That's awesome! I wish I have access to Facemocap but sadly I have an Android/Samsung phone and the app is exclusive to iPhone... It is odd because my Samsung phone is on the higher end and it seems to have some sort of AR function (Samsung only) but the data can't be transferred into the computer like Facemocap can. I am still getting used to Maya, I've been using Blender more so it is more familiar. I personally make sure the face is in one mesh, and then I create shape keys by myself for animation.
EDIT: What about iClone's i7/i8 Facewire Profile? It appears it can do recorded facial motion tracking using only the webcam. I have a Logitech C920, 1080HD 30 FPS. Not sure how good it is but would that work? I want to be able to download animation data as an FBX and then upload into any programs I use (Blender, Unreal Engine 4, Maya, etc.).
A good Samsung app for facial mocap is blendartrack by cg tinker. The back facing camera works as a 3d camera tracker and the front camera works for facial mocap.
@@zanghproductions2559 Thank you so much for the suggestion! I will try it out. I am using Blender mainly for my Senior year animation project because I am more familiar with it than Maya, still learning with Maya. I hope this app will work well with my Samsung Galaxy Note 20 Ultra (my current phone).
@@paolaanimator I have the same phone. It works pretty well in it.
@@zanghproductions2559 I just tested it out and I am happy with the results! I have a light stand so it could help out with better tracking. My only concern is that I made a custom rig from following Royal Skie's rigging series because I went with the anime eye (floating pupils) aesthetic. Rigify didn't work out for me. I wonder if this can still work with my custom made rig. I really want Rigify to work for me but I find it has too many bones and tricky to fit into a feminine face/anime face. I'm currently working on my Senior year animation project and Blender is my main program since I used Blender a long time and more familiar/used to it. I am still getting used to Maya and other programs.
I followed cg tinker Discord so I hope they could provide answers.
i cant use it in Blender 4, did I miss something?
Does this only work for the 52 frames or can we adjust if we have more expressions?
any idea why my face explodes when I hit shift d and move over?
Any way we can get the Maya version? :)
Could you upgrade the license to 'Indie License'? Now the 'Standard License' mean limit up to 2,000 sales or 20,000 views.
i don´t know why but it doesn´t work properly for me, i have a full body mesh and when i move the shape keys values it slowly affects different parts of the body but not the face
I can't add it to the Library, can somebody explain how i can download this asset?!
does someone know any good way or video that teaches you how to rig ?
Is the max 52 shape keys or can more shape keys be added?
but how do I keyframe face deformation
Hello
Does it work for vroids?
Can you use it with livelink after it?
So, this seems really easy and helpful for what im trying to archive, unfortunately I barely know what I am doing. I have a rigged face that I rigged with autorig pro. It looks different than in your video, but the rig works fine (i guess?). I did exactly as said in the video, unfortunately all of my created blendshapes only move my left eyebrow up slightly, and i cant really figure out where i went wrong here. Does anyone have any advice for me? (using blender 3.0.0)
did you ever figure this out?>
@@KUREHA3D no, unfortunately i didn't get this tool to be useful for me. But instead i just made the blendshapes myself. It's alright from the hardness level, especially if you have a jawbone and eye bones.
@@yaradotexe thanks for the reply, good on you for figuring it out yourself. best of luck on your creation journey in 2023 and onward!!!
@@KUREHA3D thank you! I wish the same for you, and hopefully you find a way to make the ARKit Blendshapes in a way that works well for you!
can i do something like this. take a face trhat already has the blendshapes for example a custom vroid face, cut out the eye and mouth sections on it and my model, fit the meshes of both together and keep the blendshapes working? this would be amazing for creating anime characters and vroids license would allow u to still sell your models any way u want
If you still have to shape every frame the blendshape needed for aninmation, what's the point ???
exactly what im thinking
will this work on vroid models ?
Hello. This application works in Cinema 4D?
so it doesn't create automatically the blendshapes, it transfers the poses as shapekeys?
Hmmm doesn’t blender just have an option to make a shape key from an armature pose?
It does, but only one pose at a time. The python script does all 52 poses at a time
so when I have the duplicated object with the shape keys, what about my rigged character? I want to use the rigged character with the shape keys to stream mocap into unreal engine. I cant have 2 seperate characters for that
did you figure this out?
@@KUREHA3D i got frustrated and bought the FACEIT addon for 100 bucks
I have this question too.
@@jungerhansmann6608 whats your experience now with FaceIt? was it worth it for Unreal Engine workflow? were you able to use it for humanoid characters that have complex facial expressions with facial mocap data?
Can I do it if I download the attached file?
I'm confused..Are we suppose to make the 52 shape keys ourselves before running the script? Doesn't that defeat the purpose of auto-generating the shape keys on a duplicate after we just made them?
There's no getting around needing to pose your rig/face bones, since you're the only person who knows where they're positioned and how you want them to look. But after the first time, you could reuse that rig forever using this method. Not to mention if you have multiple versions of the same character, or need to go back and redo them, it's all ready and waiting. I would say this met the purpose most creators would need, but not the fully automatic hopes and dreams of those people being forced to use Blender as a means to an end, just doing one model for personal use as a streamer or VR user.
Had the same question, was expecting this to auto generate all 52.
Hey, This didn't work? I did everything step by step, however when I scroll a shapekey to 1, it doesn't do anything?? Am I missing something? Why isn't any animation working?
2:04
def generate_scene_frame_set(frames: int) -> dict:
for _ in range(frames):
bpy.ops.object.join_shapes()
bpy.data.scenes['Scene'].frame_set(bpy.data.scenes['Scene'].frame_current + 1)
def set_names(frames, *args) -> list:
for i in range(frames):
args[i] = input(f'Set the name for frame {i}
')
return [name for name in args]
def set_frames() -> int:
return input('Set the number of frames you would like to generate.
')
bruh what
@@daton3630 I'm a "python guy."
@@infinianti your programmer?!!! wanna earn money with my great idea!? i get 80% then you get 20% of money!
that was a joke please don't kill me( i have a wife and children)
do u offer private help in rigs n such? I need help with this batman armor rig and have no idea where to start
be glad
I don't even know how to access the source code once I have downloaded the zip file. Do I need to extract it or install it like an add on? I am a complete novice so this is very foreign to me. It just seems easier to learn to rig a face from scatch then try and use your code.
there are no face and rigg in zip :(
i have no idea how to do it. How my skeleton should look like? Do i need all bones? What if my model is not a humanoid? I got bones for head/eyes/eyelips/uppermouth/lowermouth, they got some skinning done and that workflow doesnt work for me. Crucial! What do i exacly need to use this automation? "rigged face" yeah but what that exacly means? Rigged humanoid face with standard bones? U said it can be custom rig so i thought it can be done on my ball-ish creature but it s not so i got 2 options: Do better rig or leave this workflow.
Cool vids tho but it s hard to follow for a newbie
Bones don't matter, they are doing a join shape (copy) of the current mesh shape during each frame of the animation to a shape key, one by one. You could animate any property that affects the mesh polygon/vert positions, and it would still work.
@@pismodude2 did some experiments meanwhile and it works ;p Thanks for answer tho!