SUITLESS Real-Time Face + Body Mocap in UE5 | New VTuber Workflow with the Coil Pro
Вставка
- Опубліковано 30 тра 2024
- Check out this new, very basic workflow using the Rokoko Coil Pro to generate full upper body performance capture with just Smartgloves! This demo is intended as a proof-of-concept - lots more work would be needed to make it production ready - but the possibilities are really exciting!
NOTE
I know the head tracking was reversed on the monkey model - just a result of this being so quick and dirty :) Check out the links below for how to PROPERLY set up the head tracking from LiveLink:
Polywink Tutorial: • Unreal Engine Live Lin...
WoodyDevs Tutorial: • How to vtube using onl...
---
Learn more about Rokoko and stay updated:
Reach us at hi@rokoko.com or the chat on www.rokoko.com
Join our Discord: / discord
Official Facebook Page: / hellorokoko
IG: / hellorokoko
Twitter: / hellorokoko
LinkedIn: / rokoko
0:00 - Intro
1:30 - What is HAPPENING here?
2:58 - Prepping Metahuman Project
4:35 - Setting up Metahuman Blueprint in UE5
9:27 - Setting up Rokoko Coil Pro
14:00 - Unreal LiveLink App
15:00 - RESULT!
19:50 - Overview of Custom Monkey Character Set-Up - Фільми й анімація
I love how the character is animated at like 15fps, that really really does well to mask the awkward little movements that come out of mocap
Most important highlight of this is comfort. Wearing a suit for longer periods of time even if its straps can get uncomfortable after a couple of hours. With this you can basically go for as long as many batteries you have on hand.
This is pretty dope and opens up the doors for alot of people to get into VTubing easier. I wonder if we can make this work with iClone as well since I know we have the option to live capture just the gloves there 🤙🏿🤙🏿
I was wondering the same thing. My Ultraleap just doesn't cut it in 3D.
You can also link the head movement from the iphone live link to one of the spine bones to add addition movement to the character :)
Ah perfect!
This is really cool, ive tried to do this variois ways in the past, using things like the leap motion, but this will really make vtubing to a high standard affordable for a lot of people :)
so cool
I'd love to see a similar tutorial for Unity - lots of vtubers tend to use Unity (probably due to vrchat) and honestly wearing the smartsuit can get very...hot xD
I’m sure it would roughly similar - the nice thing is all the IK stuff for the arms is being done in Rokoko studio - so as long as you can livestream to your character from studio it should work pretty much the same. The only thing is whether or not there’s a similar face capture solution that provides neck/head movement in Unity 👀
I don't see the link to the 'resources folder' for this project in the description. Am I missing something?
Can this be set up for Motion LIVE and a webcam. (I can already link up the webcam to Motion LIVE I'm just looking for a better option to replace my Ultraleap/LeapMOTION.) I don't use iPhone if that makes a difference. Thanks!
Good. Now I'd like for you to livestream yourself using those gloves (powered on) for over an hour - no edits or cut.
The metahuman shoulder is weird !!!
Super weird for sure - but that could be fixed by just not having the mocap affect the shoulders so they stay in place, or by a bunch of other methods - I'm definitely not a very knowledgeable UE dev, but hopefully this might inspire people who know what they're doing more haha :)
Hugggggeeeeee