Quest Pro Passthrough with Live Metahuman performance
Вставка
- Опубліковано 20 жов 2024
- Streaming live metahumans in XR.
Finally figured out a way to modify the Unreal Live Link face support plugin to allow ARKit sources to be discovered by packaged Android builds. I’ve been wanting to stream live face and body performances to Quest using the Metahuman Face app and Xsens motion data. I like the idea of viewing/directing a remote performance in XR with the added bonus of simultaneously harnessing the power and fidelity of the metahuman pipeline for recording and post animation work directly in sequencer if that’s your thing.
In the same way that “Must See TV” was a ratings driver for television, maybe this is another look at how “appointment experiences” might evolve for XR.
seriously.. great work. are you using wifi?Or can it be done via internet? connecting faceapp to an exe file on pc and streaming that to VR on quest?
This is wifi lan, but live link works over public IPs if you have ports configured correctly. In this demo face app is sending udp packets directly to this standalone build. But you can broadcast LL to as many instances of the editor or builds that you want (local or cloud hosted).
I just started with questpro after using the oculus 4.27 Branch Version with quest 2 and experience REALLY bad
Performance wirh actually every use of PT (cable/airlink - ue5.1 Editor) ... unusable... you got any idea how to improve? Especially interested in working with spatial anchors ... seems not to work at all without lag from hell...
Not sure, do you get the same performance issues with a packaged build? I didn’t use PT with the editor because it never worked great for me, but builds were fine.
@@brycelynch actually it turned out i am an idiot ... after testing and testing - i noticed that i was connected to USB2 ... sorry for the stupid comment 😅
@@aprilhamlincom18 ahhh, yes! Glad you got it working! :)
this is awesome. Could you share how you did it? I was searching ways to do live mocap inside VR and never found an solution.
I would remove the passthrough and do some theater with avatars...
This is oculus branch with a whitelisted android MVN live link plugin for body and face is being driven by unreal’s live link face app with some slight modifications to ARKit plugin source to allow for android to pick up the data.
Are there any resources on how to modify ARKit. I would like to experiment with projects like this, but I can’t seem to get ARCore to work.
@@hazeyteeg1641 you have to add ARKit plugin and it’s dependencies to android platform whitelist in the plugin manifest. You’re also going to have to dig through the ARKit plugin source and find the reference to allowed platforms in arkitfacesupport and include android then recompile source. Otherwise even after you package the arkit live link won’t be detected.