VR Full Body Tracking With JUST a Quest & a Phone!
Вставка
- Опубліковано 17 чер 2024
- In todays video I will be showing you how you can get full body tracking on your Quest 1, Quest 2 or Quest Pro, using nothing but a Quest, and a phone. This does also work for any PCVR game that supports OSC. However, today we will be using it fully standalone with VRChat on Quest. I will be taking you through exactly how to set this up and how it works. This will allow you to get VR Full Body tracking with your phone's camera. We have tried something very similar in a previous video, however, it did not work well. This might be a viable option for super cheap full body tracking! I hope you enjoy the video :D
~~~~~~~~~~JOIN OUR DISCORD~~~~~~~~~~
/ discord
~~~~~~~~~~AND REDDIT~~~~~~~~~~
/ themysticle
~~~~~~~~~~~Links~~~~~~~~~~~
VRCover Link: vrcover.com/#469 [Use Code MYSTICLE]
Android App: play.google.com/store/apps/de...
iPhone App: apps.apple.com/us/app/driver4...
---------------------------Timestamps--------------------------
0:00 Intro
0:49 Why Want Full Body?
2:20 Installing & Setting Up The App
4:04 Setup In VRChat
4:39 Testing The Full Body In VRChat
6:55 Conclusion
7:54 Outro
----------------------❤️Thank You To The Patreons❤️------------------------
James Lui
TheRaymondTodd
Roz McNulty
Alan Carr
SkyEver
Michael Siebielec
Ezinor
yu gao
SpatialBarrel00
sleepydragon597
paul james
Zbigniew Rybiński
Trent Fowler
ChuchoF3TT
SxP
VeryEvilShadow
William Edney
Imnafoy
David W Ratcliff
Kenny Balter
Cerberus Velvet
Skeeva
---------------------Join Our Minecraft SMP------------------------
Java IP: mc.themysticle.com (Version 1.16.4)
Bedrock IP: mc.themysticle.com
--------------------Patreon-----------------------------
Support The Channel 😀: / themysticle
-------------------Amazon Stores----------------------
US StoreFront: www.amazon.com/shop/themysticle
UK StoreFront: www.amazon.co.uk/shop/themyst...
EU StoreFront: www.amazon.de/shop/themysticle
US Affiliate Link: amzn.to/30HaECU
UK Affiliate Link: amzn.to/2Y28Xyg
EU Affiliate Link: amzn.to/3e1mP1f
-------------------Spotify Podcast--------------------------
open.spotify.com/show/4Nfn9na...
--------------------VLOGGING CHANNEL----------------------
@mysticlemore
-----------------------Oliwiers Social Media-----------------------------
Website: TheMysticle.com
Instagram: TheMysticle
Twitter: @TheMysticle
---------------------Outro--------------------------------
If you liked the video make sure to leave it a thumbs up and if you want to be notified of future uploads subscribe and press the bell icon and see you, In the next one, Peace out.
#VirtualReality #TheMysticle #VR - Наука та технологія
![The Best VR Setup for YOU! Complete 2024 Guide [VRChat]](http://i.ytimg.com/vi/DomfNq0vNCk/mqdefault.jpg)
![The Best VR Setup for YOU! Complete 2024 Guide [VRChat]](/img/tr.png)
The app is now also out on iPhone! Exact same process, so if you have an iPhone, feel free to follow along :D
This might just be the easiest FBT Setup we have tried to date. It has it's issues, but it might be enough for some. Let me know what you think! Have a great day everyone ❤
You too thanks
can you review pinoquest? it’s the same as this but for iphone and might track semi better when turning around and sitting
Pinosoft is mutch better.
@@paxu_on_u821 Pino is better
@@paxu_on_u821 I have Pinoquest video on my channel. Seems a little bit better. It works turning around, runs faster at 30fps but still only good for basic poses. And it runs at low FPS because limited phone hardware to run the AI.
I have a version for PC with Cuda and puts 40% load on 2080ti just to run tracking from 2 cameras at 50fps.
i’ve been using this for the past few days now and honestly it’s great, it does have some bugs but for it being so cheap and new the tracking is Great so far and the experience is awesome!
Nice to see other reviews. I honestly play more fighting games in VR so I'll probably just save up for SlimeVR.
@@StarLeader44 Just a warning: Slime VR trackers have terrible battery life. You can get an hour MAX on them, most likely going to be around 45 minutes, if I remember correctly.
Please review the app with 5 stars. It is sitting with a measly 1.8 rating and people are going to think this doesn't work.
can you help me?
So this is my first time using fbt and I'm trying to use it. However, I can't pass the calibration stage. Is there something i'm missing?
the problem with the 14 fps is because the app is using an ai model to track your movement.
basically is taking each frame of the video and passing it to the model, then the model gives you each track point in x/y values, after that it starts the real integration made by driver4vr passing that data in a format that vrchat can understand via osc.
computer vision is hard and even harder on mobile devices, in a pc using movenet usually you could get around 30 and 40 fps. 15fps for a pose model working in real time is a crazy thing when you remember 3/4 years ago this was only possible using nvidia gpus on a pc.
Couldn’t they make it so it just passes the camera input to your pc and let the pc do the heavy lifting?
Not only this but it has to send that data over osc to vrchat
Hella impressive this can be done now
Because of this he predicts where the arm or body part will be, even occluded.
Maybe there would be a way to use an Android emulator on PC and take webcam input?
@@FutureKLX300SMowner you can use your phone as a camera with a 3rd party software and use driver4vr, be in mind most pc usually can run the model at 30/40fps depending of pc specs, camera and light conditions
That’s super interesting. I feel like FB could release a standalone fbt camera meant to just set up on a table and provide tracking the quest users this way and it wouldn’t be a huge barrier to entry. No mounting required, just plug it in beside your quest and set it up on a table.
They already have the portal.. I feel like they could just use that with the proper application
Sadly only for pc but i think HTC is working on inside out tracking (without base stations like the quest with its cameras for tracking) fbt trackers
driver4vr does have capabilities of using the kinect, and there's simple adapters that make it possible.
This dude has helped me with so much problems I run into with the quest, and for a person who can't even afford to go to the laundry mat I find this very helpful. I'll be using this untill my slimevr trackers come in. Thank you for the help with everything
You have a headset but cant get laundry done? Priorities right
@@Zabrakjedi yo I was saving up for 7 years till I could afford it
@@youshouldnthavegaveme30slots it took you 7 years to buy a quest? It was like 250 when the quest 2 launched, thats like a quarter of a paycheck
@@Zabrakjedi That’s incorrect actually. A Quest 2 was $350 - $400 at launch.
@@pryma1604 maybe if you got ripped off lmao
Feet Saber sounds like an incredible opportunity for exercising at home, definitely gonna do this for my setup
I always trust this man with news
I mean me too but that's such a sus thing to say
Facts
Same
He’s the most honest news sorce
Same
Wow! Thanks I'm going to give it a shot with an old android I got laying around. I'm with you that I am excited that this will only get better and I really hope people invest the time money and energy in this type of app to make it better where we don't have to actually buy any other equipment for full body that is amazing!
you never disappoint us i love your content
This is wonderful! Going to have to wait on this as my favourite VR game is Blade & Scorcery so it would need to track fast movements, but I'm so happy to see this and definitely going to keep an eye on this!
How could I get it to work on blade and sorcery
I bought this on iPhone earlier today, and my only real issue is how it seems to think my feet are farther forward than they are. I think it's most likely due to a lack of contrast in my backdrop, though, so if you upgrade to the full version, play with the angle and position of your camera in your playspace to get a full view of yourself with a high contrast backdrop. I'm sure if you have a green screen, that would work quite well.
Dude I wish I found your channel sooner! This is actually cool as hell! thanks!
I can't wait to try this in 2 months after I move back to my homestate, but TYSM Mystical for the wonderful news! 👍
There's a weekly meetup in VRC that I go to, this is going to be a game changer, since we mostly just hang out and talk (drinking may or may not be involved, LOL). SO for me this is perfect, as I use my Quest 2 with my PC, now I can have full body tracking without spending hundreds for body trackers. Thanks Mysticle, as always your content is awesome and I am glad you do what you do.
Edit... Just installed the app, took all of a few minutes to setup and test, to see it work in VRC. SO I paid the $15, even if I only use the Driver 4VR app once a week or less, it's well worth it. PLUS for the money I am also supporting a company that created this app. SO I see that as a win win for everyone.
Do you know if it works for PC VRC?
@@hidoggi ... Yes you can
holy crap this is amazing
This is super helpful I always get gelles with out full body but I love it I’ll try it out 👍
Excellent video, I'm going to try this soon.
Neat, i’ve been looking for a FBT solution to make animations and this looks to be worth a shot
Just don't move too fast and you should be fine, unless you can move it slowly and just edit that movement to make it faster. Lol
Don't waste your time
Very cool thanks. I think the obvious application is for creating 3D animation. After viewing this I am little surprised and disappointed to see Meta Quest 2 hasn't already solved these motion tracking issues. I used to do fully body motion capture with a few X box 360 cameras 7 years ago. I thought Oculus went wireless cause they had somehow perfected their technology. Definitely would like see some improvements, even it means wearing a few bluetooth trackers.
Dood this is so cool thanks for sharing this with us.
Man I can’t thank you enough I have been looking for easy full body tracking and my wish came true because of you thank you mysticle!
What is this application called?
@@SEWAYAPONEC_PAVLOVdriver4vr
My theory about why the fps is so low is that your phone is handling a lot of data stuff constantly at all times. From what I can tell, the app is; receiving the video feed, running identification software to locate what body is where, running that calculation and sending it to the receiving software via the internet, and doing all of that 14 times in a second. If there were a way to run it through a pc it would likely have less latency up to the limit of how fast your internet is.
Man, i don't know u, but u sure have a new sub here, i mean now i Will have full body track when i finally get My VR Headset
Tysm
I just love this wacky ways as a development I just love to try experimental software like this
14 fps.
use 2 phones = 28 fps
4 phones
56 fps
boom fixed
Just get fbt at that point
Oh wow this is so cool. I don't know if this is even an option because I don't know how this app works when tracking but could you disable the tracking of the arms. The controllers kinda already have those covered and I don't really care which way my elbows are facing. I wonder if disabling those will improve fps. 🤔 What you guys think?
THANK YOU MYSTICLE!!!
keep up the hard work becuse it is amzing
This seems like a great idea for people like me that can't even afford conventional full body tracking. However, I do wonder if this method can be combined with other full body tracking alternatives to improve accuracy, like owoVR, which I found to be quite easy as well. Only issue with that one is looking around for enough phones to make anyone suspicious
3:18 Driver4VR is also available on iOS too in case anyone wants to know
thank you so much,cant wait to play vrchat.
Thanks you for telling me this I had some problem but it worked great I had a great time in vrchat
15 euro is an insane price for FBT, even if it's a tiny bit janky. Seeing a work around saving yourself 100+ dollars from buying the Vive mounts or the PSVR color disc things is crazy.
So I only have a Rift S, not a Quest. Can I use my PC IP for this to work? Or is this *only* usable with a standalone?
Got it up and running this morning. I had to extend my CPU limiton my phone. But I get 20 fps in app now. Sliightly better tracking
This better be real ive been checking every couple of months hoping for a cheap alternative because I thought without it, I'd never be able to experience such an amazing feature.
Oh thank christ it's real and a great starting point a cheap accessible body tracking, I've been waiting so long
The amount of times I've read titles of body tracking in quest only to be told oh you'll need a pc, like if I had the money for a pc I wouldnt be looking this up, finally this is such a good step
Does anyone know if this works if you stream your quest through virtual desktop?
Are you able to use multiple phone cameras for this?
No I don’t think so
Thank you so much every other way I've seen cost tons of money that I don't have
It works, thank you sooo much! ^_^
Could using a mirror, angled so that your body is always visible to the cam and your body as well, improve functionality one day?
It would detect you first then detect the mirrored surface for basically 360° tracking, in theory
I like your thinking. Perhapsomes it may possibly work maybes.
You're a genius, now we need to find out if it will work like this and how to set this up properly
Yes and no bc its a ai and its very hard same if u turn 100 times whith ur eyes closed and then look directly forwords whith out going left to right very hard
or logo?
Not sure tbh. The AI might see the reflection as a seperate body and try to track that and yours at the same time, making some odd deformation in the model.
likely the app could only process the movement at 14 fps, this may or may not be true depending on how high spec your phone is and how well optimized your phone is.
Might it just be their Wi-Fi?
@@QualicSelf no. It's using an AI model that is heavy lifting for the phone processor.
Fifteen euro is actually a great price for full body! Plus waist tracking is completely free. It may have some bugs but as you mentioned in the video, it was only recently added to mobile, so bugs should be expected.
Full body VR tracking, taking Chun Li's leg attacks to new levels
Will it work through PCvr if it's connected to the quest or is it just accessible for the quest?
It also works on PCVR.
@@Nicalay2 When I put the PC's IP it says connection rejected so lol
@@DHD10000 did you enabled OSC ?
Does this work with the pcvr version on vrchat? I usually play vrc with wireless oculus link
It actually does :) You just need to enable OSC in VRChat exactly as you would on the quest.
@The Mysticle so do I go through the same process except open vrchat on pcvr instead? Because I recently attempted to do this and everytime I try to connect the app says "connection refused"
The mathe behind the full body recognition is mindblowing.
But yeah xbox kinect had a mayor advantage. The Depth data and a processor to track bones.
Yo thanks bro 🙏
Just tried it out for 15 bucks its pretty decent. hips move more realistic. Feet not tracking super great but at least when your standing its a more natural pose.
Being able to sit and it not look weird like it normally does for stand alone also a big plus
I think it's worth 15 bucks to support this company. The android version looks new so hopfuly it will continue to get better.
As long as the support keeps coming, I'm sure they'll find ways to improve on it.
Thank you so mutch
so I did try it out with just the waist tho. and it's a pretty simple setup. you just do exactly what he tells you to in the video. the waist does move but it only moves back and forth. not side to side. idk if you need the feet also to be able to do that. I will soon get the premium and try it all out together and see
please do a video on OpenXR eye tracking for Dynamic Foveated Rendering. It really needs to be pressed how much of an impact DFR could have if made standard. It's essentially *free* performance if implemented correctly and that needs to be made standard.
I HAVE THIS ITS AMAZING
Thanks for the review! Can this be used to export as animation to unity or blender?
There's other social VR games as well that support fbt. Not sure how it's done in CVR but I know for sure Neos supports it.
What a good cheap body tracking for quest2 wit no pc
i tried it and it works! though i only tried the waist tracking for now, its buggy of course but it seems like the tracking for me at least is really slow and delayed unlike what it showed for him. i dont know why that is
It's pretty good😊
OMG thancs a lot
Hey the misticle great video btw but I have a question, every time I calibrated with the t pose. It wouldn’t calibrate correctly. So I was wondering if you could help?
You are a very useful and informative person! I don't suppose you can recommend the most pain-free way to record video in VRChat using a Quest 2 with PCVR airlink? I assumed that it would be simple, yet I'm going round the houses trying to find an elegant solution. Thanks
Just get obs on your pc and record it there
It works!
very very buggy though as he said, but it still functions :)
it wont track my legs correctly, the functions swap and its strange but its something!!
Computer Vision research in early 2000's "wow we got a tracker running at 15fps using a high end PC!". Computer Vision consumer products in the early 2020's "WTF? why you no run faster than 15fps on my telephone?" XD
Great stuff! Any idea if it's possible o use this method for walk-in-place locomotion? On pc or standalone? Thank you!
I will probably use this until I get my SlimeVR!
Heyyyyy 200k subs let's goo
i sadly have to wait till august to get my slime fbt but i do think i may want to try this!
Slime VR battery life is 45 minuites☠️
I LOVE YOU!
Could this be used to add 6dof when using something like a google cardboard, Samsung gear vr or rift dk1?
thank you
Does this work with games other than vrchat, such as blade and sorcery?
analysis of an image is a very demanding task for phone/Pc (more accurate = slower)
2:23 **clicks off video on my iPhone**
Apparently there is an app that does this on iPhone. If I manage to get my hands on one I'll make sure to cover it. Unfortunately, I don't have one right now.
I've been looking for SlimeVR. I am also using Full Body Rigging with Animation and Mocap suits are very expensive.
To be honest it seems like a budget friendly solution to full body tracking. cause not everyone can afford the trackers and stuff. and yes it isnt the best, but its kinda expected to be honest since its a camera tracking your movement. ether way its a good budget friendly thing to try full body tracking out
That's amazing! What phone did you use? I wonder if the chip of the phone matters in FPS. For example I have a high end Samsung phone with a beefy chip so I wonder if it will be higher than 14 to 15 FPS?
My Samsung S21 Ultra gets 30 FPS and my Samsung S9 gets 15 FPS, so you would be correct, the more powerful the device the more FPS there is.
I pretty sure someone probably told you already but the only reason driver vr is running at 14fps is that it requires a ton of cpu power. It uses deep learning to track.
Yep, my Samsung S21 Ultra is getting 30 FPS and my Samsung S9 is getting 15 FPS.
Crayz how all these big VR youtubers all look like they really need the Metaverse
hmmm i might try this one
ty
insane
I feel like they need to update it to get the amount of updates per second to around 45 per second, 14 is a great start but nowhere near as fluid as 45 and if they make in between key frames or generate predictive key frames they might be able to get more fluidity of the tracking.
How do I do it if I want to have the quests connected to the PC? good video
on pc with improved software could probably connect 2 cameras from 2 angles and combine results = win
I have my lights led green it hella helps
Have you tried running the APK from the Windows 11 Subsystem for Android? I was wondering if it’s possible to run it from Windows 11 using my more powerful CPU and Camera.
Honestly wish meta would just include a couple small pucks with the Quest 3. some wrist, ankle, and a waist strap doesn't seem like it should be that big of a deal to me. But I'm also no expert. Full body tracking in VR definitely needs to be the future to get more people engaged, the fact meta hasn't been trying to dominate that more is beyond me... Who doesn't want to put more of themselves in VR? Surely you want to be able to interact with your whole body.
What about the hands? Are they being tracked too? Will I be able to take items?
I loved it just that I don't if I should buy it
Cameras and phones are amazing, but folk do not know what you can do with a camera and I have try to push for this around 3 years. I use a camera to 3D scan and I get precision at microscopic level when use a macro zoom, and a camere can also see you breath and that is used by some health apps, it can also see oxygen in the blood, a normal camera can also spot pressure waves in the air.
The phone also have several sensors that can be used, I have for an example an app that convert magnetic fields into sound and when I put my airphones in my armpits I can feel the vibration and feel where north is, well my point is that a phone with all sensors are so underrated and under used, and had we not have moral panic most phones today would have a near IR camera that had been even better for body tracking and had worked as the first base stations for the Oculus DK1, then all needed for tracking would be some small stickers or clothes with small reflectors.
But this is a great video and I hope more developers of VR and games look towards phones as alternatives to controllers and so, and maybe for poor mans face tracking.
Very intriguing, I'll be giving it a try this evening. Before I purchase, is there any reason this wouldn't work with PCVR? I use a Reverb G2.
It should work since it uses OSC which is also available in the PC version of VRChat 🙂 You can always make sure by using the free version first 😃
Sweet, I'll do that and try it out first, thanks for the info. Earned my sub!
Does it track facial movements? Like, could you use it just for your face? Place it up close?
i dont know if anyone will see this and actuly take it as solid info but using slow mo on the camera boosts fps SIGNIFICANTLY. so if drive4vr would use the slowmo camera instead of the base camera it would track better. using normaly camer uses slow fps. for battery reasons
Amazing but im hearing-
If I already bought driver4vr on steam and I get the app I have to buy it again? Or is there some way for me to get my steam account connected to the app?
one problem i had was the app had the blue and red circles swaped so when I swayed my hips left the model in vrchat would sway right. i don't know if there's a way I can fix this or not
Now I just need to wait for someone to make a free one
I hope that driver4vr will make it possible to use your phone as a 3-dof vr controller
Everyone needs to give their addresses to this man.
Why you have been roped into this scam is beyond me
How's it a scam
Because the trackers are bad and there is a free option which is way better.
@@CocoHeadVR123 doesn't mean it's a scam, it's bad yea, but it gives you what he's showing
@@CocoHeadVR123what free option is way better?
Its not a scam lmao
He is your crown 👑