PokeAHoleInverseMaterial allows you to see the hand passthrough in editor. This does not actually employ the Depth Occlusion from the Depth API, just a passthrough material onto a model tracked onto your hands, so, a bit misleading but still useful information!
To anyone struggling with hands in VR Preview - make sure your headset is in debug mode! I had to go into my Meta Quest app on my phone and go to settings and enable debug mode. Then when you launch the headset with Link cable you'll get a message your headset is in developer mode. You may also have to do it on desktop app if you haven't already. After that my hands work in VR Preview :)
Excellent contribution. Thank you very much for sharing. I have also had the same thing happen on vrpreview and there is nothing left but to compile to enjoy passthrough
Yeah, it's the biggest hurdle right now but if you plan it right and build the apk only when you need it, it can work quite well. For videos though it can take a while longer.
9:29 I don't know if you are still dealing with this issue but I'm able to load the scene with the hand tracking using VR preview. Disabling all OpenXR plugins seemed to do the trick for me as it has a bunch of compatibility issues with the current versions of the metaXR plugin. It also seems like the blueprints in the VRpawn are all pertaining to openXR so deleting all of those will probably solve all the compilation issues that you see when trying to play VR preview.
@@Spartinus I'm using MetaXR plugin version 59.0 on UE5.3.2 and I tried interacting with a cube to make sure the depth effect works. It did when I keep the oculusXRhand materials all on none.
Was able to get the hand tracking (with a black-hand visualization, the material is a not 'poking-a-hole') working by launching as a Standalone Game instead of VR Preview (still a bit slower but faster then doing a complete build-and-install). Great tutorials!
@@GDXR They are disabled, only thing: I disabled them after I activated the MetaXR plugin. On the integration page it states: -- OpenXR plugin should be disabled first before Meta XR Plugin is enabled because OpenXR plugin and Meta XR Plugin CANNOT be both enabled. This will be fixed in UE5.4. --- Maybe that is causing the issues? Later try that later
Make sure your headset is in debug mode! I had to go into my Meta Quest app on my phone and go to settings and enable debug mode. Then when you launch the headset with Link cable you'll get a message your headset is in developer mode. You may also have to do it on desktop app if you haven't already. After that my hands work in VR Preview :)
Fantastic video! Thanks again for all the great work and sharing it here and on your Patreon. Have to say your Patreon has been entirely worth the subscription!
my virtual hand not align with my real hand it bit offset if show to palm its aligned properly when i flip other way my two thumbs colliding how to solve this issue?
Unfortunatly i dont know. I dont belive there is a way to track other people but if you setup multiplayer it maybe possible to replicate the desired content.
If someone is struggling to find the material, click on the gear icon top right to the search option and then enable "Show Plugin Content" and then search for the same 👍
@@firedragongamestudio I think that material is not specific to Oculus. It's included with the Engine, we just have to enable view the engine content to make it appear in the search menu
@@praful0203 Found it! I had to enable not only "Show Engine Content", but "Show Plugin Content" too. Now I see it, thx for pointing me into the right direction. :)
Thanks so much for these tutorials, everything has been working really nicely. Only issue so far is since doing this tutorial there is an issue with hand/controller tracking. It feels like it bogs down to 5fps or so, with objects in the scene slowly anchoring back to their position. As soon as I hide my hands/controller so the headset is not tracking them; then the scene is stable and anchored as it should be. When they are brought back into the scene then it bogs down again. If you have any ideas about what settings could be causing this I would super grateful!
7:50 hi First of all, thank you for posting the tutorial. I'm following your tutorial and I'm facing one problem. 1. you deleted the hands on the existing VRPawn, attached the Oculus XR Hand component and moved the left hand -5.5 on the X-axis and the right hand -6 on the X-axis for offset adjustment. I followed the same approach, but the position of my hands and the transparent mesh that mimics my hands is way off. 2. I applied the transform values of the existing VRPawn's Hand (the one with the mechanical gloves) and the coordinates are even more off. 3. is there any way to adjust the offset of the hand in your method?
Thank you very much for making these awesome videos! Good information on passthrough and hand tracking is very hard to find! :) I am having an issue though and was wondering if I have missed something. I have followed your instructions but the packaged project crashes when I open it on my Quest 3. If I enable the OpenXR plugin it launches but then the passthrough does not work. Do you maybe have any idea what could be causing this? Thank you very much! :) Karl
Update: I got it working by first disabling all plugins in the Virtual reality section, including XR Base. Then restarted, and then enabled Meta XR. Strange, something must have been clashing and causing an issue. Thanks again for the awesome vids Jonathan!!
Is multimodal possible in UE yet? I can't find any info except for unity and native support so far. I'm really interested in tracking the controllers and hands simultaneously.
Does the MetaXR plugin for Unreal Engine 5.3 have support for multimodal interaction (Controller and Hands independent from each other)? I've seen Meta show some examples but I haven't seen much in their forums or official documentation. Otherwise, thank you for this video!
Yeah, in the Meta xr plugin, you can choose between hands only, controllers only or hands and controllers so if you put controllers down it will automatically switch the the hands you set up. You would need to do a check, though, in the player to see which is currently being used. As Both require very different interaction setups.
@@GDXR I see that setting yeah. But the design intention I had in mind were to have controllers and hands active together at the same time in the scene without any need for switching.
Hey there! When you make a VR game, do you need to manually code how every held object lines up with different VR systems? Or is there something in place that does this automatically? For example, if I code the held transform of a weapon in my hand to look right for my Valve Knuckles, how do I get that to look right with Vive wands?
Unfortunatly all Motion controllers seam to have a diffrent point which is used however with OpenXR in UE5+ you can choose where actors are positions on the controller when grabbed. the default is using the Motion Controller grip but you could always modify this to use the Motion contreoller aim point if required.
Hello, When I connect Meta Quest 3 to PC, Hands tracking not working at all and only works with Controllers, So I cannot do any interaction via hands in PCVR in desktop, Unreal Engine, etc, all settings I read across internet are active and configured but immediately when I do Meta Link only works with Controllers, What should I do? I use Windows 11, Meta Quest 3
Oh man I tried again and it did work!!! wow thank you so much, you are a life saver. Just one question, how can I do interaction with hands Correctly? I added Hand R Index Pinch, it grabs gun & boxed but they fall from my hand easily 😁
Ok so you removed all hand input code and now after finishing your tutorial I am able to see my hands but cannot interact with anything. How to bring interactions back?
You need to create a new interaction system so you can tell what poses your hands are doing. The old system isnt used for hand tracking at all so it wouldn't work anyway. Even if you kept it. I highly recommend you check out GitHub for the meta hand tracking example projects.
Im using the standart VRPawn, i have set the world settings to standart VR game. My project has many levels and i have put PlayerStart in a position inside a level, although whenever i play the project and a new level is opened through interactions, the player is not placed where i placed the player start. What is the problem?
So if I understand correctly, you're using a player to start in a passthrough based project and placing it at different locations, but they are not starting there. It sounds like you're not taking real-world movement into account. If someone moves to another location within the room, the player start can't spawn them there. They would have to physically move to that location.
Thanks for the tutorial But i have a problem, i packed my project and uploaded it to oculus quest 2, i try to launch, 3 seconds of black screen and crash, but if i turn on mobile hdr, i can enter the game, but one lens will see my game world and the other lens will see the outside world, what could i have missed?
It sounds like you may be trying to use non-supported rendering techniques in your application, Mobile HDR should be disabled and I'd recommend making sure you don't have anything like a sky atmosphere of fog in your scene.
Hi, how did you fix this issue, my apk build also shows one eye passthrough the other is only game objects. PCVR passthrough works and is laggy, as expected. @@Kanermi
You need to disable mobile hdr in project settings, remove all that relates to the sky in the location such as Sky light, DirectionalLight, ExponentialHeightFog,SkyAtmosphere,VolumetricCloud, also need to check in Project Settings Target Hardware, that are Mobile and Scalable@@thehotpotatosquad
Sorry about my English If you also have problems starting up properly after packing into quest, more like not starting in VR mode, please note the other videos of the author, MetaXR plugin must be placed in the Engine Plugins and not the project Plugins！！
How to only see your real hands but not use passthrough of the real world environment. So I see my hands but I keep the full virtual environment. And do you have video explain how to set up your hands using this method to interact with the environment? Thank you
from what i know so far, you would need passthrough for this, but you can try adding in a sky sphere and setting the passthrough layer to Overlay instead.
PokeAHoleInverseMaterial allows you to see the hand passthrough in editor.
This does not actually employ the Depth Occlusion from the Depth API, just a passthrough material onto a model tracked onto your hands, so, a bit misleading but still useful information!
To anyone struggling with hands in VR Preview - make sure your headset is in debug mode! I had to go into my Meta Quest app on my phone and go to settings and enable debug mode. Then when you launch the headset with Link cable you'll get a message your headset is in developer mode. You may also have to do it on desktop app if you haven't already. After that my hands work in VR Preview :)
Excellent contribution. Thank you very much for sharing. I have also had the same thing happen on vrpreview and there is nothing left but to compile to enjoy passthrough
Yeah, it's the biggest hurdle right now but if you plan it right and build the apk only when you need it, it can work quite well. For videos though it can take a while longer.
9:29 I don't know if you are still dealing with this issue but I'm able to load the scene with the hand tracking using VR preview. Disabling all OpenXR plugins seemed to do the trick for me as it has a bunch of compatibility issues with the current versions of the metaXR plugin. It also seems like the blueprints in the VRpawn are all pertaining to openXR so deleting all of those will probably solve all the compilation issues that you see when trying to play VR preview.
how can i find the pokeahole material? it doesnt show up , or how can i make a material like it
@@Spartinus I couldn’t find it either, it might be an openxr option something like that. im just using the default material
@@gingertew9249 problem is the depth effect wont work when doing any other material
@@Spartinus I'm using MetaXR plugin version 59.0 on UE5.3.2 and I tried interacting with a cube to make sure the depth effect works. It did when I keep the oculusXRhand materials all on none.
in your content browser you can enable show plugin content. enabling this should help you find the material.
Was able to get the hand tracking (with a black-hand visualization, the material is a not 'poking-a-hole') working by launching as a Standalone Game instead of VR Preview (still a bit slower but faster then doing a complete build-and-install). Great tutorials!
Disabel the openxr plugins and then try again
@@GDXR They are disabled, only thing: I disabled them after I activated the MetaXR plugin. On the integration page it states:
--
OpenXR plugin should be disabled first before Meta XR Plugin is enabled because OpenXR plugin and Meta XR Plugin CANNOT be both enabled. This will be fixed in UE5.4.
---
Maybe that is causing the issues? Later try that later
@@GDXR update: the 'hole poking' worked in the build (like you predicted ;-)
Make sure your headset is in debug mode! I had to go into my Meta Quest app on my phone and go to settings and enable debug mode. Then when you launch the headset with Link cable you'll get a message your headset is in developer mode. You may also have to do it on desktop app if you haven't already. After that my hands work in VR Preview :)
Fantastic video! Thanks again for all the great work and sharing it here and on your Patreon. Have to say your Patreon has been entirely worth the subscription!
Crossing enemy lines and watching this as a Unity VR dev. Unreal Engine is.. interesting.. haha
Welcome to the other side :) I hope you like it.
same! and funny, I can't get passthrough over link working in Unity playmode either..
Hey, the black texture is on the front of my hands (black texture) using the PokeAHoleMaterial, any solutions? ty
my virtual hand not align with my real hand it bit offset if show to palm its aligned properly when i flip other way my two thumbs colliding how to solve this issue?
we can also get hand tracking without building apk i just tested
Nice video !!! I need more videos!! I'm joining the patreon
Thank you for the great video. Can this be applied to feet or other people besides hands?
Unfortunatly i dont know. I dont belive there is a way to track other people but if you setup multiplayer it maybe possible to replicate the desired content.
Great video, thanks!
What is that PokeAHoleMaterial?
It is the material that will be around the hand (the color)
If someone is struggling to find the material, click on the gear icon top right to the search option and then enable "Show Plugin Content" and then search for the same 👍
@@praful0203 Tried it, but didn't work :/ Apparently my OculusXR folder is not here. Any suggestions what may went wrong here?
@@firedragongamestudio I think that material is not specific to Oculus. It's included with the Engine, we just have to enable view the engine content to make it appear in the search menu
@@praful0203 Found it! I had to enable not only "Show Engine Content", but "Show Plugin Content" too. Now I see it, thx for pointing me into the right direction. :)
Thanks so much for these tutorials, everything has been working really nicely. Only issue so far is since doing this tutorial there is an issue with hand/controller tracking. It feels like it bogs down to 5fps or so, with objects in the scene slowly anchoring back to their position. As soon as I hide my hands/controller so the headset is not tracking them; then the scene is stable and anchored as it should be. When they are brought back into the scene then it bogs down again. If you have any ideas about what settings could be causing this I would super grateful!
To add to this - after the guns in the scene disappeared through a gap in the mesh - the scene then became stable....
Update: deleted the guns from the scene and added my own grabbable cubes and that has got rid of the issue in case anyone has the same problem
I followed your method of packaging and it worked, but after I installed it on oculus, what happened when I opened him back to the flashback?
7:50
hi
First of all, thank you for posting the tutorial.
I'm following your tutorial and I'm facing one problem.
1. you deleted the hands on the existing VRPawn, attached the Oculus XR Hand component and moved the left hand -5.5 on the X-axis and the right hand -6 on the X-axis for offset adjustment.
I followed the same approach, but the position of my hands and the transparent mesh that mimics my hands is way off.
2. I applied the transform values of the existing VRPawn's Hand (the one with the mechanical gloves) and the coordinates are even more off.
3. is there any way to adjust the offset of the hand in your method?
the MetaXR app maybe using a diffrent position on the controllers than mine may be. you can take a look in Project settings > meta XR.
Thank you very much for making these awesome videos! Good information on passthrough and hand tracking is very hard to find! :)
I am having an issue though and was wondering if I have missed something. I have followed your instructions but the packaged project crashes when I open it on my Quest 3. If I enable the OpenXR plugin it launches but then the passthrough does not work. Do you maybe have any idea what could be causing this? Thank you very much! :) Karl
Update: I got it working by first disabling all plugins in the Virtual reality section, including XR Base. Then restarted, and then enabled Meta XR. Strange, something must have been clashing and causing an issue. Thanks again for the awesome vids Jonathan!!
Is multimodal possible in UE yet? I can't find any info except for unity and native support so far. I'm really interested in tracking the controllers and hands simultaneously.
Does the MetaXR plugin for Unreal Engine 5.3 have support for multimodal interaction (Controller and Hands independent from each other)? I've seen Meta show some examples but I haven't seen much in their forums or official documentation.
Otherwise, thank you for this video!
Yeah, in the Meta xr plugin, you can choose between hands only, controllers only or hands and controllers so if you put controllers down it will automatically switch the the hands you set up. You would need to do a check, though, in the player to see which is currently being used. As Both require very different interaction setups.
@@GDXR I see that setting yeah. But the design intention I had in mind were to have controllers and hands active together at the same time in the scene without any need for switching.
Great! Just great!
Please do face an eye tracking I can’t seem to get it to work on my quest pro.
How did you remove the background and open the camera?
Hey there! When you make a VR game, do you need to manually code how every held object lines up with different VR systems? Or is there something in place that does this automatically? For example, if I code the held transform of a weapon in my hand to look right for my Valve Knuckles, how do I get that to look right with Vive wands?
Unfortunatly all Motion controllers seam to have a diffrent point which is used however with OpenXR in UE5+ you can choose where actors are positions on the controller when grabbed. the default is using the Motion Controller grip but you could always modify this to use the Motion contreoller aim point if required.
please please help my virtual hand not properly moving with my hand
Anyone else missing Poke a hole material? what package is that in?
In the content browser, make sure to show plugin content and then you will get it.
@@GDXR If that can help someone else because I got the issue, you have to enable "show engine content" and "show plugin content"
Hello, When I connect Meta Quest 3 to PC, Hands tracking not working at all and only works with Controllers, So I cannot do any interaction via hands in PCVR in desktop, Unreal Engine, etc, all settings I read across internet are active and configured but immediately when I do Meta Link only works with Controllers, What should I do? I use Windows 11, Meta Quest 3
Need more info like any errors your seeing. Also without screen shots it's extremely difficult to help.
Oh man I tried again and it did work!!! wow thank you so much, you are a life saver. Just one question, how can I do interaction with hands Correctly? I added Hand R Index Pinch, it grabs gun & boxed but they fall from my hand easily 😁
Ok so you removed all hand input code and now after finishing your tutorial I am able to see my hands but cannot interact with anything. How to bring interactions back?
You need to create a new interaction system so you can tell what poses your hands are doing. The old system isnt used for hand tracking at all so it wouldn't work anyway. Even if you kept it. I highly recommend you check out GitHub for the meta hand tracking example projects.
Im using the standart VRPawn, i have set the world settings to standart VR game. My project has many levels and i have put PlayerStart in a position inside a level, although whenever i play the project and a new level is opened through interactions, the player is not placed where i placed the player start. What is the problem?
So if I understand correctly, you're using a player to start in a passthrough based project and placing it at different locations, but they are not starting there.
It sounds like you're not taking real-world movement into account. If someone moves to another location within the room, the player start can't spawn them there. They would have to physically move to that location.
Thanks for the tutorial But i have a problem, i packed my project and uploaded it to oculus quest 2, i try to launch, 3 seconds of black screen and crash, but if i turn on mobile hdr, i can enter the game, but one lens will see my game world and the other lens will see the outside world, what could i have missed?
It sounds like you may be trying to use non-supported rendering techniques in your application, Mobile HDR should be disabled and I'd recommend making sure you don't have anything like a sky atmosphere of fog in your scene.
@@GDXR Thank you I got it, I have one more question, is it possible to transfer what the first player sees with the through camera to another player?
Hi, how did you fix this issue, my apk build also shows one eye passthrough the other is only game objects. PCVR passthrough works and is laggy, as expected.
@@Kanermi
You need to disable mobile hdr in project settings, remove all that relates to the sky in the location such as Sky light, DirectionalLight, ExponentialHeightFog,SkyAtmosphere,VolumetricCloud, also need to check in Project Settings Target Hardware, that are Mobile and Scalable@@thehotpotatosquad
Sorry about my English
If you also have problems starting up properly after packing into quest, more like not starting in VR mode, please note the other videos of the author, MetaXR plugin must be placed in the Engine Plugins and not the project Plugins！！
How to only see your real hands but not use passthrough of the real world environment. So I see my hands but I keep the full virtual environment. And do you have video explain how to set up your hands using this method to interact with the environment? Thank you
Is there somewhere to get information about my question? Others tuturials or something? Thank you
from what i know so far, you would need passthrough for this, but you can try adding in a sky sphere and setting the passthrough layer to Overlay instead.
Discord is a good place, youtube doesnt give me notifications for questions so i can only see them when im online and checking.
Thanks for your reply! And could you guide me to somewhere to setup up Meta XR with hand tracking to teleport and other interactions?@@GDXR
With 5.4.2, do I still need to do all these steps?
Yeah.
can you share the template please? I got one eye issue :(
Bro is Flexing his PC storage with 5 Unreal engine versions.