nDisplay with Vive Tracker in UE4 (Tutorial)
Вставка
- Опубліковано 6 лип 2024
- Today we are looking at nDisplay using the HTC vive tracker and a projector.
@filmriot Projector VP Video: • Virtual Production wit...
Project Download on Patreon: f40.gumroad.com/l/cscmoc
Get Project files, additional help and suggest tutorials on my Patreon: f40.gumroad.com/l/cscmoc
Join Unreal Animation Discord server: / discord
00:00 Intro
02:18 Calibration
04:03 Plugins
06:04 nDisplay Config
14:18 Camera Tracker
22:25 Launch nDisplay
24:42 Testing Setup
25:40 Using A Different Map
26:56 Final Result
#VirtualProduction
#Unreal
Great release! Thank you for the great content! I recently picked up a used 75" LED TV to mess with this exact setup. Also, really enjoyed the cat interruption. So funny, laughed hard. I'm excited for the next video Aiden Wilson!
my guy! this was incredibly comprehensive and probably the vid that paves the way for my VP undertaking:) Appreciate you putting this out for us!
I have to agree, brilliant that you built everything from scratch, makes so much more sense without all the clutter!
Thank you for this tutorial! I followed your instructions and everything works fine.
yeeees! this is exactly stitching together some gaps i had. thank you so much aiden!
amazing content! Please post more videos with this quality about Ndisplay please!
Great vid bro...
22:07 INTRUDER!!! ALLERT ALLERT!!! RED CODE :D
Thank you very much for this video ! It's really help a lot! Keep Doing the stuff! Commented - Liked - Subscribed!
22:07 really got me. Perfect comedic timing.
She is the most annoying cat i've ever had.
Nice video!!!!
Hello! Excelent tutorial it helped me so much! I want to ask you how to connect the zoom to the blueprint with a second tracker, right now is only working in editor for me. Thanks!!!
This is amazingly helpful. Can you specify the base stations/tracker model and headset. you mentioned Cosmos for the headset. 2.0 lighthouses? 3.0 trackers?
I was using 2.0 trackers as well however the version of them does not make a difference to the setup and cosmos elite for the headset
Thx a lot for your tutorial ! I am so amazed by what what can be done with virtual production :) Everything is working almost fine ! however when i move close to my screen the object become little, and when i am further to my screen the object are becoming bigger ?!!! I think my tracker has inverted the Y axis. Is there a way to invert these moves ? (sorry for my english;))) and thx a lot for your help
That is correct yes, when you move closer the camera will see a smaller part of the screen so the window shrinks
Thank you for the information it really helped me get started. Is there a way to get better resolution on the NDisplay camera once Switchboard is triggered? The scene is tracking well, but the environment I'm using looks pixelated and there seems to be a ton of aliasing.
By default it should run at 100% resolution, but remember that the inner frustrum on the camera is only rendered on a small section of your display so it will be lower resolution than the display itself.
hey aiden, around 12:25 you talked about Camera Render Mode and "Disable Chromakey".. can you explain a bit how would chromakey fit in the ndisplay? is this displaying green on the fustrum?
So you can either set the Uber frustrum to be green or if you like the entire LeD wall to be green, alternatively you can instead load a texture to display in the inner frustrum or the entire wall
@@3d_Something ah ok.. interesting. i'm thinking to loopback the ICVFX footage back to UE for composure LED set extension ( so when camera point outside the wall it transition off to CG composure), but i think there will be issues of Color matching and time sync..like resolume. i think i might be able to blur the alpha edges a bit for blending the color but not sure about time sync yet
As much I would like to tackle this I don’t think I will be able to unless I get access to an LeD stage to do some research and testing, would be cool to see a completely in engine solution for this
I got this working, but one issue I'm having is that when I run the level in Switchboard the control of my camera tracker is weird. The pitch is reversed and it seems like the tracker's location is offset. I think it's starting position is at whatever values it's getting from LiveLink when I the level in the viewport, and I can't change them to 0 manually while it's active.
Is there a way I can save the level with everything at 0, or a node in the blueprint I can use to set it's initial position?
20:38 go to windows display, enable HDR ( or enable disable) it will be fixed
Hi, did you use any card for output display on wall like BMD deck l ink or something?
You output directly out of the GpU, as far as I know there is no way to use a Blackmagic card
Thank you for this tutorial. I hope you can answer my question. I use Null Driver so Without a Headset. In this case when calibrating the 1st time the CENTER where is my center since there is no headset physically.? Appreciate your efforts.
Hello, i have never used the vive without a headset before so i'm afraid i have no idea where it sets the center.
@@3d_Something Thank you Aiden for your reply. Have a great day.
Thanks for the amazing video. If I buy a vive tracker I need a base station for this ?
Yes, and I would recommend a headset as well but you can use it without
Fantastic work. Thank you for your time. Does this method achieve Parallax? I might have missed that part.
Yes it does, you can see it at the very end as the camera moves about
@@3d_Something amazing. Thanks again. So long green screen!
Nice is this best used with projector or can a green screen work?
Greenscreen can work but it’s a completely different setup
Great video !!
I have a question, hope that you can explain what to do.
I don’t have any trackers. But I still want to use unreal engine to project with my short throw projector, so I only can see the screen with my projector, how do I split the screen? What step to follow in your video ? Hope that my question make sense 🙏
Looking forward to hearing from you.
Instead of attaching the Livelink camera to your Cinecamera just move the cine camera to roughly match where your real camera is and you will still get the right depth and effect.
Greate tutorial! I've successfully set up nDisplay and genlock by following your tutorials. However where I could genlock the editor viewport to decklink, when I enter nDisplay the genlock doesn't work. Do I need a quadro card to do this? Thank you.
Yes you would need a quadro card to genlock the graphics card, although in practice running at 2 or 3 times your cameras framerate on the screen should work just as fine if you can't genlock.
Great video! I'm not using a headset, do you know what I should do to set a zero point?
Not 100% sure, this is one of the reasons i just forked out for a headset
May I ask what projector you using? I am trying to setup and learn. I am only using a green screen. I subbed to learn more. Thanks for the video.
Just a cheap on off of ebay, I would not recommend it
@@3d_Something Thanks
Good Tutorial !! 27:00 How to fix inner camera view port on screen(projector)?
Great video Aiden. I'm using Antilatency as a tracker device and it comes with the blueprint ready and I'm trying to skip the part with making a blueprint for HTC in order to work but when I run Ndisaplay, the tracker doesn't work. It works only when I hit Play in the Engine, so I assume I'm missing something...
If antilatency is using livelink then you will need the Apply to client node and setup from the vive blueprint to get it to work. It's still worth creating the vive blueprint in it's entirety even if you aren't using vive, because the blueprint does more than just get the data from the vive tracker.
For your bgh1 do you use sdi output or hdmi and do you also use the time code and gen lock as well
For this tutorial setup I was not using genlock or timecode and i wasn't using any output on the camera, it was all captured in camera so there is no need to plug the camera into the PC.
Good tutorial man, even if you are late to the party!
I had a question, why did you chose to use XR instead of VRPN?
I've been using vrpn and it's less temperamental and the setup is easier without the use of blueprints.
Live link I find easier there is even a VRPN to Livelink plugin, however almost all tracking solutions now have a Livelink Plugin,
Blueprints are also nice in case I want to insert code to run in the chain
@@3d_Something sorry I should have specified;
Livelink XR vs Livelink VRPN
@@antxionrhykel2699 ah, well that is entirely up to you then
Hi，when i use Switchboard，and i can not choose my level, it just show "default map",What could be the problem?
You mentioned at the end that the htc vice tracking is not so good. Im new to unreal and would like to experiment with camera tracking and was actually thinking of buying the HTC vive (not the pro) set with two base stations, controllers, but im afraid the tracking will not be good even for simple camera movement. Would love to hear your thoughts on this. Great vod anyways!
It’s great for learning however I would steer clear of it in any production environment
Has anyone ever gotten a "Controller can't control selected component" message next to the "Transform Role" under "Role Controllers" in the "LiveLinkComponentController" ?
that flickering layout bug related to GPU roll back to your previous driver✌
hello， ue5 switchboard Can't start, is there any way?
BOSS, How can I use linetracebychannel correctly with switchboard???
Thank you for offering great session! but unfortunately, I must be doing something wrong... If I fire up ndisplay, fps is really low like 3~5 fps. what should I do to solve this problem?
With ndisplay you can expect 2-4 time worse performance then in the editor depending on the resolution of your nDisplay screen
I followed the video step by step like 6 times, and everything is ok but when i enter switchboard the tracking stop working. I've seen some people that didnt add a value to cineCamera in the blueprint but just cant find my problem
I'm currently dealing with this same issue, any luck?
I lost the control of vive tracker in Switchboard! please help
Does this mean that the CP ( the Headset) is always on the floor or can I choose it further away, so it isnt in the way when you are operating the camera freely?
Thank you
Oh once you set the Center point with the headset you can move it to whoever you want
@@3d_Something Thank you dou you have a Discord server or smthg?
Yea in the video description is a link
Would it be possible to solicit your remote help? Following your guide but hitting a wall when Switchboard tries to "play". Says the following: "Node_0: nDisplay uasset transfer failed: Error while trying to write to D:\UnrealD\Test5\Content
DisplayConfig.uasset"
Perhaps try running switchboard as administrator
I have a Radion graphic card would it be able to handle this or do I have to go through an Nvidia GTX card
Well AMD makes the ai upscale so I imagine it will work with it
Can we keep the editor open when we start connexion in switchboard ?
You can however editing the scene in editor will not update in nDisplay and you will need to relaunch nDisplay to get the updated scene. Alternatively you can use multiuser editing which I may tackle soon.
My LiveLinkXR movement isn't translating into my Game Mode! I've done everything up to 16:30, and when I press "Play", my camera is stationary. However, the camera + nDisplay screen both move with my LiveLinkXR controllers, but ONLY when not in Game Mode! I've been using UE4 for about 2 weeks so I have no idea what I'm doing or where to even begin looking for answers. Does anyone have any ideas? Thanks! :)
Did you remember to set the camera value in the blueprint? I believe someone else had the same issue and that was their solution
I think I messed up something, as the camera track is nowhere near the camera. Any help would be much appreciated.
great video, but does someone know if this whole process is supported in UE5 already? It is hard for me to wrap my head around the process for connecting the Vive Tracker (with Mars) to all the nDisplay-Stuff at make it work.
It works just fine in Unreal 5 although I plan on making a new video for it in Unreal 5
@@3d_Something yeah, nice! Looking forward to it!
I would also love to this this tutorial in 5.
But i really did enjoy this tutorial aswell :)
When I press "play" on the portion for "Camera Tracker", my tracker doesn't move the camera. However in the blueprint viewport, it moves just fine. Any insights as to why that might be?
Did you include the apply to client node setup? That's the section in required to get it to run in nDisplay.
@@3d_Something I did. Followed the video faithfully, having watched it over and over. I'll keep at it though, because as a student in charge of getting the LED wall set up, the chance to learn all this is nuts
@@mandragoramedia i am in the exact same situation with the exact same problem
Hello. Can I ask if this set up can run in UE5 yet? thank you!
It can, simply follow the exact same setup, or you can make it in 4.27 then transfer it over to 5.0
Hey Aiden, I followed your tutorial and live link is working in the editor but it’s not working while running ndisplay from switchboard. Any idea why tracker data is not coming into ndisplay ?
did you setup the node to re add the livelink preset on event begin play?
@@3d_Something Ya, I setup the blueprint as its in the video and selected the right livelink preset in it. I'm getting the ndisplay out with inner frustrum displaying the ndisplay camera view but only camera tracking data is not passing to the ndisplay camera
I got it working now. I missed adding Cine camera in the camera tracker blueprint. Thank you very much for this tutorial.
@@tilkymacky all good
Can you please show your hardware set up
The hard ware setup is described at the start of the video
Hey Aiden, I'm not sure if you are still looking into moving on a controller only, now that I assume you are using specifically vive, but in the past 10 days or so I've been running through a lot of your videos among other resources setting up a test project to present to my college and I've managed to figure out how to use the joystick to control movement properly, similar to an fps character and the dpad method, so that where the camera is facing that is always forward, rather than the world forward always being forward on the joystick for the Oculus Quest 2
I haven't tested non dapad esque joystick movement just quite yet, but I've figured out a lot of things, if you're interested at all feel free to leave a reply, I'm not entirely sure how to privately contact you but if need be I can leave an email address.
Is that using a character movement component? I am aware of quite a few ways to move around an object using a joystick now in Unreal, I know a lot more now then I did when I started, however if you do wish to share feel free to reach out to me on another platform such as linked in
@@3d_Something OK I sent a connection request, since messages are locked, the movement is based on the tracked camera bp, the way I've done it, so using a controller and not being in vr
For some reason when trying to connect the device in switchboard it says "Socket error: (ip address:2980). and the node won't connect to the device. Any ideas?
Did you launch the list we and the controller? Both need to be running for it to work.
@@3d_Something Wow. You're a genius. I had switchboard listener close out. I got it working great! Only thing now is figuring out the offset maybe from the tracker to the lens. When I look through the view finder it is a pretty big gap between the frustum and the cameras fov. Is there a way to make it so that the camera sees closer to just the inner frustum. Also thanks for all the work you put into these videos, they've been a huge help.
Also in the video you mention how to render the parts outside the inner frustum at a lower resolution, but you cat interrupted and I didn't catch how to do it, any tips?
@@benjaminbrown282 if you click on the frustrum plane there is a resolution multiplier
Hi, Vive tracker not workig in switchboard please help!
Make sure you save the Livelink XR as a preset and use the Apply to client node in the blueprint.
@@3d_Something Thanks 😀
there is a fix for that windows not working thing in UE4, it's an NVIDIA issue
Does this work in Unreal 5 yet?
Yes it does
@@3d_Something I have a problem where even after connecting the tracker and base stations then opening unreal (@ 5:55 in the tutorial) there is no XR subject that appears. I can't seem to figure out what I'm missing, but when I add a LiveLinkXR Source I don't get anything as an option. Any ideas what this could be? I followed this tutorial step-by-step.
could you redo this for 5.3 :P
Yes
error on unreal 5.1
LogSwitchboard: Display: Started FlipMode monitor 9604: ../../../Engine/Binaries/ThirdParty/PresentMon/Win64/PresentMon64-1.5.2.exe -session_name session_16180 -output_stdout -dont_restart_as_admin -terminate_on_proc_exit -stop_existing_session -process_id 16180
LogSwitchboard: Display: Process 9604 (flipmode_monitor) exited with returncode: 6
LogSwitchboard: Display: Output:
warning: PresentMon requires elevated privilege in order to query processes
started on another account, so these processes won't be targetted by name
and will be listed as ''. Further, there may be tracking errors
near process termination, and if they are targetted
-terminate_on_proc_exit won't work.
error: failed to start session (error=5).
how do i run switchboard as administrator? I can't fix it in any way, I also try to run unreal as administrator but nothing.
i got this error
LogSwitchboard: Display: Started FlipMode monitor 17740: ../../../Engine/Binaries/ThirdParty/PresentMon/Win64/PresentMon64-1.5.2.exe -session_name session_19564 -output_stdout -dont_restart_as_admin -terminate_on_proc_exit -stop_existing_session -process_id 19564
LogSwitchboard: Display: Process 17740 (flipmode_monitor) exited with returncode: 6
LogSwitchboard: Display: Output:
warning: PresentMon requires elevated privilege in order to query processes
started on another account, so these processes won't be targetted by name
and will be listed as ''. Further, there may be tracking errors
near process termination, and if they are targetted
-terminate_on_proc_exit won't work.
error: failed to start session (error=5)
Looks like you need to run it as administrator
@@3d_Something you mean change switchboard, ndisplay ip adress? my vive tracker doesnt move like u
@@sj5041 that error looks like you need to run switchboard as an administrator on your Pc
thank u… how about why my tracker doesnt move..? i follow all step like u
Hi, I have a strange glitch where a sphere appears in the scene when I hit Play... Has anyone else had this issue and found a fix? Here is a link to a pic of the issue: drive.google.com/file/d/1MMdGT0nxoiRQjHtARMUflu5_nIeMnQre/view?usp=sharing
What your seeing is the player, i did do a video about this along time ago, another option is to start the camera very far away from your scene when you hit play or create a begin play very far away. Or the last option would be to build your setup so that it includes a player pawn.
I found an easier solution from user Greg Corson :
Another way to get rid of it is to go to Edit->Project Settings->Maps & Modes and set Default Pawn Class to "None". Then Unreal won't spawn it when you press play.
If "Default Pawn Class" is greyed out, In the selection of Game Mode, click +, New Game Mode and then you can put pawn as none.