It’s incredibly helpful seeing companies starting at square 1 and seeing them take each step. Congratulations on getting your series made, looking forward to seeing more!
I used to be in this field then I quit for 4 years from my mental health. I was too scared to back to this field again. Now after your inspiration which you made things so possible... thanks dude
Brilliant video, you perfectly captured the frustations a lot of people had at the beggining of the lockdown and trying to learn virtual production on the fly, failing a lot on the way.
I recently did a very low-budget music video where I built two walls of LED-strips and used pixel-mapping to get lighting which matched a rear-projected background. It really sold the immersion. So I'm sure something similar might be able to work for them too.
The UA-cam gods sent me here. This is freakin amazing. I've been looking for alternatives to on-set filming for forever. I can't animate or draw, but I can definitely act like an idiot in front of a greenscreen.
And Jim Henson was revolutionizing the virtual production studio way back in 1987 for The Storyteller, and in 1989 for The Jim Henson Hour. Kind of amazing how far we've come in making the tech smaller-better-faster-cheaper, but the results from using chroma screen look exactly the same today, but in 4K. Volumetric studios are the best virtual ones, as rear projection work has always been better than chroma screens, and now we're taking it further with video walls and tech that tracks the camera in near real-time. Very exciting stuff! Thanks for this generous BTS video!
Incredible achievement. I'm sure you've got lots of other things to do (that help you pay the bills), but a tutorial on how you did it would be fantastic. Or even a list of the links you found most hopeful.
The jittering is absolutely not a limitation of the VR system. I don't know what caused this in your case, but you can definitely get a smooth image with the Vive tracker on your camera. Anyway, this was terrific! Virtual Production evolves so quickly, it's astounding! Last year we had to manually calibrate the camera, now we have a simple, almost automatic calibration system. Here's a few advice : - Virtual Production is even more interesting when you move the camera. For that, you need to make sure, in the UE scene, that the virtual screen (the rectangle on which you project the actor) keeps facing the camera. That way you can shoot scenes with a moving camera, instead of just fixed shots. - You can get shadows automatically with the proper material settings on the virtual screen. No need to put it on each frame manually! The limitation to that though is the shadow is projecting a 2D surface, so there's an angle at which point it doesn't feel like the natural shadow anymore ; but that's just a limitation to keep in mind when working on the lights! Aside from that, you get real-time accurate shadows.
Really good video. One very cheap option for camera matching is to control the real camera with the virtual camera. A pan and tilt on a tripod controlled with an arduino uno then mapped to the scene camera is simple but effective, certainly as a rig to "play with" as a learning exercise. I use Blender and OBS studio.
Love this! I stream VR games in mixed reality, and do sketches too. The fact the tech is colliding in so many people's minds is brilliant. Glad production tools are catching up. Realising the 3080 Ti graphics card in my machine was powerful enough to do something like Mando, and in real time, was a game changer.
I don't know how I came across this but you guys are amazing for having made such a detailed, comprehensive BTS on this process. I shot my first VP commercial last year and enjoyed it.
Tundra are making a big tracker that is designed to replace the Vive one for virtual production and Vive are working on some hardware to support this stuff too. Exciting time to be into VR tech!
You should get some low res but bright LED Panels with diffusion on them, that are linked to live feeds of virtual cameras so that you get some interactive lighting. Ian Hubert talked about similar stuff on his Patreon a while ago.
Wow, UA-cam has just told me about this now so I'm a bit late to the par-tay! The mandalorian volume is actually new application of a very old VR space called a cave as opposed to the HMD we are now more used to. As you say technology is moving rapidly in this area, I saw a package from Quanta systems thar used its RGBWW tubes that to cast light in a similar manner to LCD panels but at much less cost. Thus getting the real world sense of the human interacting with the environment but without the cost. Its interesting that you didn't look at using rear/front projection especially to get the light scatter on the humans. More videos about what you learnt or want to achieve would be interesting But thanks for this I forgot to ask did you look at all the great work done on this by Robert Rodriguez especially with his use of green screen with Spy Kids & the Sin City series??
Really fascinating. They say you should never let daylight in on magic, but I'm grateful! We use straight CSO / chromakey for our training programs but this is truly astonishing! Congratulations!
What a time to be alive! Can't wait to be talking to some friends about a Tv show in 10 years, and i'm just like " Oh ya Them? I saw how they started " :D
This is really excellent. I do a lot of work in After Effects and Cinema4D for Augmented Reality. But it’s all post production. Unreal Engine is something I’d love to get involved more in with and this is great video for starting with it. Thanks so much
I'm sure that virtual production is the future. Technologies like The Volume as well as use of Unreal Engine like you've done so here. They're exciting times we live in!
This is fantastic! I'm about to make the jump back into live action and am really eager to get my head around Unreal 5 in an FX context. It would be great to be able to have comedy sketches having access to real-time FX in their scenes, hell even if it's as cheesy as South Park using live action explosions in the scenes there could be great comedy in it.
I want more!!! Show me how I can start today. Just like you all started. Or make a video on what you think the next tech step you think would need to be.
Just a suggestion, but whilst robot arms are indeed expensive, there are still several good motorized sliders available (even a couple that support the Mini Pro) for reasonable pricing.
This was amazing and so insightful!! Thank you so much for sharing! I'm based up in the North East of England and love watching and seeing content like this. Hope to one day do EPK in film and TV and so seeing all of your BTS is really awesome and inspiring. Keep up the great work and will definitely check out your series on iplayer! :D
This is something Disney (and the team behind Lazy Town too!) has been doing for a few years now. Off the top of my head, the Muppet crew uses a mocap puppet rig in "Earth to Ned" that they use to drive a CG character in Unreal that gets comped into a scene with real actors. That way they can get all the eyecandy of CG animation with the spontaneity of their classic puppetry.
The Vive trackers have such weird issues. The first gen were great, but the second gen, while ostensibly better, had a lot more 'wiggle', or could be off by a solid 2CM. And these problems could be caused by anything from the USB port not being fast enough to something reflective on set confusing it. You can minimise it by using 4 lighthouses instead of 2, but that doesn't totally mitigate their issues. They're great bits of kit, and those original errors you show look more like update speed or rendering issues than tracking issues (long USB extender?), but they're so fiddly you're right to get away when you can.
You're so right about the Vives being fiddly and frustrating at times. Sometimes they would work brilliantly, giving very smooth movement on even a complex map, other times they would be really jittery, losing their orientation as well sometimes. We eventually put the wireless dongles on the camera, as close as we could to the actual trackers, by using long USB cables back to the PC. This did sort out a lot of the problems. I see the new Vive Mars system uses a box under the tracker to provide a hard-wired ethernet link back to the PC, so that should be better, but at a cost! That link can also include data from lens encoders. I've just been on a course to learn more about a different system, which has a similar but more powerful unit on the camera to send data back to the PC.
@@pogglemoose That sounds like a game-changer. Since I didn't want to paint my house green (like a coward) I started by using Kinect depth cameras to create the chroma. But the lasers the Kinect used interfered with the trackers so much they'd float away instantly. In the end I went tracker-less. The MR app 'LIV' uses some voodoo to get an excellent calibration in 30 seconds, so doing that after each camera move was easy enough. But I haven't got 4 actors staring at me between shots.
Absolutely love this - fantastic work and it's great to see how you learned and improved your techniques as you went along. Can't wait to see more from you :)
Awesome job, it's great to see the creativity! About the 'choppy' camera movement with the tracker, I think that is a solvable problem. If I remember correctly the trackers track position at 200hz but the headset works at a 1000hz (to make the vision more smooth). So it should for sure be possible to smoothly track a camera with the vive system! You can perhaps even 'upgrade' the trackers to 1000hz.
This was an awesome video! Great overview of the process: very informative! I definitely want to make a show like this, but I ditched greenscreen for mocap a few years ago.
It’s incredibly helpful seeing companies starting at square 1 and seeing them take each step. Congratulations on getting your series made, looking forward to seeing more!
Exactly. Really inspiring
Thanks!
This is really incredible, i'd love to hear more about the education behind pairing virtual production and small budget filming, it's pretty amazing
I used to be in this field then I quit for 4 years from my mental health. I was too scared to back to this field again. Now after your inspiration which you made things so possible... thanks dude
Brilliant video, you perfectly captured the frustations a lot of people had at the beggining of the lockdown and trying to learn virtual production on the fly, failing a lot on the way.
Perhaps the next step would be to invest in some hue-controllable lights, to make compositing easier!
I recently did a very low-budget music video where I built two walls of LED-strips and used pixel-mapping to get lighting which matched a rear-projected background. It really sold the immersion. So I'm sure something similar might be able to work for them too.
Yes but Hue Lights arent really the key here.
I imagine that soon the lights will communicate with the system and set the right hue automatically (if it's not already possible).
@@undergroundo that's what Martin Jones just said. Pixel mapping on the led lights.
The UA-cam gods sent me here. This is freakin amazing. I've been looking for alternatives to on-set filming for forever. I can't animate or draw, but I can definitely act like an idiot in front of a greenscreen.
Woa… ok I just clicked into it and was Shazaming the intro music and noticed only on the green fade over that this was already VP. Crazy feeling.
Thanks dear god, I struck gold today ! Huge love and respect to you guys for Sharing this !
And Jim Henson was revolutionizing the virtual production studio way back in 1987 for The Storyteller, and in 1989 for The Jim Henson Hour. Kind of amazing how far we've come in making the tech smaller-better-faster-cheaper, but the results from using chroma screen look exactly the same today, but in 4K. Volumetric studios are the best virtual ones, as rear projection work has always been better than chroma screens, and now we're taking it further with video walls and tech that tracks the camera in near real-time. Very exciting stuff! Thanks for this generous BTS video!
Thanks for also sharing some timings, which gives a better feeling for how long a start in virtual production really takes.
This is incredible Phillip thanks
It all came together with the last gritty shot with the blue cast on the talents face - brilliant!
Wonderful insight! Thanks for sharing!!!
Small and clever productions is an extremely accurate name 😂
Incredible achievement. I'm sure you've got lots of other things to do (that help you pay the bills), but a tutorial on how you did it would be fantastic. Or even a list of the links you found most hopeful.
This video just changed my life
The jittering is absolutely not a limitation of the VR system. I don't know what caused this in your case, but you can definitely get a smooth image with the Vive tracker on your camera.
Anyway, this was terrific! Virtual Production evolves so quickly, it's astounding! Last year we had to manually calibrate the camera, now we have a simple, almost automatic calibration system.
Here's a few advice :
- Virtual Production is even more interesting when you move the camera. For that, you need to make sure, in the UE scene, that the virtual screen (the rectangle on which you project the actor) keeps facing the camera. That way you can shoot scenes with a moving camera, instead of just fixed shots.
- You can get shadows automatically with the proper material settings on the virtual screen. No need to put it on each frame manually! The limitation to that though is the shadow is projecting a 2D surface, so there's an angle at which point it doesn't feel like the natural shadow anymore ; but that's just a limitation to keep in mind when working on the lights! Aside from that, you get real-time accurate shadows.
This is simply Great! Thank you for sharing your journey!!!
I love this showing of the process!
thank you very much for sharing this. good luck with your show.
Inspiring and cool!
Love it! Thanks for the shout-out!
Really good video. One very cheap option for camera matching is to control the real camera with the virtual camera. A pan and tilt on a tripod controlled with an arduino uno then mapped to the scene camera is simple but effective, certainly as a rig to "play with" as a learning exercise. I use Blender and OBS studio.
A brilliant overview of Virtual Production…. and interesting real world example.
That's way more impressive than I initially thought before clicking on the video. Going to learn more about this stuff !!
I jump on my side to the terrific but exciting VR world in France. Great job guys
Thanks for posting. Really generous of you 💗
Love this! I stream VR games in mixed reality, and do sketches too. The fact the tech is colliding in so many people's minds is brilliant. Glad production tools are catching up.
Realising the 3080 Ti graphics card in my machine was powerful enough to do something like Mando, and in real time, was a game changer.
Thanks for sharing your journey!
How is this trick with qr code and camera positioning called? Cant find it anywhere
Greg Corson has done a couple of tutorials on this now.
Thanks a bunch for the video, really amazing! I want to dabble in such productions myself, but it'll still be a while until I can do so
I don't know how I came across this but you guys are amazing for having made such a detailed, comprehensive BTS on this process. I shot my first VP commercial last year and enjoyed it.
Tundra are making a big tracker that is designed to replace the Vive one for virtual production and Vive are working on some hardware to support this stuff too. Exciting time to be into VR tech!
Very well done, and very useful information. Thanks for sharing
this video explained omse concepts abot virtual production much better than others.
Many thanks for the explanations …
Very inspiring stuff! Always great to see piers at the BBC pushing modern practises. I hope to shake your hand one day
You should get some low res but bright LED Panels with diffusion on them, that are linked to live feeds of virtual cameras so that you get some interactive lighting. Ian Hubert talked about similar stuff on his Patreon a while ago.
So much problem solving. Wonderful storytelling. Thank you for inspiring and showing what’s possible
Wow, UA-cam has just told me about this now so I'm a bit late to the par-tay! The mandalorian volume is actually new application of a very old VR space called a cave as opposed to the HMD we are now more used to.
As you say technology is moving rapidly in this area, I saw a package from Quanta systems thar used its RGBWW tubes that to cast light in a similar manner to LCD panels but at much less cost. Thus getting the real world sense of the human interacting with the environment but without the cost. Its interesting that you didn't look at using rear/front projection especially to get the light scatter on the humans.
More videos about what you learnt or want to achieve would be interesting
But thanks for this
I forgot to ask did you look at all the great work done on this by Robert Rodriguez especially with his use of green screen with Spy Kids & the Sin City series??
Thank you very much for sharing!!!
Really fascinating. They say you should never let daylight in on magic, but I'm grateful! We use straight CSO / chromakey for our training programs but this is truly astonishing! Congratulations!
Excellent stuff, was very interesting to see the BTS and see a step by step process. Cheers for sharing
Thanks for sharing! Really inspiring.
this is soo cool. Hats off to you gents. Love this
Well done guys. Fun to watch and the tracker hint is nice too.
Cheers
Very nice job. Thank you for sharing.
That's really cool! Makes me want to play around with some stuff!
What a time to be alive! Can't wait to be talking to some friends about a Tv show in 10 years, and i'm just like " Oh ya Them? I saw how they started " :D
Awesome you have a team like this
Very interested to see more of this kind of content
Really nice overview! Congrats on the BBC show.
Thanks for the mention in the description. :-)
Looking forward to seeing what you do next!
This is great, thanks for sharing.
Really cool and inspiring!
Thanks for sharing!
This is really excellent. I do a lot of work in After Effects and Cinema4D for Augmented Reality. But it’s all post production.
Unreal Engine is something I’d love to get involved more in with and this is great video for starting with it. Thanks so much
This was great guys! I want to do this myself (in Canada). Thanks for sharing your process.
I'm sure that virtual production is the future. Technologies like The Volume as well as use of Unreal Engine like you've done so here. They're exciting times we live in!
Very very inspiring. Thanks for sharing
This is fantastic! I'm about to make the jump back into live action and am really eager to get my head around Unreal 5 in an FX context. It would be great to be able to have comedy sketches having access to real-time FX in their scenes, hell even if it's as cheesy as South Park using live action explosions in the scenes there could be great comedy in it.
This is amazing! Must be a lot of fun once you got it all worked out.
Underrated
I want more!!! Show me how I can start today. Just like you all started. Or make a video on what you think the next tech step you think would need to be.
Wow. Incredible growth
Thanks for the video. Very informative
Thanks for this super video! There's plenty of advices (maybe all xD) to start working in virtual production from scratches eheh
So awesome!! And inspiring. Bravo!! Hello from Melbourne, Australia 🤙
awesome overview, thanks so much for this :)
Brilliant video, excellent work, really interesting!
Just a suggestion, but whilst robot arms are indeed expensive, there are still several good motorized sliders available (even a couple that support the Mini Pro) for reasonable pricing.
Amazing 👏. Just wow
Excellent. I love seeing problems solved on a budget.
This is truly amazing!
that is pretty cool - thanks for sharing! Super interesting
VERY informative. Thanks for the insight ❤
This is amazing I had no idea that a lot of this software that I've used is capable of this stuff
Thanks for sharing! I've been trying to figure this whole thing out and there isn't much straight forward info out there.
Congrats guys!
it's really useful thank you , will have a lot to ask later
Great work guys!!!
This was amazing and so insightful!! Thank you so much for sharing! I'm based up in the North East of England and love watching and seeing content like this. Hope to one day do EPK in film and TV and so seeing all of your BTS is really awesome and inspiring. Keep up the great work and will definitely check out your series on iplayer! :D
Superb video! Thanks for sharing! Have been doing lots of VR and would love to dive into this area for some productions later on
This is an amazing use of virtual reality!
Great video, thanks for sharing and congratulations on the results!
Amazing.. Thanks for sharing this video
Incredible!
This is incredible !!
A really good and useful video, thank you
Amazing job, wouldn’t it be cool to have a character in animation in this interact with a real one like this?
This is something Disney (and the team behind Lazy Town too!) has been doing for a few years now. Off the top of my head, the Muppet crew uses a mocap puppet rig in "Earth to Ned" that they use to drive a CG character in Unreal that gets comped into a scene with real actors. That way they can get all the eyecandy of CG animation with the spontaneity of their classic puppetry.
Awesome video and I loved seeing how you worked things out on a small budget with a great result, thanks for sharing!
Loved everything about this video, very informative and entertaining. Cheers.
The Vive trackers have such weird issues. The first gen were great, but the second gen, while ostensibly better, had a lot more 'wiggle', or could be off by a solid 2CM. And these problems could be caused by anything from the USB port not being fast enough to something reflective on set confusing it. You can minimise it by using 4 lighthouses instead of 2, but that doesn't totally mitigate their issues.
They're great bits of kit, and those original errors you show look more like update speed or rendering issues than tracking issues (long USB extender?), but they're so fiddly you're right to get away when you can.
You're so right about the Vives being fiddly and frustrating at times. Sometimes they would work brilliantly, giving very smooth movement on even a complex map, other times they would be really jittery, losing their orientation as well sometimes. We eventually put the wireless dongles on the camera, as close as we could to the actual trackers, by using long USB cables back to the PC. This did sort out a lot of the problems. I see the new Vive Mars system uses a box under the tracker to provide a hard-wired ethernet link back to the PC, so that should be better, but at a cost! That link can also include data from lens encoders. I've just been on a course to learn more about a different system, which has a similar but more powerful unit on the camera to send data back to the PC.
@@pogglemoose That sounds like a game-changer. Since I didn't want to paint my house green (like a coward) I started by using Kinect depth cameras to create the chroma. But the lasers the Kinect used interfered with the trackers so much they'd float away instantly.
In the end I went tracker-less. The MR app 'LIV' uses some voodoo to get an excellent calibration in 30 seconds, so doing that after each camera move was easy enough. But I haven't got 4 actors staring at me between shots.
Thanks for this video, this is really cool to see how you progressed!
Great work I'v need the realsense to my studio :)
U also happy 2 share knowledge 💪❤
Really impressive!
Absolutely love this - fantastic work and it's great to see how you learned and improved your techniques as you went along. Can't wait to see more from you :)
Awesome job, it's great to see the creativity! About the 'choppy' camera movement with the tracker, I think that is a solvable problem. If I remember correctly the trackers track position at 200hz but the headset works at a 1000hz (to make the vision more smooth). So it should for sure be possible to smoothly track a camera with the vive system! You can perhaps even 'upgrade' the trackers to 1000hz.
Thanks sooo much for this!
Great work guys 👍
This is very nice! Its incredible
This was an awesome video! Great overview of the process: very informative! I definitely want to make a show like this, but I ditched greenscreen for mocap a few years ago.