Whattup green screen purists! I have more in depth tutorials coming - new never before seen techniques on the way! If this video helps you please considering subscribing! Even if you never watch a video of mine again, it actually helps me greatly. My insta is @andrewdileo but I also answer any and all questions in my comments!
If anyone is struggling to find the USD Stage Editor, what worked for me was to enable the USD Importer in the plugins menu. Thank you so much for sharing this awesome workflow Andrew!
Why?! Why this post doesn't have more views??? This is the type of information that truly helps others, speeds up the creative process, avoids mental breakdowns, and unnecessary firings.
man you left out so many important steps here, at least for a beginner like me but now a week later I am actually thankful for that because I figured it out and learnt SO MUCH along the way trying to get it to work and I finally did! If I quit I'd probably hate you so much hahah, thanks!
This is an awesome video showing the best workflow from Blender to UE5. But please make an in-depth tutorial for how you tracked this footage so well. Because the background is so blurry but you managed to track it. It would be like a full course if you do a tracking video!
@@cms3d really happy I could help. I can’t stress enough that if you make a hard push with this virtual production stuff right now, you will be leaps and bounds ahead of other people in 2 years time. I’m doing a massive project and have been screen capping some more advanced workflows, so another tut soon!
@@cms3d another fun idea, which I will be doing a tutorial for, includes working with transforms on your root parent, you can do a lot of fun things like shoot someone in a static car and make it look like it’s moving, even if the shot has advanced camera movement in the original. Really all you need is a solid track and a USD file and the possibilities are endless. Endless dude! Dream big.
wow never knew you could do it via USD, always thought you had to use the blender to unreal script thingy. did you end up doing the composite in unreal engine itself or are you rendering out the plates from unreal and compositing elsewhere?
Yeh neither did I, this solution is definitely the best, believe me I have looked everywhere! Compositing done in Final Cut by Director…but you can DEFINITELY export no problem with the MF KEYER node in the material you create! If you don’t, it’s important to tick off the render visibility in the details panel under your media bundle BP attached to your camera!
Every other video I've seen that explained how to get footage into unreal required that I use some other expensive device while shooting to get camera movement information, and I always asked myself... WHY? I knew camera movement was something you could track just from the footage, so I'm really glad I came across this video
Thank you! Yeh, this is only necessary if you want to do this in real time, with realtime tracking input. I will be doing some videos on that workflow too -- but I prefer to help people find the free or cheap alternatives!
@@andrewdileo I really appreciate it! I have lot's of big ideas that would just not be possible without a workflow like this, and being able to utilize unreal and blender to create environments just completely opens up a world of possibilities, as well as next level control in filmmaking. You just earned yourself a follower on both youtube and ig, I'm beyond impressed with your portfolio. I know I got a lot to learn but I can see how everything pieces together, and it's beginning to click in my head how a lot of vfx shots are made. Anyways, I can see the end goal, can't wait to be able to put myself in worldly environments. I have a lot of music video ideas , and a lot of them just would not be possible without this sort of vfx pipeline. Cheers Andrew, perhaps one day we'll meet when I make it big time in the industry, but for now, just wanted to leave this comment and let you know that you sparked a tremendous trajectory for me!
@@TristanA-d2b Dude of course! I wish you were in my area I'd bring you in and teach you what I know with that attitude! This is a really solid time to get going with visual effects. More tools and resources than ever! You won't regret subbing, I have been making a really hard push with my video business so I have enough money and time to post proper tuts more often. Anyways many thanks from the bottom of my heart, really means a lot to me that I can inspire someone. My goal is to learn and distill as much knowledge as I can to empower someone like you to make really amazing things!!! My one piece of advice I'd like to give to you is that we can storyboard, block out shots, do pre-viz...but the bottom line is that none of us ever truly have a plan with a shot. We say yes to the idea, and we problem solve to make it happen! So just try something and see where it takes you. Cheers friend.
Every couple weeks I check to see if there is a new video for this workflow 😩, finally something...🙏 There is a ton of stuff and experience compacted into this video!
USD Stage Editor is directly under WINDOW instead of under the Virtual Production tab on my setup (MAC). In case anyone else is trying t figure this out!!
@@AhmadKadi10 I think I need a follow up video. Long story short, the media bundle blueprint is available when you click film/virtual production as your starter project. I don’t think it’s available if you choose something like third person game.
Congratulations on a very good workflow! Note: To display the "BP Media Bundle... (Media Bundle Package)", you must activate the Media Framework Utilities in PLUGINS.
@@Vilaroucas honestly even that doesn’t work anymore for me. The best way to do this is to just create a project using the “blank” television/film template. And if it’s a downloaded environment, just migrate everything over to a new blank television and film template project.
nice video man, i would really want to dive in a virtual production unreal engine carrier but i need a course of high value from a good calibre like you
your video really help me solving tons of problems !!! thank you so much!!! from now on i'm gonna explain how to make media plane like you. create a media player create texture and....just look it up lol
Do you actually need help with that, or is that a joke? Hope this truly was insightful! Happy to make a video breaking down the media plane in more detail if anyone is interested!
I have a 3090, 40xx series will work better. VRAM is most important. I recommend 24GB V-RAM minimum if you want reliability especially if it's your profession and time is a factor. (for those that don't know RAM is different than VRAM). You don't need a 4090 but that's going to save you some pain. I'm personally waiting for 50 series Nvidia GPU's to drop to buy a new system, and I always buy pre-builds (time is money)
@@kisaki1198 video come out very soon where I show how to set these up - but I find it’s actually better to have your tape a shade brighter or darker - as long as it is green! Or same color as your chroma screen! This will help create more contrast, because if it was a little blurry behind you, you wouldn’t be able to see tape that is the exact same color, right?
Thank you for sharing your workflow, it's very rare to find content like this for Unreal Engine. Can a camera tracking plugin like fSpy be used for a project like this? And for final compositing and keying, do you do it in Unreal Engine or in post-production software like After Effects or similar software?
Thanks, friend, yes, you can absolutely use fSpy… The process would be to open your .fspy file in blender, add a floor plane and then export the scene. When you’re exporting the USD file, you just don’t really want to bother ticking “animation” checkbox if it’s only a static image! Keying goes to post production software, but you can render it out right in UE5, just won’t be as clean! But for some shots, you can get away with it
One question! If I am planning to color grade after rendering UE project, should the keyed footages be in ACES format? (The footages played via media player) Appreciate any help!
@@whasuklee great question man. The answer is shockingly simple: unreal engine is an sRGB native linear workspace. If you want to use an ACES workflow and want to do your keying in unreal or bring in pre-keyed footage you should load sRGB tagged footage into unreal. When it comes time to export, just use your OCIO config file in the color management tab in the movie render queue. This way you’re not applying an ACES conversion TWICE. Only once consistently across your entire scene. Then everything should convert back to linear display spaces just fine. Hope that helps!
I get that doing the camera tracking in blender is probably what makes the most sense, but I wonder if it would be possible and/or make sense to use an iPhone (or maybe even an android phone) 3D tracking on the camera rig to get the camera movement information instead? I wonder if that would make it possible to do everything directly in Unreal Engine directly? (Provided I don't know anything about any of this and I am just wondering)
@@FuwakamiMana yes, you’re on the money. Using trackers that are placed on the physical camera is something that a lot of research and engineering dollars have gone into. The problem comes when you’re trying to make sure your frame rates are synchronized. iPhones are hard to genlock, which is the process of synchronizing frame rates between devices such as audio recorders, cameras, and software, like unreal engine. The other issue is that you need to store that data and keep it consistent with your various takes, which is a nightmare. Lastly, iPhones do a very poor job accounting for micro-vibrations, which truly make or break a track. At least in any moderately professional setting, you want to make sure that all those tiny little bumps and jitters get factored into your camera solve. That’s all I got on that! Of course there’s other various software that you can use, including syntheyes and 3-D equalizer. For me I just tend to use blender and when you really figure out exactly how to carry these shots over to unreal, you’re typically not disappointed.
@@andrewdileo thank you very much for this detailed explanation! It’s very appreciated! I learned a lot from this video, and while I don’t think I’ll ever attempt to make anything professional or realistic, I think I learned a few useful things! ＾＾
@@Loko_Gaming for a production of this size, no. For commercial purposes, that can be beneficial. Virtual production should be done on a project by project basis. This is the traditional method and allows us a lot more freedom and flexibility in post.
@@SangSync very simple, instead of translating or rotating the cinema camera actor, instead on the timeline you can add “camera component”. Usually this is there by default but if you don’t see it, you can click the + next to cinema camera actor on the timeline in select camera component. Do all your transforms to this! that way it is an additive move being applied as opposed to changing the key frames of the existing camera animation. I hope that makes sense.
I've found the solution. In Content Drawer, under "All" directory should be "Content" and "Engine". In my case, "Plugins" directory isn't visible under "Engine". On the right side of Content Drawer, there is Settings. Click it and check Show Plugin Content.
Thanks a lot for this video! I was battling with this recently. To be honest, I am battling with UE overall, as I feel like it just keeps throwing sticks under my legs while trying to achive things that are super basic and primitive to do in most DCCs. May I ask you what is your approach when adding shadows? Or overall compositing the characters into the scene so that there is a shadow, perhaps reflections. As this is only slapping the greenscreen on top of movement-matched 3D scene. Thanks so much!
Shadows are always a pain deal with, and there’s no set-in-stone method to achieve them. One method is with shadow casters- you can use the plane that is imported as the ground, and set up some rough geometry similar to your actors to cast shadows onto a white plane for compositing. Most of the time, Ben, who does the composing, will actually try to pull out the shadows that were actually on the green screen in the shot, and use those. If you have your actor inside unreal as an actual “plane”, that’s when you’ll get hyperrealistic reflections, without crazy compositing. You might just have to switch your material, playing back the video to “masked” for the transparency handling. I will make a video about this exploring some different options. Maybe one day AI will do it for us, that’s the kind of stuff I would like to see AI be used for 🥴 shadow gen
@@andrewdileo Thanks a lot for your answer. Yes, pulling out the shadows from greenscreen works in some cases, especially for contact shadows you can't really get other way afaik. However, if there's for example a wall in the virtual space behind the actor (or tree, anything) and it wasn't there when shooting the greenscreen, you're out of luck as you can't composite the ground shadow on it. At least I can't imagine that. About importing planes into UE, I tried it with Ian Hubert's method of scaling the plane with the origin at camera's space. However, I had so much trouble doing that in UE. Would really welcome a quick video on that if you ever managed to do that. I managed to make it work with some workarounds but it wouldn't render. The character would be behind the scene for whatever reason although in viewport it was perfect. This however is too limited in two way from what I found: if the camera is slightly tilted downwards which it usually is, further foot will float in the air and although visually they are touching the ground in 3D, the shadow doesn't match. Second thing is that if the shadow is supposed to be cast from the left or right, it won't work as the plane is flat obviously. Moreover, I dunno if there's any way to composite it later that way. Having a separate render layer for the character, shadow and rest. Sorry for such a long reply, I'm just kinda summarizing my thoughts and struggles trying to find some other possible way or combination of techniques. Really wonder how it's made in professional production. Thank you in advance if you have any more input to this.
@@ArtemGms yeah, I have to digest this and honestly, I can definitely get to work on a video breaking down shadows a bit more in the workflows that I would opt for personally - if you want to achieve the Hubert method of scaling the plane in relation to the camera, so the feet are clipping the ground, the best way to do that is to do it in blender itself, and then export that with the animation as a USD file and then literally just use that plane as the plane that you’re projecting media onto…I just did it recently on a Jacob Collier project for the final scene in Mi Corazon. It worked out great. I should make a video about that as well. I think everyone has studied in Hubert and is trying to channel that in unreal and I have been devoting the past several months to learning how to transfer that skill set over I will make a video devoted to just showing how to do Ian Hubert stuff in unreal lol
@@ArtemGms I know what you’re talking about as far as the other foot floating in the air, one workaround is to have a duplicate that’s only acting as a shadow caster and then obviously you would just translate that down on the Z
@@andrewdileo Sorry for the late response. That would be amazing if you could make a video about that. Haven't tried USD workflow, I tried to do it in UE. Only did the track in Blender and it was terrible. That floating feet workaround sounds interesting but you then have to somewhat merge those 2 shadows, right? Which leads me to a 2nd question. Is it possible to do render layers? So I can even achieve something like that. Having environment, greenscreen character and its shadows on different layers? So I can perhaps add extracted shadows from greenscreen in composite. Plus I'd like to have the ability to color it properly (so having the character in log). I would simply like to have it as non-destructive / flexible as possible. But I have no idea how does UE even approach such data, if it's able to keep the information in it if you rendered the greenscreen keyed plane in log and exr or something. Thanks again for your time and willingness.
@@егоркаминский-у6ъ I got you, I always click “don’t use a default cache” on that pop up. Since you’re importing, this does not matter. This is only if you are using the USD stage for persistent changes between programs.
Hey there, friend, make sure that in your content browser you are selecting “All” folder, the root folder of your content hierarchy. Also make sure you don’t have any search filters. Type in “mediabundle” and you should see “BP_media bundle_plane_16-9” pop up! That is the component you need to drag onto your camera component under your cinecam as shown! Hope that helps 🥳 Working as of 5.4 stabile release!
@@vellg5813 don’t say that! This video is sped up and I’m not some robot with this stuff, I’m also still figuring it all out! I want you to know that about a year and a half ago I sat down and opened it up. Overwhelmed, confused, not knowing basics like how to navigate the viewport and getting use to hotkeys for transforms on objects. A part of me thought I would never get the hang of it. Take it one small step at a time. Everyone who uses this software is always learning, and there’s a lot of information out there to learn from. Don’t get discouraged, appreciate your small wins first. Start with environment design, that will keep you busy and learning for a few months!!!
Life is so expensive man. I wish I could work with everyone! But yes services are in the tens of thousands, you see how much work goes into just one shot.
brooooooooooooooooooooooo 4:41 i stopped watching and only came back an hour later after laughing for 1 hour non stop....... he said "thingy.... just look it up"" hhhahahahahahahahaha
cool video, but are you copying the voice intonation of ian hubert? The pacing / voice jokes were like as if I was watching someone trying really hard to copy his style lol
Ha! Yeah….I’m really excited when I talk about this stuff, have also studied him extensively, and also it’s proven to hold peoples attention, why reinvent the wheel!
Whattup green screen purists! I have more in depth tutorials coming - new never before seen techniques on the way! If this video helps you please considering subscribing! Even if you never watch a video of mine again, it actually helps me greatly. My insta is @andrewdileo but I also answer any and all questions in my comments!
Very generous of you. Appreciate the pace
Really glad you found value in it! This has been a fun proof of concept that I should just keep going with these!
I am obsessed with this video! you guys are great
Thanks! Always happy to hear that!
If anyone is struggling to find the USD Stage Editor, what worked for me was to enable the USD Importer in the plugins menu. Thank you so much for sharing this awesome workflow Andrew!
@@ed7string no prob! Yes you need usd importer plugin enabled!
@@ed7string working on a much more detailed tut! Sub please! Also a greattttt workflow vid for shadows that you won’t want to miss!!
@@andrewdileowaiting ⌛️
@@famegame2148 I’m on it!!!! 😅😅 quality control
Thanks for this. How did you learn all of this stuff!? You are the greatest of all time.
Combination of reading suggestive documentation and happy accidents!
the compositing & color grading are ELITE we need a video for those two skills :O !!!
Why?! Why this post doesn't have more views??? This is the type of information that truly helps others, speeds up the creative process, avoids mental breakdowns, and unnecessary firings.
Thanks so much, I need to keep going! Been busy but I know it’s worth it to get this out there! MORE CREATIVE DECISIONS!
@@andrewdileo need moreeeeee videos, fr this was great
@@414SHOTiT thank you so much. I’m on it!!!
great blender, i just started learn blender building environments and this is wild alien to me but intriguing
No problem, you will figure it out soon! Believe you can do it because you can!
🔥🔥 such a great video. Thanks for all the great tips
Dude my pleasure! Keep on that grind!
Really informative video...need more blender to unreal workflow about green screen compositing
I will most definitely make a follow up! Thanks for commenting friend!
man you left out so many important steps here, at least for a beginner like me but now a week later I am actually thankful for that because I figured it out and learnt SO MUCH along the way trying to get it to work and I finally did! If I quit I'd probably hate you so much hahah, thanks!
This is an awesome video showing the best workflow from Blender to UE5. But please make an in-depth tutorial for how you tracked this footage so well. Because the background is so blurry but you managed to track it. It would be like a full course if you do a tracking video!
Andrew, you are a saviour!
Just shot more content!!! Excited to share!!
Am blown apart so true there is no information on this kind of work flow....super thankyou to you sir...god bless you with more subs
Thank you so much, feeling very inspired to keep going on these!
@@andrewdileo you could only get this level of info from CGWorld JP. Thank you so much for sharing.
@@activemotionpictures thanks! More on the way, working on a project right now
i've been looking for something like this foreverrr!
@@emersonvernon333 glad I could fill in the blank!
Brother! This is the type of workflow I was looking for, using Blender and UE on a daily basis, can't wait to try it out! Thank you so much
@@cms3d really happy I could help. I can’t stress enough that if you make a hard push with this virtual production stuff right now, you will be leaps and bounds ahead of other people in 2 years time.
I’m doing a massive project and have been screen capping some more advanced workflows, so another tut soon!
@@cms3d another fun idea, which I will be doing a tutorial for, includes working with transforms on your root parent, you can do a lot of fun things like shoot someone in a static car and make it look like it’s moving, even if the shot has advanced camera movement in the original.
Really all you need is a solid track and a USD file and the possibilities are endless. Endless dude! Dream big.
@@andrewdileo Cool idea 😲, can't wait for the tutorial 🙏💚
wow never knew you could do it via USD, always thought you had to use the blender to unreal script thingy. did you end up doing the composite in unreal engine itself or are you rendering out the plates from unreal and compositing elsewhere?
Yeh neither did I, this solution is definitely the best, believe me I have looked everywhere!
Compositing done in Final Cut by Director…but you can DEFINITELY export no problem with the MF KEYER node in the material you create! If you don’t, it’s important to tick off the render visibility in the details panel under your media bundle BP attached to your camera!
Every other video I've seen that explained how to get footage into unreal required that I use some other expensive device while shooting to get camera movement information, and I always asked myself... WHY?
I knew camera movement was something you could track just from the footage, so I'm really glad I came across this video
Thank you! Yeh, this is only necessary if you want to do this in real time, with realtime tracking input. I will be doing some videos on that workflow too -- but I prefer to help people find the free or cheap alternatives!
@@andrewdileo I really appreciate it! I have lot's of big ideas that would just not be possible without a workflow like this, and being able to utilize unreal and blender to create environments just completely opens up a world of possibilities, as well as next level control in filmmaking.
You just earned yourself a follower on both youtube and ig, I'm beyond impressed with your portfolio. I know I got a lot to learn but I can see how everything pieces together, and it's beginning to click in my head how a lot of vfx shots are made.
Anyways, I can see the end goal, can't wait to be able to put myself in worldly environments. I have a lot of music video ideas , and a lot of them just would not be possible without this sort of vfx pipeline. Cheers Andrew, perhaps one day we'll meet when I make it big time in the industry, but for now, just wanted to leave this comment and let you know that you sparked a tremendous trajectory for me!
@@TristanA-d2b Dude of course! I wish you were in my area I'd bring you in and teach you what I know with that attitude! This is a really solid time to get going with visual effects. More tools and resources than ever! You won't regret subbing, I have been making a really hard push with my video business so I have enough money and time to post proper tuts more often.
Anyways many thanks from the bottom of my heart, really means a lot to me that I can inspire someone. My goal is to learn and distill as much knowledge as I can to empower someone like you to make really amazing things!!!
My one piece of advice I'd like to give to you is that we can storyboard, block out shots, do pre-viz...but the bottom line is that none of us ever truly have a plan with a shot. We say yes to the idea, and we problem solve to make it happen! So just try something and see where it takes you. Cheers friend.
This is an amazing tutorial, and XXL was a bop
Thank you sir, more coming soon! We’ve been busy!
@@andrewdileo You about to do it XXL 🔥
Very Ian Hubert. Well done.
@@JWS1968 bruh don’t even…that’s insulting to Ian Hubert 😅 I ain’t there yet
if anyone struggling to find media bundle , try to enable the plugin Media Framework Utilities
This comes available by default as of 5.5 if you set up your scene with the "film and TV" starter project (When you are first setting project up!)
🔥🔥 Thanks Andrew! Keep the tutorials coming.
@@Apuat oh man!!! I will
That hard good creative, confuse my brain 🧠 definitely gotta get blender play around wit the program definitely good 3d animation videos
You can do it! Give it a shot!
Every couple weeks I check to see if there is a new video for this workflow 😩, finally something...🙏 There is a ton of stuff and experience compacted into this video!
Thanks!!! High-budget project pressure sure is one hell of a drug. Learned the hard way! 😂
aye i see ben prioux in the cut lol
USD Stage Editor is directly under WINDOW instead of under the Virtual Production tab on my setup (MAC). In case anyone else is trying t figure this out!!
@@KenzieRocket they moved this is 5.4 and up I believe!
Thank you so much for sharing. It's so inspiring.
hope they brought you back for their new video! if so, looking forward to a video on ATE THAT!!
@@andyjones2224 they absolutely did 🥹thnx for your continued love and support ❤️
this is amazing, I didn't understand how did you attached a greenscreen footage to the camera. I couldn't find the mediabundle thing!
@@AhmadKadi10 I think I need a follow up video. Long story short, the media bundle blueprint is available when you click film/virtual production as your starter project. I don’t think it’s available if you choose something like third person game.
Congratulations on a very good workflow!
Note: To display the "BP Media Bundle...
(Media Bundle Package)", you must activate the Media Framework Utilities in PLUGINS.
@@Vilaroucas honestly even that doesn’t work anymore for me. The best way to do this is to just create a project using the “blank” television/film template. And if it’s a downloaded environment, just migrate everything over to a new blank television and film template project.
extremely impressive
Jordan Orme should see this
@@seorin. he’ll see it! He’s been watching them - we’ve been watching him watch them! We love Jordan’s vids
nice video man, i would really want to dive in a virtual production unreal engine carrier but i need a course of high value from a good calibre like you
You can do it! I will be posting a lot more tutorials now that I’ve wrapped up the big project I’ve been working on
your video really help me solving tons of problems !!! thank you so much!!! from now on i'm gonna explain how to make media plane like you. create a media player create texture and....just look it up lol
Do you actually need help with that, or is that a joke? Hope this truly was insightful! Happy to make a video breaking down the media plane in more detail if anyone is interested!
@@andrewdileo i do really please make one
I love you I have been wanting this you have a subscriber sir thank you
Let’s goooo!
What gpu will work for a complex workflow like this? I’ve been considering Rtx 4070, will it suffice? Thanks and great work. Cheers
I have a 3090, 40xx series will work better. VRAM is most important. I recommend 24GB V-RAM minimum if you want reliability especially if it's your profession and time is a factor. (for those that don't know RAM is different than VRAM).
You don't need a 4090 but that's going to save you some pain. I'm personally waiting for 50 series Nvidia GPU's to drop to buy a new system, and I always buy pre-builds (time is money)
hey if i have setup green screen and for track point like same a green tone but like more brighter little bit, did the shoot still easy to track?
@@kisaki1198 video come out very soon where I show how to set these up - but I find it’s actually better to have your tape a shade brighter or darker - as long as it is green! Or same color as your chroma screen! This will help create more contrast, because if it was a little blurry behind you, you wouldn’t be able to see tape that is the exact same color, right?
Thank you for sharing your workflow, it's very rare to find content like this for Unreal Engine. Can a camera tracking plugin like fSpy be used for a project like this? And for final compositing and keying, do you do it in Unreal Engine or in post-production software like After Effects or similar software?
Thanks, friend, yes, you can absolutely use fSpy… The process would be to open your .fspy file in blender, add a floor plane and then export the scene. When you’re exporting the USD file, you just don’t really want to bother ticking “animation” checkbox if it’s only a static image!
Keying goes to post production software, but you can render it out right in UE5, just won’t be as clean! But for some shots, you can get away with it
One question! If I am planning to color grade after rendering UE project, should the keyed footages be in ACES format?
(The footages played via media player)
Appreciate any help!
@@whasuklee great question man. The answer is shockingly simple: unreal engine is an sRGB native linear workspace. If you want to use an ACES workflow and want to do your keying in unreal or bring in pre-keyed footage you should load sRGB tagged footage into unreal.
When it comes time to export, just use your OCIO config file in the color management tab in the movie render queue. This way you’re not applying an ACES conversion TWICE. Only once consistently across your entire scene. Then everything should convert back to linear display spaces just fine. Hope that helps!
thank you so much, this is amazing❤
Please make more videos!!!!
Okay just for you!!!!
I get that doing the camera tracking in blender is probably what makes the most sense, but I wonder if it would be possible and/or make sense to use an iPhone (or maybe even an android phone) 3D tracking on the camera rig to get the camera movement information instead?
I wonder if that would make it possible to do everything directly in Unreal Engine directly?
(Provided I don't know anything about any of this and I am just wondering)
@@FuwakamiMana yes, you’re on the money. Using trackers that are placed on the physical camera is something that a lot of research and engineering dollars have gone into.
The problem comes when you’re trying to make sure your frame rates are synchronized. iPhones are hard to genlock, which is the process of synchronizing frame rates between devices such as audio recorders, cameras, and software, like unreal engine.
The other issue is that you need to store that data and keep it consistent with your various takes, which is a nightmare.
Lastly, iPhones do a very poor job accounting for micro-vibrations, which truly make or break a track. At least in any moderately professional setting, you want to make sure that all those tiny little bumps and jitters get factored into your camera solve.
That’s all I got on that! Of course there’s other various software that you can use, including syntheyes and 3-D equalizer. For me I just tend to use blender and when you really figure out exactly how to carry these shots over to unreal, you’re typically not disappointed.
@@andrewdileo thank you very much for this detailed explanation! It’s very appreciated! I learned a lot from this video, and while I don’t think I’ll ever attempt to make anything professional or realistic, I think I learned a few useful things! ＾＾
can you make a guide on doing it without all that gear ?
Absolutely. I’ll do a DSLR/ pop up green screen version of this!
@@andrewdileo great waiting for it 💯💯💯
is it not better to have some sort of motion sensor taped to camera and "record" camera motion to not track these + on green screen?
@@Loko_Gaming for a production of this size, no. For commercial purposes, that can be beneficial. Virtual production should be done on a project by project basis. This is the traditional method and allows us a lot more freedom and flexibility in post.
Thank you for this!
Glad to help!
how did you move the camera in Unreal? when i try to move..it always snap back to same place
@@SangSync very simple, instead of translating or rotating the cinema camera actor, instead on the timeline you can add “camera component”. Usually this is there by default but if you don’t see it, you can click the + next to cinema camera actor on the timeline in select camera component. Do all your transforms to this! that way it is an additive move being applied as opposed to changing the key frames of the existing camera animation. I hope that makes sense.
@@andrewdileo ok thank you very much
Awesome tips.
At 4:22, I couldn't find BP_MediaBundle_Plane_16-10.
Where can I find it? ...
I've found the solution.
In Content Drawer, under "All" directory should be "Content" and "Engine".
In my case, "Plugins" directory isn't visible under "Engine".
On the right side of Content Drawer, there is Settings.
Click it and check Show Plugin Content.
@@whasuklee yup, gotta click “all” and I believe it’s only in there when you select “film and television” as the project template
Thanks a lot for this video! I was battling with this recently. To be honest, I am battling with UE overall, as I feel like it just keeps throwing sticks under my legs while trying to achive things that are super basic and primitive to do in most DCCs.
May I ask you what is your approach when adding shadows? Or overall compositing the characters into the scene so that there is a shadow, perhaps reflections. As this is only slapping the greenscreen on top of movement-matched 3D scene. Thanks so much!
Shadows are always a pain deal with, and there’s no set-in-stone method to achieve them. One method is with shadow casters- you can use the plane that is imported as the ground, and set up some rough geometry similar to your actors to cast shadows onto a white plane for compositing. Most of the time, Ben, who does the composing, will actually try to pull out the shadows that were actually on the green screen in the shot, and use those. If you have your actor inside unreal as an actual “plane”, that’s when you’ll get hyperrealistic reflections, without crazy compositing. You might just have to switch your material, playing back the video to “masked” for the transparency handling. I will make a video about this exploring some different options.
Maybe one day AI will do it for us, that’s the kind of stuff I would like to see AI be used for 🥴 shadow gen
@@andrewdileo Thanks a lot for your answer. Yes, pulling out the shadows from greenscreen works in some cases, especially for contact shadows you can't really get other way afaik. However, if there's for example a wall in the virtual space behind the actor (or tree, anything) and it wasn't there when shooting the greenscreen, you're out of luck as you can't composite the ground shadow on it. At least I can't imagine that.
About importing planes into UE, I tried it with Ian Hubert's method of scaling the plane with the origin at camera's space. However, I had so much trouble doing that in UE. Would really welcome a quick video on that if you ever managed to do that. I managed to make it work with some workarounds but it wouldn't render. The character would be behind the scene for whatever reason although in viewport it was perfect.
This however is too limited in two way from what I found: if the camera is slightly tilted downwards which it usually is, further foot will float in the air and although visually they are touching the ground in 3D, the shadow doesn't match. Second thing is that if the shadow is supposed to be cast from the left or right, it won't work as the plane is flat obviously. Moreover, I dunno if there's any way to composite it later that way. Having a separate render layer for the character, shadow and rest.
Sorry for such a long reply, I'm just kinda summarizing my thoughts and struggles trying to find some other possible way or combination of techniques. Really wonder how it's made in professional production.
Thank you in advance if you have any more input to this.
@@ArtemGms yeah, I have to digest this and honestly, I can definitely get to work on a video breaking down shadows a bit more in the workflows that I would opt for personally - if you want to achieve the Hubert method of scaling the plane in relation to the camera, so the feet are clipping the ground, the best way to do that is to do it in blender itself, and then export that with the animation as a USD file and then literally just use that plane as the plane that you’re projecting media onto…I just did it recently on a Jacob Collier project for the final scene in Mi Corazon. It worked out great. I should make a video about that as well. I think everyone has studied in Hubert and is trying to channel that in unreal and I have been devoting the past several months to learning how to transfer that skill set over I will make a video devoted to just showing how to do Ian Hubert stuff in unreal lol
@@ArtemGms I know what you’re talking about as far as the other foot floating in the air, one workaround is to have a duplicate that’s only acting as a shadow caster and then obviously you would just translate that down on the Z
@@andrewdileo Sorry for the late response. That would be amazing if you could make a video about that. Haven't tried USD workflow, I tried to do it in UE. Only did the track in Blender and it was terrible. That floating feet workaround sounds interesting but you then have to somewhat merge those 2 shadows, right? Which leads me to a 2nd question. Is it possible to do render layers? So I can even achieve something like that. Having environment, greenscreen character and its shadows on different layers? So I can perhaps add extracted shadows from greenscreen in composite. Plus I'd like to have the ability to color it properly (so having the character in log). I would simply like to have it as non-destructive / flexible as possible. But I have no idea how does UE even approach such data, if it's able to keep the information in it if you rendered the greenscreen keyed plane in log and exr or something. Thanks again for your time and willingness.
"This project has no default USD Asset Cache set"
When trying to impromptu in UE5 this error appears
@@егоркаминский-у6ъ I got you, I always click “don’t use a default cache” on that pop up. Since you’re importing, this does not matter. This is only if you are using the USD stage for persistent changes between programs.
Great Video
Thanks so much!
Unreal Engine 5 как добавить видео четкости хромакей видео в проекте???
how do we do shadow catching or reflection catching
Video coming soon on this exact thing. There’s no one magic trick, just different methods.
Very informative video! USD Stage is not showing up for me. I have enabled the plugins and all :/
Thanks!
@@Soundar24x7 dammit, yes they moved things around. Now it’s located under “virtual production” in the same menu.
Or in some submenu, they just moved it!
Thanks! I'll try seaching for it
in the mean time, it works in 5.3!
I can't see the BP mediaBundel file even with the plugin loaded. Did it get removed from unreal?
Thanks for pointing this out to me, I’ll look into! Only have the beta of 5.4 but I’m updating now.
Hey there, friend, make sure that in your content browser you are selecting “All” folder, the root folder of your content hierarchy. Also make sure you don’t have any search filters.
Type in “mediabundle” and you should see “BP_media bundle_plane_16-9” pop up! That is the component you need to drag onto your camera component under your cinecam as shown! Hope that helps 🥳
Working as of 5.4 stabile release!
@@andrewdileo I found it. It just seems to depend on what starter fie I use. Must be something to do with my install.
@@nickhenderson5781 that’s another thought I had. Use Film as a template - I always start with blank
@@andrewdileo good advice. one more question. is there a way to improve the quality of the plate on playback? Mine looks pretty average
So sick what the hell
Noe just gotta follow it up lol
@@gronnip HA yes soooooooo soon promise. Later this month, got something super special for yall here
I love you
I love you too.
How much do you charge to make a music video?
@@kennethnoriega4546 I just do my part on videos for now. This was directed by Ben Proulx. I ain’t cheap, but I get the job done typeshiii
OMG!!!! can somebody PLEASE guide me to the very beginning? i just heard about Unreal a month ago. From this video i will NEVER be able to use it.
@@vellg5813 don’t say that! This video is sped up and I’m not some robot with this stuff, I’m also still figuring it all out! I want you to know that about a year and a half ago I sat down and opened it up. Overwhelmed, confused, not knowing basics like how to navigate the viewport and getting use to hotkeys for transforms on objects. A part of me thought I would never get the hang of it. Take it one small step at a time. Everyone who uses this software is always learning, and there’s a lot of information out there to learn from. Don’t get discouraged, appreciate your small wins first. Start with environment design, that will keep you busy and learning for a few months!!!
@@andrewdileo thank you for the words of encouragement. I'm so tired of my videos looking like the 1980's. I will keep studying
It was so fast, but THANK
@@chrissoba5104 here to save time!
andrew do you work with upcoming african artist in terms of editing or your budget is off the roof
Life is so expensive man. I wish I could work with everyone! But yes services are in the tens of thousands, you see how much work goes into just one shot.
@@andrewdileo i agree man salute you good job
Can’t import usd camera to unreal .. its sucks
@@ApexArtistX maybe I can help…what is your current procedure to import?
@@andrewdileo camera tracking animation importing into unreal from blender ..
@@andrewdileo blender camera tracking to unreal. Baked camera still no animation import its a bug in unreal
ok
brooooooooooooooooooooooo 4:41 i stopped watching and only came back an hour later after laughing for 1 hour non stop....... he said "thingy.... just look it up"" hhhahahahahahahahaha
I need a whole video devoted to that one thing haha!
cool video, but are you copying the voice intonation of ian hubert? The pacing / voice jokes were like as if I was watching someone trying really hard to copy his style lol
Ha! Yeah….I’m really excited when I talk about this stuff, have also studied him extensively, and also it’s proven to hold peoples attention, why reinvent the wheel!
@@andrewdileodoing great 👍🏽 nice video
So NOOB DONT KNOW WHAT AE TO UNREAL (C4D PIPELINE IS) AND WHAT MEDIA PLATE IS