To clarify, I claim that Stencil Layers/Render Layers are better than UNREAL'S Cryptomatte/Object ID exclusively. Reason being, Cryptomatte/Object ID is pretty limited in its current state in the Unreal engine. And in order to get proper depth of field and motion blur, you need to render out Zdepth and Motion Vector passes, and combine them in a relatively complicated way, along with some math to get yout Cryptomatte to show up properly. Stencil layers are much simpler and cleaner way to work. Other renderers like Arnold, Octane, Vray, etc, have vastly superior Cryptomatte support.
I tried to follow unreal documents to combine depth and velosity for motion blur but I am new user for nuke and I can't get the point I hope to make tutorial to clearify these math settings and if the new update in 4.26.1 provide new enhancement for post production work Thanks for al your effort
@@stepscourse4727 Yeah it really isn't super obvious, and Epic is not very clear in their documentation about how to do this. I got it to work, but it's really not super well explained!
In another video you mentioned Zdepth not having a 32-bit option in MRQ. Does that mean there is no viable way to use the Cryptomatte, because you can't add proper depth of field/motion blur back in?
When I tried to add scene depth pass to movie render queue , I only get black layer , there is no any depth in it in gray or white , what control this in the scene, Although when I get world depth only pink colour
Incredible Incredible Incredible. Between you and Winbush, the sky is the limt for what UR4 can be used for. You've inspired me to ACTUALLY start a favorites list for learning UR4. Anyone learning Unreal for FX and graphics needs to watch this video. Well done!
Thanks Will! I've started learning Unreal recently, and coming from a compositors point of view I feel like there's a lack of tutorials on higher production level workflows and how to use Unreal together with Nuke - and what I can expect from the Unreal renders inside of Nuke. So all of your videos are greatly appreciated and useful, big thanks!
Hey Tim! Hope you're doing well over in Oslo! You're absolutely right, there is so little content for the compositing side of things. The main reason for that is because, well, simply put it's kind of messy right now. Most of the tools for compositing have been duct taped into the engine in a way. There's "support" for exporting lots of things but it's kind of hacky, and doesn't feel quite fully fleshed out yet. Like Cryptomatte for example. It does work, but it isn't a true crypto, and getting crypto to get motion blur and depth of field to work with it is kind of tedious. There is no 32-bit support either (although in theory the new 4.26.1 release does support it, it would seem like it's more of a 24-bit file under the hood, but I can't confirm this). I do think things will change for the better in Unreal Engine 5. I'll certainly be releasing some more Post-Production oriented videos so stay tuned!
@@WilliamFaucher Yes duct tape is a perfect metaphor for how I've experienced it so far lol, I'm constantly questioning myself if I've done something wrong. - But yes let's hope it's getting more streamlined in that aspect soon, I find myself stumbling around in Unreal quite a bit trying to do "comp stuff" while really wanting to do it in Nuke instead. But again your videos are a gold for helping with that exact problem, thanks for carrying the torch!
@@Bobbiloff You're not the only one to be stumbling around! It is important to keep in mind Unreal is first and foremost a games engine, not a renderer designed for VFX work. Any kind of support geared towards VFX artists has been added in as a sort of afterthought, I don't believe UE4 was designed with VFX in mind. UE4 was released back in 2014-2015, I'm not sure the devs in their wildest dreams expected it to come this far in terms of visual fidelity so it isn't totally surprising that things aren't the way we'd expect. Now that they've had a chance to see what the demand is, UE5 has the potential to rectify this. Fingers crossed.
Very useful tut...what I really like about your videos is your idea about an issue ..you go straight out to the topic and clarifies it very well...I have learned a lot from your video tutorials William...thx
Hello William, thanks for all your help. One question, with Unreal 5, i'm trying to get layers on a multilayer EXR using Path tracing. but I'm failing at it. is there any useful trick for this to work on Unreal 5? thanks
So, will this work with dynamic stuff like fx? Say I want to TS some falling dust for motion blur, but I don't want other elements in the scene to be affected the same way. Could I put the effect in a Stencil Layer and have it work properly?
Love this stuff, keep it up. You're videos are helping me alot to make the transition from Vray to Unreal for television work. This is without a doubt the future of production work.
Hi, does anyone know if this is broken in UE5? i get my background layer with a mate of my character on it and one of my layers thar is supposse to have an object renders empty :(
Heyo. Nice video. VFX comper here. The black haloing is due to the fact that your background image has the foreground stenciled from it. So there will always be an area where the fine reduced value edge pixels from both FG and BG don't add up. If you were building a proper comp, (say a matte painting as your background) you'd have colour values where the two edges overlap that the foreground overlaps. You could try unpremultiplying the background which might squeeze out a few pixels of solid colour so that overing works correctly. Alternatively. If you were doing layering as simple as yours. You could just render the background without any of the foreground stuff visible. So you get the whole image. Plusing can cause issues down the line if used incorrectly. You can also start to introduce super pixel values and nans the more layers and FX you apply. Like with lightwrap,blooms etc.
Hi there! Thanks so much for chiming in and sharing your explanation of things, it is much appreciated! The main reason I used plus here is because that is what was recommended by Epic themselves, so, I figured they're smarter than me in regards to this. That said, your thoughts on the matter will have me experimenting with this further. Completely agreed in regards to the issues you may run into with plus, it's not what I typically would use in a production. Thanks again!
Hello, I am not sure I you still read the comments after all this time - really cool that you reply to so many people in the first place :) I am pretty sure I followed every step like you did - but I still do have, perhaps not a black halo, but apretty jagged edge around my masks... Are there other things that could cause my masks to break? The setting you have to change in the project settings gives me the options 'allow through tonemapper' and 'linear color space ' - same problem with both... I do have also TemporalAA.Upsampling enabled for the better DOF. Also my Sky that comes from the SkyAtmo is not in the defaul layer - which is strange.
Helo.. thank you for the video! btw, i have a question.. how to import the render result into After Effects? cause everytime i import, it always imported as 1 layer (BG). i cant separate between FG and BG.. thank you
I am so happy you posted this, but one question? Have you ever tried this workflow but instead of NUKE have you ever gone to Photoshop instead to use the layers for paint over? Im running into some issues with the layers on the Photoshop end of things. Any advice would be 1000% welcome. Thanks!
Hey Will, I was wondering; is there a way to render out using this technique but not having the MRQ render a Final Image alongside a stencil layer? I am only trying to render my layers individually without the default layer rendering, and it's becoming quite taxing on my system in terms of time restraints. I've turned off Add Default Layer, yet it is still rendering the default layer. Any way to counteract this? Thanks!
Hi William, I followed all your steps and it still won't render with an alpha mask. Although, I haven't been able to confirm the 'Disable screen percentage' - where is this found? Thanks!
Just getting into proper compositing so trying out nuke after Unreal in my workflow and this video is a lifesaver! Thanks! Edit: Just set my first movie render queue render going with path tracing and a couple layers on a simple 10 sec/240 frame shot. The estimated time of 6 hours just made my heart sink. Worth it in the end though.
Hi, I spend hours to get it work in UE5.0.3 but I just get the complete scene rendered out. No layer are created. Do you have an advice for me how to get it work in UE5?
Amazing tut as always! Got an issue with trying to render God Rays on transparent background (using Exponential Height For w/ Volumetric Fog) and can't seem to make it work. It does show on black background but when rendered in EXR layers the God Rays are not showing. Any idea? Thanks 🙏
Hello. Can you tell me how to render layers in "Unreal Engine 5". The usual layers are there, but in "UE5" they don't work. As I understood now instead of them "Data Layers" but they do not work with "Movie Render Queue".
Again an absolutely great tutorial. Just switched from Fstrom to Unreal as 4.26 finally has everything I need. is the scene available for purchase or download anywhere?
Hey Will! trying this out in 4.27 with the path tracer and I get a black output in the exr file, switched to deferred and it instantly works. Is this unsupported with the path tracer?
Hey William! I ran into an issue with this technique, and it involves meshes that use blueprints to be connected. I have a city environment, and there are city block meshes which use blueprints to basically be connected all together and moved collectively as one mesh. The issue is that, when I actually add them to one layer, they don't render as if they were IN that layer. Is there any way to solve this issue? Thanks a lot!
Blueprints might have some kind of limitation in this regard. I don't use blueprints that often so I can't help you out in this specific case unfortunately ;(
Awesome video! Plus is always needed when you're combining two layers that have been mutually excluded from each other, such as this case. The matte line might not always be visible in the rgb, but if you switch to view alpha you'll see it. It's amazing that you can do this in Unreal now!
Ahhh this actually makes a lot of sense! Thanks for clearing that up! I found it odd at first that we couldn’t just get a proper layer we could A over B. Now I get it!
i'm doing step by step but when import it in nuke, my layers haven't alpha data. i can separate layers but every layer has a black background. where is the problem?
I'm not sure what the issue is here but I simply cannot get this to render my scene. I have been trying since around 11am this morning and it's now 00:44. I have an Asus Scar 18 with a 13980HX, 32Gb DDR5 ram and a 4090 and no matter what I do I get an out of GPU memory error as well as paging file too small. I have set the paging file to system managed and tried multiple settings all the way up to 128gb to no avail. I have a tried making a blank scene with a landscape with no textures. Bog standard unreal default lighting setup and a single megascan rock! if I render with Deferred renderer without stencil layers, it renders no problem. As soon as I add stencil layers it simply will not render my scene. I have searched forum after forum for answers and have come up with nothing! I just want to render my landscape separately from the chopper I have in my scene and I want the weather particle effects to be on a seperate layer too so I can comp this correctly and have some semblance of control. I even tried using no render layers and just switched off the layers in the layer panel and restart the render, but even though they are hidden they still render. I am continuously hitting walls with the movie render queue. I recently sold my Desktop PC which was a 13900k with 3090 and 32gb Ram. and even that had the same issue and that desktop 3090 has 24gb Vram. Why would adding stencil layers cause the render to think its running out of Vram and page file yet rendering everything all at once is fine??? Edit: I also tried switching to dX11 to no avail and I tried turning hardware ray tracing off to no avail. I even purchased light forge to optimize for cinematic rendering...to no avail. I simply cannot render stencil layers. I deleted landscapes from the scene...to no avail. I am using ultra dynamic skys. Also Unreal version - 5.3.2
@WilliamFaucher Hello Everyone I have a question I am being able to apply different material to same mesh using keyframe in sequencer, but at the same time I want the mesh to also change, what is the best method to do so? It is regarding mask renders only not blueprints(not for play mode). Also wanted to know can we change materials of a group of objects/multiple objects at the same time. Actually I have a tried using actors hidden in game in a sequencer, just wanted to know is there any other method to do so. Thank you in advance.
I don't get it. Whenever I try to right-click in the layers window, nothing happens. Moving an object to the window also does nothing. I cannot create any layers...
Hi William, is there a way to create an alpha hold out on a stencil layer? I have a person on green screen footage in a U5 environment and I get the stencil layer working so I can grade the live action footage indpendently from the 3D environment, but when a 3D object in the foreground obstructs a portion of the person (on the media plate actor), then those portions that should be obstructed appear in front of the 3D environment. I know there is a way to do this in Blender, and I saw the alpha hold out instructions in UE documentation but it doesn't look like it means the same thing as in Blender and doesnt work the same way. When my camera in UE is filming the person and a stone fence passes in front of the media plate, I need the stenciled out layer to but cut out where the object passes in front, so that it looks correct... like the person is in the environment instead of floating on top of it all. This issue would be a major advancement in being able to integrate live action footage into a UE environment, but I can't find anything about it. Any tips would be greatly appreciated!
I can't bring this to work together with Nanite. As soon as I have ANY Nanite-enabled object in the scene, nothing is rendered out anymore, only black frames. Am I missing something here? Thank you very much!
THANK GOD FOR THIS COMMENT!!! I'VE BEEN TRYING TO SOLVE THIS PROBLEM FOR MONTHS ALMOST SWORE TO NEVER TOUCH UNREAL EVERY AGAIN IN MY LIFE. THANK YOU SO SO MUCH. I WILL BE THANKFUL UNTIL THE DAY I DIE, WHICH WOULD HAVE BEEN TODAY IF I DIDN'T FIND THIS COMMENT! YOU HAVE SAVED MY LIFE
Am I correct in seeing some limitations to the Nuke Unreal Reader when it comes to stencil layers? I seems I can only add individual objects as stencil layers, but in UE I can add multiple objects into a single stencil layer. Am I understanding this right?
Hey William, I am a compositor. Trying to get UE to render out passes and watched all the content related to that on your channel. Great stuff! I was wondering if there is any update on rendering the traditional render passes from UE? I know you can export passes like World Normal, World Pos, Base Color, Spec, Roughness etc but most are not what traditional renderers use. Things like reflection pass, refraction pass, sss pass etc. Any ideas on how to do it?
I scratch my head so hard when compositing landscape in exr files with AfterEffects coz it's breaking up to smaller pieces so this is the answer. thanks !!!!
Hey William, nice video! I am trying to get an alpha layer rendered with EXR but there is sky / fog in the back ground. Is there anyway to get the sky / fog to be transparent and not opaque? It affects my alpha. Thanks.
is this still up to date or is the default cryptomatte workflow more suffisticated these days? Looking for ways to get the motion blur pass working, but it just stays green for me. I do get MB in the beauty pass though.
Thanks William for your lessons! Maybe you have testes it with glass in UE5? Glass behaves broken, not written to layers. In the Default Layer, it becomes visible through the object standing behind the glass. Maybe it's the lumens. In 4.26 everything worked well
Hi ! Could it be possible to update this tutorial for UE5.1 ? Can't seem to have alpha render correctly in 5.1 (noise instead of alpha in EXR/PNG). Thanks in advance !
@@jozinzzzzzbazin I have since managed to render the alpha on the .exr/.png. I had some noise instead of the alpha layer, the problem was not with my settings but with a parameter in the PostProcessVolume. Since then, no more problem to render the alpha.
Hey Will! I just ran into an issue I thought you'd find interesting regarding Stencil Layers. Firstly, let me start off by saying that I have been using PNG instead of EXR while keeping all other settings intact and it has been working just fine (afraid to use EXR since my monitor is shit). But anyways, so this method you showed actually does NOT work if you are using Console Variables. And I mean ANY kind of console variable as far as my experiments went. I kept getting my foreground layers with black backgrounds, and then I realized that when I turned off my Console Variables the alphas were rendering just fine. My temporary solution is to save 2 setting presets. First one is going to be following your tutorial with the Deferred Rendering enabled and Console Variables disabled, then the second one I enable my console variables and disable the Deferred Rendering.
I'm not sure Stencil Layers works all that well with PNG. The reason I use EXR is because EXR is not just a fileformat, it's a container for data, capable of storing several layers of data within the same file. In one single EXR file I can have all the render passes, all the alpha passes, and all the Object ID's. It is vastly superior in every single way over PNG.
Hello everyone, I have a small problem with a translucent material while rendering my video. I use MovieRenderQueue to export an EXR file with two stencil layers, one for my foreground and another for the translucent material in question (the windshield of a car). My problem is that when doing an export using path tracing, the layer is empty. By doing the same export in DeferredRendering I have my translucent material displayed on the layer. Does anyone have any tips for solving this problem? Thanks in advance.
Thank you for tutorial! Just had a question, at 4:45 you say we should render in linear sRGB. Is there any reason why you didn't mention or recommend ACEScg, which from my understanding is also linear?
Because the output from Unreal is always going to be linear sRGB, regardless of whether you apply an ACEScg transform with the colorspace settings (thats just going to bake in a transform into the render). It's cleaner to just render in linear sRGB and apply a color space conversion in Nuke or Resolve. But really, this all depends on what your color pipeline is like and what's needed.
Is there a way to add object that are inside blueprints to be added separately to a layer? Right now its adding the entire blueprint, but i just want some parts to be separate.
wow amazing its what i being trying to do for a long time , sorry but i have a dumb question is it possible to export the shadows from the objects so u can later put the animation in real footage in a composition sowtware so u can make a 3d real word integration along with the data smith plugin that can read camara tracking ? , thanks ur videos and patienece is awesome
Hello Thanks for your support and your time i need your help about shadow catch in render . i mean render alpha without ground plan but shadow is render with object in un real engine
Hi there! I've never tried actually, but if if can be toggled in the layers panel, then yes! My guess would be that it would have to be both landscape and landscape foliage in a single layer, no individually. But it's worth trying out.
Has anyone tried this in Unreal 5 ? For me the level sequence I add in the movie render queue does not use the camera I have set in that sequence. If I open the level sequence up and render from there it works just fine. But I really need those render layers.
Hi! anyone help me how to use this techniques in unreal 5.1. In UE 5.1 we have actor layers and data layers in stencil clip layer I create my layer and add in actor layer but it not render properly. I can see it in render preview.
Hey Will thanks again for this great info. So I tried this stencil layers method in Nuke and for some reason the render layers appear really blown out in the Shuffle and Merge nodes. I rendered in pathtracer from MRQ and ensured that I disabled tone curve and checked on Accumulator includes alpha. default beauty pass and alphas look fine however in Nuke. I know you mentioned it wont be complete 1:1 from Unreal to Nuke but it is appearing way off. I'm using unreal 5.3.1. Any thoughts on why this might be the case? Much appreciated legend
Hello @William,i have an issue about Directional light Raytraced Shadows on Foliage UE 4.26 when the Raytraced is ON Shadows are gone, Can you please help me about that issue, Sorry for off topic question
Oof that's a good question! Raytracing sometimes does weird things with geo that is not two-sided. In my experience, if I'm working with scenes with a LOT of foliage, I tend to not use raytraced shadows, at least for directional lights. I use raytraced shadows on lights like the rect light, but for directional? Sometimes it just looks better without raytraced shadows.
@WillIamFaucher Is there a way to render out an alpha that includes the alpha with the shadow from the object included? So I can output a final image that has only the object and the shadow in the alpha.
HeyHo ... any Idea how that works now in UE5.02? In there it seams i cant create Layers anymore - it sais in the docs i should use DataLAyers - ok - but MovieRenderQue asks for Layers which are NOT Datalayers... hm ...
@@WilliamFaucher thanks - I will have a look dear Mr Faucher... 😊 Renderings in our company are called "fauchers" by the way ... thank you for all the great tutorials! 😁
wyllyam,good day. tks for t video. I have a problem here and i dont know why- Is persistent. i did all the same process several times, but the secuence o EXR doesnt work. tIME AND time only deliver 1 unique frame ... i send another time, and again only get 1 exr file... Can sayme how can i fix this ? tks!!!
Thank you very much for your sharing, it is very helpful to me, but I found a problem, MoveRender cannot render the cable component correctly through layering, the cable is not visible, I hope you can find a solution
Really strugling to get this working in either 4.27 or 5. I'm not using pathtracing, I set Scale Percentage to manual 100%, I've got all the settings you mention here turned on. And yet both after effects and photoshop with ProExr just see a blank alpha layer? Any advice anyone?
Make sure you're using EXR, not jpg or PNG. Be sure to have Accumulator includes Alpha in the render settings. And you need to specify what the alpha is going to be by splitting up your layers. If you don't have any layers set up, you won't get any alpha.
@@WilliamFaucher I love your videos. They have helped me so much. I have done everything you just mentioned but no alpha. Under project setting/rendering/postprocessing should enable alpha channel support be set to Linear color space only? Is there anything else I could be missing? I am trying to alpha out he HDRI map. It renders with alpha if I turn it off but then the scene is not lit correctly.
hey WF. Thanks for your awesome videos!!! Is there a way to convert a world partion to a normal level so I can use actor layers instead of data layers as for some reason I cannot make data layers work as stencil layers
Hi Will, thanks for your videos, they have been tremendously helpful to my journey in learning unreal engine for film making. Would like to know if you know a way to render out niagara particles with alpha channel? I have been trying to do this but it just doesn't work. The particles don't show up in compositing softwares.
One quick question: I'm trying to implement unreal as my main render engine for VFX. Do you already have or would you please be so kind and make a tutorial for a shadow catcher in unreal? That would be such a big help, especially since your tutorials feel much more informative than most others on UA-cam. Thanks for your great work!
Ah thanks so much man! I appreciate that! I genuinely don't think UE4 is quiiiiite ready for a true VFX application. When you say shadow catcher, I assume you mean rendering out a shadow pass? As far as I'm aware there is no native solution for that. You might be able to render out something with the help of a Post Process Material but I can't really say more than that, as I have not yet found a tried and true solution for this myself. Sorry if that wasn't the answer you were hoping for!
Sweet jesus I've been troubleshooting for hours as to why my alpha wasn't coming through, now seeing in your video a brief mention that screen percentage is not supported. Weird that this isn't in the docs. THANK-YOU!
How is it fine when the linear renders dont match the Viewport output of UE? What's the point of getting that out into comp? Also, if we have a post volume along with god rays etc this would be tough to work with.
Because final grade is usually done by compers/colorists, and if you're working in UE, you're not doing mission-critical, high budget VFX work. It is a bummer, I know, and this really should be fixed, either by Epic providing a LUT or something along those lines, but it is what it is.
Hey William, great stuff as always! I do have a question: what can I do to have all the layers in "full"? Stencils leave a black mask where the foreground is, which is great for compositing, but sometimes I need the full background behind the foreground.
@@WilliamFaucher I tried that, but I was wondering if there was a way to automate this as it's a little time consuming to do; I guess I'll keep doing it manually. Thanks!
@@Kuckooracha add a tag to your foreground objects and make an event track in sequencer that gets all actors and with that tag and hides them on frame 0
hey William. Thanks for your amazing tutorials man really helps out allot. Quick question, hopefully you see this. I cant seem to get a proper alpha when using stencil clip in the movie render queue. When I bring it into nuke the alpha is solid and fills the frame even though the rgb of the layer is premultiplied. Im curious if you have encountered this before. Im using multichannel exr as well.
@@WilliamFaucher Thank you for the reply. I figured it out. It was actually because I ended up remembering that I messed with the movieRenderQueue_customStencil material trying to fix it a few days ago. This ended up breaking the alpha. Before I asked this question I had already had already turned accumulate alpha on, but had a broken shader. Fixed by patching my install. Lesson learned ;) next time ill make sure to not mess with engine materials. If I do ill be sure to make a backup. Im a CG lead artist at the mill in LA by the way. It really helps having UE4 artist out there that are happy to help out online. The industry is changing quickly and its all hands on deck.
@@WilliamFaucher Sry, since you are doing such a great job teaching me Unreal, I had to rework my answer two give you a more detailed explanation, so I can sleep well. (= An over multiplies the BG with the inverted alpha of the FG and then adds the two layers together. A+B(1-a) The Disjoint-over adds the two layers directly, but respect values, that go over 1 and treat them accordingly, giving better results and you can add layers inbetween: Algorithm: A+B if a+b
To clarify, I claim that Stencil Layers/Render Layers are better than UNREAL'S Cryptomatte/Object ID exclusively. Reason being, Cryptomatte/Object ID is pretty limited in its current state in the Unreal engine. And in order to get proper depth of field and motion blur, you need to render out Zdepth and Motion Vector passes, and combine them in a relatively complicated way, along with some math to get yout Cryptomatte to show up properly. Stencil layers are much simpler and cleaner way to work.
Other renderers like Arnold, Octane, Vray, etc, have vastly superior Cryptomatte support.
I tried to follow unreal documents to combine depth and velosity for motion blur but I am new user for nuke and I can't get the point
I hope to make tutorial to clearify these math settings and if the new update in 4.26.1 provide new enhancement for post production work
Thanks for al your effort
@@stepscourse4727 Yeah it really isn't super obvious, and Epic is not very clear in their documentation about how to do this. I got it to work, but it's really not super well explained!
@@WilliamFaucher
If you could get the point from documents ,please clearify to us
In another video you mentioned Zdepth not having a 32-bit option in MRQ. Does that mean there is no viable way to use the Cryptomatte, because you can't add proper depth of field/motion blur back in?
When I tried to add scene depth pass to movie render queue , I only get black layer , there is no any depth in it in gray or white , what control this in the scene, Although when I get world depth only pink colour
He William, really great tutorials style, honest, direct and clear. Ill be recommending to my team. Looking forward to seeing more.
Thanks so much! Short sweet and concise is what I strive for! Thanks for writing :)
Incredible Incredible Incredible. Between you and Winbush, the sky is the limt for what UR4 can be used for. You've inspired me to ACTUALLY start a favorites list for learning UR4. Anyone learning Unreal for FX and graphics needs to watch this video. Well done!
Thanks so much! I appreciate that! :) Best of luck with your work!
Thanks Will! I've started learning Unreal recently, and coming from a compositors point of view I feel like there's a lack of tutorials on higher production level workflows and how to use Unreal together with Nuke - and what I can expect from the Unreal renders inside of Nuke. So all of your videos are greatly appreciated and useful, big thanks!
Hey Tim! Hope you're doing well over in Oslo!
You're absolutely right, there is so little content for the compositing side of things. The main reason for that is because, well, simply put it's kind of messy right now. Most of the tools for compositing have been duct taped into the engine in a way. There's "support" for exporting lots of things but it's kind of hacky, and doesn't feel quite fully fleshed out yet. Like Cryptomatte for example. It does work, but it isn't a true crypto, and getting crypto to get motion blur and depth of field to work with it is kind of tedious. There is no 32-bit support either (although in theory the new 4.26.1 release does support it, it would seem like it's more of a 24-bit file under the hood, but I can't confirm this). I do think things will change for the better in Unreal Engine 5.
I'll certainly be releasing some more Post-Production oriented videos so stay tuned!
@@WilliamFaucher Yes duct tape is a perfect metaphor for how I've experienced it so far lol, I'm constantly questioning myself if I've done something wrong. - But yes let's hope it's getting more streamlined in that aspect soon, I find myself stumbling around in Unreal quite a bit trying to do "comp stuff" while really wanting to do it in Nuke instead. But again your videos are a gold for helping with that exact problem, thanks for carrying the torch!
@@Bobbiloff You're not the only one to be stumbling around! It is important to keep in mind Unreal is first and foremost a games engine, not a renderer designed for VFX work. Any kind of support geared towards VFX artists has been added in as a sort of afterthought, I don't believe UE4 was designed with VFX in mind. UE4 was released back in 2014-2015, I'm not sure the devs in their wildest dreams expected it to come this far in terms of visual fidelity so it isn't totally surprising that things aren't the way we'd expect.
Now that they've had a chance to see what the demand is, UE5 has the potential to rectify this. Fingers crossed.
Very useful tut...what I really like about your videos is your idea about an issue ..you go straight out to the topic and clarifies it very well...I have learned a lot from your video tutorials William...thx
Hello William, thanks for all your help. One question, with Unreal 5, i'm trying to get layers on a multilayer EXR using Path tracing. but I'm failing at it. is there any useful trick for this to work on Unreal 5? thanks
This has been a headache for a month and you saved me 😂 you are a lifesaver
So, will this work with dynamic stuff like fx? Say I want to TS some falling dust for motion blur, but I don't want other elements in the scene to be affected the same way. Could I put the effect in a Stencil Layer and have it work properly?
My best channel in vfx making with unreal engine. I am Really thankful
One of the most useful Unreal Engine tutorials I've seen. From the first to the last minute. Thank you very much. Greetings from Poland.
Love this stuff, keep it up. You're videos are helping me alot to make the transition from Vray to Unreal for television work. This is without a doubt the future of production work.
Thanks Lee! Unreal still has a ways to go but it's really exciting to see just how good it is getting!
Hi, does anyone know if this is broken in UE5? i get my background layer with a mate of my character on it and one of my layers thar is supposse to have an object renders empty :(
Is there a way to add foliage to layer? I don't seems to be able to find one. Or maybe a way to convert existing foliage to static meshes?
Heyo. Nice video. VFX comper here. The black haloing is due to the fact that your background image has the foreground stenciled from it. So there will always be an area where the fine reduced value edge pixels from both FG and BG don't add up.
If you were building a proper comp, (say a matte painting as your background) you'd have colour values where the two edges overlap that the foreground overlaps.
You could try unpremultiplying the background which might squeeze out a few pixels of solid colour so that overing works correctly. Alternatively. If you were doing layering as simple as yours. You could just render the background without any of the foreground stuff visible. So you get the whole image.
Plusing can cause issues down the line if used incorrectly. You can also start to introduce super pixel values and nans the more layers and FX you apply. Like with lightwrap,blooms etc.
Hi there! Thanks so much for chiming in and sharing your explanation of things, it is much appreciated! The main reason I used plus here is because that is what was recommended by Epic themselves, so, I figured they're smarter than me in regards to this. That said, your thoughts on the matter will have me experimenting with this further. Completely agreed in regards to the issues you may run into with plus, it's not what I typically would use in a production. Thanks again!
Hello,
I am not sure I you still read the comments after all this time - really cool that you reply to so many people in the first place :)
I am pretty sure I followed every step like you did - but I still do have, perhaps not a black halo, but apretty jagged edge around my masks...
Are there other things that could cause my masks to break?
The setting you have to change in the project settings gives me the options 'allow through tonemapper' and 'linear color space ' - same problem with both...
I do have also TemporalAA.Upsampling enabled for the better DOF.
Also my Sky that comes from the SkyAtmo is not in the defaul layer - which is strange.
Helo.. thank you for the video! btw, i have a question.. how to import the render result into After Effects? cause everytime i import, it always imported as 1 layer (BG). i cant separate between FG and BG.. thank you
I am so happy you posted this, but one question? Have you ever tried this workflow but instead of NUKE have you ever gone to Photoshop instead to use the layers for paint over? Im running into some issues with the layers on the Photoshop end of things. Any advice would be 1000% welcome. Thanks!
Hey Will, I was wondering; is there a way to render out using this technique but not having the MRQ render a Final Image alongside a stencil layer? I am only trying to render my layers individually without the default layer rendering, and it's becoming quite taxing on my system in terms of time restraints. I've turned off Add Default Layer, yet it is still rendering the default layer. Any way to counteract this? Thanks!
Hi William, I followed all your steps and it still won't render with an alpha mask. Although, I haven't been able to confirm the 'Disable screen percentage' - where is this found?
Thanks!
When I switch to alpha inside Nuke, there is no Aplha present (no white, pure black) use the same setting as this video, any thoughts?
Just getting into proper compositing so trying out nuke after Unreal in my workflow and this video is a lifesaver! Thanks!
Edit: Just set my first movie render queue render going with path tracing and a couple layers on a simple 10 sec/240 frame shot. The estimated time of 6 hours just made my heart sink. Worth it in the end though.
Hi, I spend hours to get it work in UE5.0.3 but I just get the complete scene rendered out. No layer are created. Do you have an advice for me how to get it work in UE5?
YES! This is what I was looking for coming from Redshift and Arnold's render layer
Amazing tut as always! Got an issue with trying to render God Rays on transparent background (using Exponential Height For w/ Volumetric Fog) and can't seem to make it work. It does show on black background but when rendered in EXR layers the God Rays are not showing. Any idea? Thanks 🙏
Hello. Can you tell me how to render layers in "Unreal Engine 5". The usual layers are there, but in "UE5" they don't work. As I understood now instead of them "Data Layers" but they do not work with "Movie Render Queue".
Again an absolutely great tutorial. Just switched from Fstrom to Unreal as 4.26 finally has everything I need. is the scene available for purchase or download anywhere?
Thanks man! This scene is available on the Epic Marketplace for free. If it’s not there, check their Learn section, might be there.
Hey Will!
trying this out in 4.27 with the path tracer and I get a black output in the exr file, switched to deferred and it instantly works. Is this unsupported with the path tracer?
Pathtracing doesn't support stencil layers, sorry! Nor does it support alpha channels at this time.
@@WilliamFaucher gotcha, can't wait for path tracing to develop further. Seriously impressive stuff, much faster than Redshift
Hey William! I ran into an issue with this technique, and it involves meshes that use blueprints to be connected. I have a city environment, and there are city block meshes which use blueprints to basically be connected all together and moved collectively as one mesh. The issue is that, when I actually add them to one layer, they don't render as if they were IN that layer. Is there any way to solve this issue? Thanks a lot!
Blueprints might have some kind of limitation in this regard. I don't use blueprints that often so I can't help you out in this specific case unfortunately ;(
Awesome video! Plus is always needed when you're combining two layers that have been mutually excluded from each other, such as this case. The matte line might not always be visible in the rgb, but if you switch to view alpha you'll see it. It's amazing that you can do this in Unreal now!
Ahhh this actually makes a lot of sense! Thanks for clearing that up! I found it odd at first that we couldn’t just get a proper layer we could A over B. Now I get it!
William, this is super helpful. Thanks so much for sharing
The pleasure is mine! I only learned about how this works recently, figured I wasn't the only one to think this is gold!
Thanks for watching Sepp!
@@WilliamFaucher I'm like using this as we speak :)
Good luck!
*Nxt Level Nxt Level...Waiting For More Content* 😍
Thanks man!
How do you separate the 2 render stencil layers with After Effects? PLEASE Help
If anyone knows I'd be super appreciative
EXCEPTIONAL WORK!
i'm doing step by step but when import it in nuke, my layers haven't alpha data. i can separate layers but every layer has a black background. where is the problem?
Greatest tutorials on youtube!
Best commenter on youtube!
Another brilliant tutorial! Thank you.
Thank you so much!
Great videos! Loving these
I'm not sure what the issue is here but I simply cannot get this to render my scene. I have been trying since around 11am this morning and it's now 00:44. I have an Asus Scar 18 with a 13980HX, 32Gb DDR5 ram and a 4090 and no matter what I do I get an out of GPU memory error as well as paging file too small. I have set the paging file to system managed and tried multiple settings all the way up to 128gb to no avail. I have a tried making a blank scene with a landscape with no textures. Bog standard unreal default lighting setup and a single megascan rock! if I render with Deferred renderer without stencil layers, it renders no problem. As soon as I add stencil layers it simply will not render my scene. I have searched forum after forum for answers and have come up with nothing!
I just want to render my landscape separately from the chopper I have in my scene and I want the weather particle effects to be on a seperate layer too so I can comp this correctly and have some semblance of control. I even tried using no render layers and just switched off the layers in the layer panel and restart the render, but even though they are hidden they still render. I am continuously hitting walls with the movie render queue. I recently sold my Desktop PC which was a 13900k with 3090 and 32gb Ram. and even that had the same issue and that desktop 3090 has 24gb Vram.
Why would adding stencil layers cause the render to think its running out of Vram and page file yet rendering everything all at once is fine???
Edit: I also tried switching to dX11 to no avail and I tried turning hardware ray tracing off to no avail. I even purchased light forge to optimize for cinematic rendering...to no avail. I simply cannot render stencil layers. I deleted landscapes from the scene...to no avail. I am using ultra dynamic skys.
Also Unreal version - 5.3.2
@WilliamFaucher Hello Everyone I have a question I am being able to apply different material to same mesh using keyframe in sequencer, but at the same time I want the mesh to also change, what is the best method to do so? It is regarding mask renders only not blueprints(not for play mode). Also wanted to know can we change materials of a group of objects/multiple objects at the same time.
Actually I have a tried using actors hidden in game in a sequencer, just wanted to know is there any other method to do so.
Thank you in advance.
Holy crap this is awesome! Fantastic content dude, thanks!
Thanks so much! Hope it helps!
I don't get it. Whenever I try to right-click in the layers window, nothing happens. Moving an object to the window also does nothing. I cannot create any layers...
Hi William, is there a way to create an alpha hold out on a stencil layer? I have a person on green screen footage in a U5 environment and I get the stencil layer working so I can grade the live action footage indpendently from the 3D environment, but when a 3D object in the foreground obstructs a portion of the person (on the media plate actor), then those portions that should be obstructed appear in front of the 3D environment. I know there is a way to do this in Blender, and I saw the alpha hold out instructions in UE documentation but it doesn't look like it means the same thing as in Blender and doesnt work the same way. When my camera in UE is filming the person and a stone fence passes in front of the media plate, I need the stenciled out layer to but cut out where the object passes in front, so that it looks correct... like the person is in the environment instead of floating on top of it all. This issue would be a major advancement in being able to integrate live action footage into a UE environment, but I can't find anything about it. Any tips would be greatly appreciated!
did you get any replies?
I can't bring this to work together with Nanite. As soon as I have ANY Nanite-enabled object in the scene, nothing is rendered out anymore, only black frames. Am I missing something here? Thank you very much!
THANK GOD FOR THIS COMMENT!!! I'VE BEEN TRYING TO SOLVE THIS PROBLEM FOR MONTHS ALMOST SWORE TO NEVER TOUCH UNREAL EVERY AGAIN IN MY LIFE. THANK YOU SO SO MUCH. I WILL BE THANKFUL UNTIL THE DAY I DIE, WHICH WOULD HAVE BEEN TODAY IF I DIDN'T FIND THIS COMMENT! YOU HAVE SAVED MY LIFE
Am I correct in seeing some limitations to the Nuke Unreal Reader when it comes to stencil layers? I seems I can only add individual objects as stencil layers, but in UE I can add multiple objects into a single stencil layer. Am I understanding this right?
Unfortunately this doesn't work for Unreal 5. Is there a workaround?
Hey William, I am a compositor. Trying to get UE to render out passes and watched all the content related to that on your channel. Great stuff!
I was wondering if there is any update on rendering the traditional render passes from UE? I know you can export passes like World Normal, World Pos, Base Color, Spec, Roughness etc but most are not what traditional renderers use. Things like reflection pass, refraction pass, sss pass etc. Any ideas on how to do it?
Awesome video, thanks for making these. Is there a way to render just the FG layer and leave the BG as is without stencil applied? Thanks
I scratch my head so hard when compositing landscape in exr files with AfterEffects coz it's breaking up to smaller pieces so this is the answer. thanks !!!!
Hey William, nice video! I am trying to get an alpha layer rendered with EXR but there is sky / fog in the back ground. Is there anyway to get the sky / fog to be transparent and not opaque? It affects my alpha. Thanks.
i create still images in unreal engine using the path tracer, will this work in 5.1? im trying to create alphas for my images for objects in my scene.
is this still up to date or is the default cryptomatte workflow more suffisticated these days? Looking for ways to get the motion blur pass working, but it just stays green for me. I do get MB in the beauty pass though.
Thanks William for your lessons! Maybe you have testes it with glass in UE5? Glass behaves broken, not written to layers. In the Default Layer, it becomes visible through the object standing behind the glass. Maybe it's the lumens. In 4.26 everything worked well
Hi !
Could it be possible to update this tutorial for UE5.1 ? Can't seem to have alpha render correctly in 5.1 (noise instead of alpha in EXR/PNG).
Thanks in advance !
Actually it doesnt work since 4.27 (black textures both for png and empty alphas for exr)
@@jozinzzzzzbazin I have since managed to render the alpha on the .exr/.png. I had some noise instead of the alpha layer, the problem was not with my settings but with a parameter in the PostProcessVolume. Since then, no more problem to render the alpha.
Do not forget to allow alpha trough tonemapper in the settings
@@jozinzzzzzbazin Select all the actors you want, place them in a layer, and render only this layer in the deferred rendering
Hey Will! I just ran into an issue I thought you'd find interesting regarding Stencil Layers. Firstly, let me start off by saying that I have been using PNG instead of EXR while keeping all other settings intact and it has been working just fine (afraid to use EXR since my monitor is shit). But anyways, so this method you showed actually does NOT work if you are using Console Variables. And I mean ANY kind of console variable as far as my experiments went. I kept getting my foreground layers with black backgrounds, and then I realized that when I turned off my Console Variables the alphas were rendering just fine.
My temporary solution is to save 2 setting presets. First one is going to be following your tutorial with the Deferred Rendering enabled and Console Variables disabled, then the second one I enable my console variables and disable the Deferred Rendering.
I'm not sure Stencil Layers works all that well with PNG. The reason I use EXR is because EXR is not just a fileformat, it's a container for data, capable of storing several layers of data within the same file. In one single EXR file I can have all the render passes, all the alpha passes, and all the Object ID's. It is vastly superior in every single way over PNG.
YAAAAASSSS!!! Thank you so much, VAT works, alpha works, me so happy 😊
Cheers, and good luck!
Hello everyone, I have a small problem with a translucent material while rendering my video. I use MovieRenderQueue to export an EXR file with two stencil layers, one for my foreground and another for the translucent material in question (the windshield of a car).
My problem is that when doing an export using path tracing, the layer is empty. By doing the same export in DeferredRendering I have my translucent material displayed on the layer.
Does anyone have any tips for solving this problem? Thanks in advance.
Thank you for tutorial! Just had a question, at 4:45 you say we should render in linear sRGB. Is there any reason why you didn't mention or recommend ACEScg, which from my understanding is also linear?
Because the output from Unreal is always going to be linear sRGB, regardless of whether you apply an ACEScg transform with the colorspace settings (thats just going to bake in a transform into the render). It's cleaner to just render in linear sRGB and apply a color space conversion in Nuke or Resolve. But really, this all depends on what your color pipeline is like and what's needed.
thanks william it does work with png well in 5.32 now just need to get the shadows on an alpha layer
Hey William Can you make video about detailed use of GPU lightmass in 4.26 as there are no in depth tut about it ?
I can look into it yeah! Added to the list! :)
@@WilliamFaucher :)
Amazing! Would this work with path tracing?
Is there a way to add object that are inside blueprints to be added separately to a layer? Right now its adding the entire blueprint, but i just want some parts to be separate.
wow amazing its what i being trying to do for a long time , sorry but i have a dumb question is it possible to export the shadows from the objects so u can later put the animation in real footage in a composition sowtware so u can make a 3d real word integration along with the data smith plugin that can read camara tracking ? , thanks ur videos and patienece is awesome
Hello
Thanks for your support and your time
i need your help about shadow catch in render . i mean render alpha without ground plan but shadow is render with object in un real engine
Super Thank you Great Faucher
Cheers! Happy to help!
First of all, thank you for a great tutorial, now to the question:) : do you know why stencil layers broken in 4.27 and how it could be fixed?
any idea why is not working on groom?
Nicely Explained does Stencil layers Support Landscape Foliage?
Hi there! I've never tried actually, but if if can be toggled in the layers panel, then yes! My guess would be that it would have to be both landscape and landscape foliage in a single layer, no individually. But it's worth trying out.
This was awesome thanks!
How do you add reflections in the foreground layer if you are using a transparent glass material?
Has anyone tried this in Unreal 5 ? For me the level sequence I add in the movie render queue does not use the camera I have set in that sequence. If I open the level sequence up and render from there it works just fine. But I really need those render layers.
Hi! anyone help me how to use this techniques in unreal 5.1. In UE 5.1 we have actor layers and data layers in stencil clip layer I create my layer and add in actor layer but it not render properly. I can see it in render preview.
Hey Will thanks again for this great info. So I tried this stencil layers method in Nuke and for some reason the render layers appear really blown out in the Shuffle and Merge nodes. I rendered in pathtracer from MRQ and ensured that I disabled tone curve and checked on Accumulator includes alpha. default beauty pass and alphas look fine however in Nuke.
I know you mentioned it wont be complete 1:1 from Unreal to Nuke but it is appearing way off. I'm using unreal 5.3.1. Any thoughts on why this might be the case?
Much appreciated legend
actually managed to get the layers to match my beauty pass by using multiply slider with a grade node it turns out
Hello @William,i have an issue about Directional light Raytraced Shadows on Foliage UE 4.26 when the Raytraced is ON Shadows are gone, Can you please help me about that issue, Sorry for off topic question
Oof that's a good question! Raytracing sometimes does weird things with geo that is not two-sided. In my experience, if I'm working with scenes with a LOT of foliage, I tend to not use raytraced shadows, at least for directional lights. I use raytraced shadows on lights like the rect light, but for directional? Sometimes it just looks better without raytraced shadows.
@@WilliamFaucher Thanks for answer i will try this,i have open world project you think Rec light work for this ?
Superb, Thanks William, Is it possible to put some more highlight on Color management tool in Movie render queue.
Translucent materials will appear in each layer ，how to fix it? thanks
Have found a fix for this issue ?
@WillIamFaucher
Is there a way to render out an alpha that includes the alpha with the shadow from the object included?
So I can output a final image that has only the object and the shadow in the alpha.
Unfortunately shadow passes aren’t a thing in Unreal for the time being :(
HeyHo ... any Idea how that works now in UE5.02? In there it seams i cant create Layers anymore - it sais in the docs i should use DataLAyers - ok - but MovieRenderQue asks for Layers which are NOT Datalayers... hm ...
I would recommend looking into the UnrealReader, I made a tutorial on it! Stencil Layers work :)
@@WilliamFaucher thanks - I will have a look dear Mr Faucher... 😊
Renderings in our company are called "fauchers" by the way ... thank you for all the great tutorials! 😁
Awesome teacher, thank you.
Thanks so much!
i love you .. i really find every answer i need in your videos ..
great great video once again mate.
Thank you!
wyllyam,good day. tks for t video. I have a problem here and i dont know why- Is persistent. i did all the same process several times, but the secuence o EXR doesnt work. tIME AND time only deliver 1 unique frame ... i send another time, and again only get 1 exr file... Can sayme how can i fix this ? tks!!!
theres no secuence !! only one unique file... EXR..into the folder.. TKSSSS FOR THE REPPLY!!
great tut as always! Do you know where I might find a stencil / render layer workflow from Unreal to Davinci Resolve?
You just use the exact same approach I used here, except import the renders into resolve and use fusion to comp the layers
Thank you very much for your sharing, it is very helpful to me, but I found a problem, MoveRender cannot render the cable component correctly through layering, the cable is not visible, I hope you can find a solution
Thankyou!! Very useful. Well explained.
Pleasure is mine!
Really strugling to get this working in either 4.27 or 5. I'm not using pathtracing, I set Scale Percentage to manual 100%, I've got all the settings you mention here turned on. And yet both after effects and photoshop with ProExr just see a blank alpha layer? Any advice anyone?
Make sure you're using EXR, not jpg or PNG. Be sure to have Accumulator includes Alpha in the render settings. And you need to specify what the alpha is going to be by splitting up your layers. If you don't have any layers set up, you won't get any alpha.
@@WilliamFaucher I love your videos. They have helped me so much. I have done everything you just mentioned but no alpha. Under project setting/rendering/postprocessing should enable alpha channel support be set to Linear color space only? Is there anything else I could be missing? I am trying to alpha out he HDRI map. It renders with alpha if I turn it off but then the scene is not lit correctly.
Just discovered your channel n it's really helpful very good content 👌 subscribed!
Hey there! Thank you very much! Happy to hear it helps! :)
Does this wroks for 360 capturing ?
Amazing. Thank you pal, these videos are absolute gold :-)
The pleasure is mine! Thanks for writing!
hey WF. Thanks for your awesome videos!!! Is there a way to convert a world partion to a normal level so I can use actor layers instead of data layers as for some reason I cannot make data layers work as stencil layers
Great tuto but apparently no one has the solution to separate the foreground and background on after effect... 😔😔😔😔😔😔
Thank you so much!
Hi Will, thanks for your videos, they have been tremendously helpful to my journey in learning unreal engine for film making. Would like to know if you know a way to render out niagara particles with alpha channel? I have been trying to do this but it just doesn't work. The particles don't show up in compositing softwares.
Did you find a way of making it work?
One quick question: I'm trying to implement unreal as my main render engine for VFX. Do you already have or would you please be so kind and make a tutorial for a shadow catcher in unreal? That would be such a big help, especially since your tutorials feel much more informative than most others on UA-cam. Thanks for your great work!
Ah thanks so much man! I appreciate that! I genuinely don't think UE4 is quiiiiite ready for a true VFX application. When you say shadow catcher, I assume you mean rendering out a shadow pass? As far as I'm aware there is no native solution for that. You might be able to render out something with the help of a Post Process Material but I can't really say more than that, as I have not yet found a tried and true solution for this myself. Sorry if that wasn't the answer you were hoping for!
I'm getting a lot of help from your channel. Thank you.
How to render Nanites view in movie render queue?
Sweet jesus I've been troubleshooting for hours as to why my alpha wasn't coming through, now seeing in your video a brief mention that screen percentage is not supported. Weird that this isn't in the docs. THANK-YOU!
Glad to hear it helped! It's such a small thing that can break it haha
How is it fine when the linear renders dont match the Viewport output of UE? What's the point of getting that out into comp? Also, if we have a post volume along with god rays etc this would be tough to work with.
Because final grade is usually done by compers/colorists, and if you're working in UE, you're not doing mission-critical, high budget VFX work. It is a bummer, I know, and this really should be fixed, either by Epic providing a LUT or something along those lines, but it is what it is.
Hey William, great stuff as always! I do have a question: what can I do to have all the layers in "full"?
Stencils leave a black mask where the foreground is, which is great for compositing, but sometimes I need the full background behind the foreground.
You have to hide the foreground before rendering then!
@@WilliamFaucher I tried that, but I was wondering if there was a way to automate this as it's a little time consuming to do; I guess I'll keep doing it manually. Thanks!
@@Kuckooracha add a tag to your foreground objects and make an event track in sequencer that gets all actors and with that tag and hides them on frame 0
@@brandonrosado6384 You know what that's actually super simple and elegant, thanks for the idea
can i access these stencil layers in after affects
hey William. Thanks for your amazing tutorials man really helps out allot. Quick question, hopefully you see this. I cant seem to get a proper alpha when using stencil clip in the movie render queue. When I bring it into nuke the alpha is solid and fills the frame even though the rgb of the layer is premultiplied. Im curious if you have encountered this before. Im using multichannel exr as well.
Hey! Thank you!
This happens if you didn't have "Accumulator Includes Alpha" checked on in the "Deferred Rendering" tab. Very important.
@@WilliamFaucher Thank you for the reply. I figured it out. It was actually because I ended up remembering that I messed with the movieRenderQueue_customStencil material trying to fix it a few days ago. This ended up breaking the alpha. Before I asked this question I had already had already turned accumulate alpha on, but had a broken shader. Fixed by patching my install. Lesson learned ;) next time ill make sure to not mess with engine materials. If I do ill be sure to make a backup. Im a CG lead artist at the mill in LA by the way. It really helps having UE4 artist out there that are happy to help out online. The industry is changing quickly and its all hands on deck.
FYI, if you want to use an Over in Nuke, then disjoint-over should work fine
Oh interesting, I've never heard of disjoint-over, will have to look into that. What does it do exactly?
@@WilliamFaucher Sry, since you are doing such a great job teaching me Unreal, I had to rework my answer two give you a more detailed explanation, so I can sleep well. (=
An over multiplies the BG with the inverted alpha of the FG and then adds the two layers together. A+B(1-a)
The Disjoint-over adds the two layers directly, but respect values, that go over 1 and treat them accordingly, giving better results and you can add layers inbetween: Algorithm: A+B if a+b