My mind was blown when you started playing with the lighting on a flat image in real time. I felt like a kid again, seeing something I never could have thought existed, like magic!
The secret of course is that it's not really a "flat" image at all, but a 3-D image encoded as colors for representation purposes. The normal buffer holds all the direction information necessary to reproduce the first light bounce off the object when combined with the material information such as the albedo (surface color) buffer. It's like you are storing a bas relief of the entire visible scene. The concept of modifying the light in an otherwise static scene is actually a whole field in computer graphics called "relighting" that is dedicated to finding ways to do this in real time for cases beyond the simple first bounce from point sources that's shown in the video.
In 35 minutes you managed to completely explain systems that I have been trying to understand for years. No other video on the topic comes close for beginners, and a worthy follow up the Mario Galaxy video. This is extraordinary
Fun fact now that TotK is out: They actually tried fixing the Bridge of Hylia one, but there’s some of it that they missed and you can still see a cel-shadeless Link (although it is a bit more finicky). It seems some changes to Link’s textures were made (aside from just Rauru’s arm) as now you can see thick, black outlines around where Link’s abs would be
i don't think they missed it i just think they fixed it only where the decal is fully transparent, fixing it further probably is not possible with their engine
I definitely feel like lighting is one of those things where if you do it right, people won't be sure you've done anything at all. it feels so natural when well executed it's easy to take for granted all the dedication it took behind the scenes to be that way. even coming into this video i didn't expect how many layers there were to it. fascinating stuff
A lot of these passes happen in parallel. The GPU is really good at doing a lot of similar operations in parallel. What the video failed to mention is that the techniques also evolve according to the hardware. In the past, these multi-pass techniques would be severely limited by the amount of parallelism available; it wouldn't necessarily slow down frame rate, but would introduce a lot of delay until the frame is ready for displaying. That would make the game feel like it has input lag, even though it has good frame rate overall; each frame takes a long time to bake in the GPU.
All of these calculations would have to happen even if they were still done separately for each object -- calculating the pixel's facing direction, computing stuff like shadows and fog and wrinkle maps, accumulating all the incoming light, and doing the material shading. It just feels like a lot more when each one of those stages is broken out to a lot of different buffers.
@@JasperRLZ It's like seeing the water is higher when it's poured into a narrower glass. It's the same amount of water, but it's taken in a different shape that may look like more.
@@JasperRLZ it is impressive that Nintendo got a high quality deferred rendering solution working so well on what is essentially mobile class hardware. A lot mobile games go with forward rendering because, for one, differed tends to require a lot more memory (need to store all the various buffers somewhere). Not to say the forward is limited to mobile. I believe Call of Duty goes with what they call a “Forward+” renderer.
@@TheCrewExpendable The switch uses the Nvidia tegra GPU. It's a true Nvidia graphics chip so it can handle the higher bandwidth. Also it's possible to do deferred rendering on mobile via tiled rendering - basically just breaking up the buffer into smaller sections. A lot of newer games are using this method.
"When you do things right, people won't be sure you've done anything at all." Perhaps one of my most favourite things of all is seeing how something is done right, and learning to appreciate it all the more. It doesn't just make me appreciate the thing itself, but the people doing them, and I think it's made me appreciate people in general more. The things we're willing to do to entertain others can feel like such an unnecessarily nice gesture.
Fantastic video. Almost everything you went over there about rendering was unknown entirely to me, despite what i thought i knew about it. when you were going over the fan theories as to why it happened, I figured it would be some kind of mesh in the way with a transparent material over it, And all the leadup to you mentioning what it actually was, all just lined up perfectly as you were explaining exactly how it all fit together.
The section around 16:36 where you demonstrated how a normal map can make a 2D image appear to have 3D lighting literally made me gasp. It seems so obvious having it explained this way, but I'm sure it took a lot of people a lot of work to get it to this point! Please keep making videos on whatever fascinates you most. You will have an audience :).
Indeed. I've known about Normal Maps for a while, knew what they do and how they look in their raw form but I never knew WHY they look like that or the specific details of how they work. This video explains it incredibly well.
@@Diwasho Maps like that make a lot more sense when you don't think of the channels as representing color! Just like the channels in an ordinary image are data representing color, the channels in a map represent all kinds of details relevant to rendering.
I don’t think I’ve ever learned as much about a single field within such a short time span. This video has led to me understanding many systems within game dev that I haven’t been able to graft just yet. I really hope you make more videos just like this one, analyzing the reason for something and meanwhile explaining so many other useful and interesting graphics tech solutions and standards. Mad respect! ❤
I've seen a lot of normal maps from my own decades of peeking behind curtains, but never *truly* understood them until this video. I felt totally dumbstruck when you made it sound almost basic- and it's such a cool technique! I'm thoroughly floored by how easy you made all of these incredibly complex concepts sound, and how naturally you explained them. Whatever you deep-dive into next: I can't wait to hear about it. And take your time- we can't imagine the amount of polish and edge-sanding that goes into making these videos seem so simple ;)
This is probably the best video on introduction to buffers I've seen. After effects scene blew my mind even though I've programed shaders before. Congrats dude, this seems like it was a ridiculous amount of work, but well worth it!
My best guess for the buffer at 18:54 is probably a roughness buffer, or more likely some kind of specular multiplier/exponent buffer. The second one at 19:00 might be a mask for world weather/rain effects. (Link is likely black because the weather effects on characters is likely handled entirely separately from the world.)
this video gets into all of the fun stuff about the rendering in an easy way to understand for me! and when it finally gets back to the main question, i really felt that "a-ha!" moment when every little detail you mentioned throughout fell into place, it was really satisfying to watch!
what you said about lighting is also true when movies went from black and white to color: because there were no colors to worry about with b&w film, all you had to worry about was the lighting, and even tho those movies have no color many have aged well because film makers were paying attention to that, whereas when color film became available a lot of movies had very flat lighting because film makers became too focused on the colors in their shots and the cinematography suffered. it really wasn’t until movies like The Godfather that we started see really dynamic lighting in color movies like we did with b&w ones.
I sort-of understood deferred shading before, but this video made it super clear, and the glitch at the core of the video really helped me understand everything. In the demo at 16:37, are you also using the depth buffer? Otherwise it seems like the far wall would be illuminated, and the light wouldn't have the falloff on the ground.
Yes, that's correct. I didn't want to take the time to explain a depth buffer. My original video script used a directional light, and those two buffers would indeed be all that's needed for that case, but an animating point light looks a lot better and is easier to understand, so I took a small creative liberty and changed the directional light to a point light, even though I still don't mention the depth buffer. Good eye :)
@@JasperRLZ Just for directional lighting? Even for a point light I don't understand how the system would know the far wall is too far away to receive light without looking at the depth buffer.
Whoa! I actually found this lighting bug on my own a long time ago. I didn’t realize it was so significant to people online. Breath of the Wild is so big that it feels cool and rare that I found this organically.
Dude, that was amazing. I can't imagine the amount of time and effort you put into making this, but you nailed it. You made it simple and cohesive enough that even a dummy like me could follow without feeling completely lost, and that's a truly impressive feat. Thanks for all your hard work, and I look forward to the next video.
hahaa i literally typed this comment 5 seconds ago and now i see heaps of comments mentioning this exact thing. really goes to show how good this vid was
16:40 That's a really good explanation and I can't believe I never knew lighting was as efficient as this from the computer's perspective. That you can re-light an entire scene as though you hadn't even taken the picture yet is kind of a mindfuck when you break it down like that. Fascinating!
I just rewatched your Super Mario Galaxy video a few days ago and this was a nice treat. I love your deep dive videos, you manage to explain games in an incredibly visual way to the point where it's crazy you don't have a devkit or something. Keep em up!
I've been looking forward to you showing off more BOTW since your short from last year. Nobody dives as deep or creates visuals like you Jasper. This blew me away! Thank you so much for all the work you put into these videos and your website.
Definitely more of this. Reflections could be an interesting deep dive. You could start with the Build engine (Duke 3D, Shadow Warrior, etc) requiring you to create a hollow room on the other side of a mirror to duplicate geometry into. Then move up to reflection maps, reflection cubes, dynamic reflections, screen space reflections, etc.
This is one of the coolest videos I’ve ever seen… i never thought I’d ever be able to wrap my head around something like lighting and graphics in video games, it always seemed so complicated. Game development is so sick
I went through most of the video going "It's an occlusion pane!" Until you went SUPER indepth into deferred lighting, and I was half correct, but it was super interesting to learn why I was only half right, that something was being occluded, but not in a way an occlusion pane works. Super interesting and I love learning about rendering techniques in games. Great video as per usual :)
Fascinating. Some concepts are familiar to me, but to put into perspective all the different layers that an engine must render before showing you the final frame is insane. Now to do it 60 times or more each second sounds like a luxury.
I'm convinced that in a few years this channel will have measurably helped a generation of graphics programmers excel at their craft seriously you always do such a great job of explaining complicated concepts without using even a hint of allegory or metaphor. a mess of normals, front buffers, back buffers, vertex shaders, pixel shaders, whatever the fuck else, somehow made accessible with good rhetorical ability
This is genuinely one of the most informative and entertaining videos I have ever found. You have an amazing talent that you should be proud of. Your channel is awesome and I can only hope you grow fast
I love people actually explaining what’s going on behind the scenes instead of giving me half-assed explanations Lighting is very scary for me because I don’t even know how you put all of that stuff together
This man deserves at least 1 million subs. This is the type of stuff that I find really interesting, but really intimidating so it’s nice to have something that is really easy to understand, with showing tons of visuals and examples, yet still teaches a lot and goes to over half an hour.
Definitely not scared away by the technical details. As a technical artist that dabbles in rendering, it's extremely interesting! I like seeing the breakdown, because watching your process might actually help me get better at tackling rendering bugs that get thrown at me ha
I feel proud of myself for correctly guessing the basic idea of the problem (overzealous occlusion shading cutting out an expensive step because it thinks the pixels are occluded for some purposes but not others), but I had no idea how complicated it was. Cool video, I always love learning about the thought and effort that goes into something I barely consciously noticed.
I remember finding out that new super Mario bros U is shaded with normal maps. I thought it was really interesting how they baked in the lighting data like that. Now I’m finding out that breath of the Wild simplifies shading by converting every rendered object into a normal map to then apply the lighting onto. What’s even crazier is that we have the capability of rendering the same frame like 20 times, then compiling all the separate information, and that all happens 60 times a second on Nintendo hardware. Takes me back to when I learned how CRT TV’s work.
BOTW targets 30, not 60. It barely hits 30 most of the time too. Deferred shading on switch is incredibly poorly performant since rendering to a pixel more than once basically obliterates the GPU, and you're extremely memory capped so it's hard to store much information in the gbuffer.
You don’t render the same frame 20 times. I doubt even a 4090 doesn’t have the room to do that. There’s a hardware feature called Multiple Render Target (MRT) allowing one render pass outputting to multiple buffers that stores different geometry data. Also switch is pretty limited on memory bandwidth. And a full-fledged deferred shading pipeline eats bandwidth like a monster. I’m actually very surprise to see how many Nintendo first party games managed to pull out deferred shading in switch - especially many with physically based materials
This all reminds me of my 8th grade math teacher. He would do this thing pretty often where he would answer someone's question with something seemingly unrelated, but as the explanation goes on he slowly shows ties with what he's explaining and the original question, until eventually the thing he was explaining which at first seemed completely unrelated is now the basis of the exact, concise answer to the student's question. Thanks for the great video :33
I've known about normals for a while, but this video did such a good job of explaining the how and why, and the after effects demonstration was brilliant. Also, OH MY GOD YES MORE PLEASE!
This was an amazing video! There was so much dense information made very clear and easy to understand. It felt like it was perfectly paced so that you don’t have to pause/replay to fully understand but it was also moving at a break neck pace. I’d love more videos like these. You spoke of spending so long polishing a game that no one could can notice the hard work, and I know lots of that is going on in this video
This notification popped up right as I got on UA-cam to watch something. Guess I don't have to pick a video this time! With that and me re-watching the original Wind Waker cell-shading video a few days ago it almost feels like you're spying on me...
Heya! I'm currently doing my final project in university on Breath of the Wild inspired lighting! This video is super helpful with understanding the techniques used in the game. Great work!
To be fair in the village example, realistically forward rendering should be using occlusion culling to prevent the whole village from being rendered as well.
Yeah I think that line in particular undersells the main benefits of deferred lighting, which is mostly that adding more light sources is practically free
Even without occlusion culling, the opaque pass typically draws front to back and can early-out on fragments which are obscured... Buuuuut I think the example works well in a "lies to children" sort of way. For folks who know none of this stuff, it's a close-enough description of one of the main benefits of deferred lighting which is easy to understand.
@@HadynLander Doesn't that mean that the objects have to be sorted before being sent to the gpu? Is the sorting overhead less than what it would've costed to shade obscured pixels? What about large objects like terrain?
@@cylemons8099 nah, you can also do a depth prepass and then only draw pixels at equal depth in the main pass. Sorting would just make that a little more efficient, but a depth prepass makes sorting not necessary.
lot's of people are confused why he'd make such a long video about a single bug, but I think the bug was really just an excuse to learn about the deffered lighting and I'm 100% fine with that. I love learning about this sort of stuff.
Blows my mind to see so many passes to get this game looking like it does. Major props to the artists and the devs that made it possible. And nice to see you back with another video! Always dig your breakdowns
I absolutely love how in depth your videos are and how much effort you put into making the visuals. It looks like you're using something like RenderDoc to get access to the buffers which is great. But it's absolutely impressive that you went the extra mile and showed the buffers getting written to and what they would look like before and after the decals. Especially on the glowing orb thing. That would've been so easy to just gloss over without going through the effort of getting a visual for it but you did the extra work and it's appreciated. I hope you take pride in this because you absolutely should.
When I clicked on this, I wanted to know why the glitch happened. And then I just wanted to know everything about lighting in games. This video is really good.
I love your technical videos the most. I'm surprised at how often the technique of building from first principle surprises me. Especially the things I already know. I've already used normal textures in a 2D environment, but seeing them manipulated manually in real-time in a video editor still gave me a youthful joy.
Incredibly well made video, actually made my day. Didn't think I'd be happily sat here watching 40 minutes of rendering techniques, but oh boy, I was fully engaged. Thank you Jasper!
This video taught me so much about stuff I technically knew about but didn't understand. From "what actually is deferred rendering besides something that tends to prevent devs from using MSAA" to "why is transparency crap or non existent in a bunch of newer games I've played", and "what are those material masks for in some PBR assets I've downloaded".
I absolutely love the effort you put into these videos. Not only do you explain why the bug happens. Not only do you say that fixing these bugs would have taken effort by the development team. No. You break down all of the technical issues in intricate detail. You program/visualize what exactly happens. You even talk about the development process for these games in a way that anyone would understand. Being a developer myself, this is mighty impressive. Good job!
Live-demonstrating how deferred lighting works using Aftereffects tools blew my mind. I already had a sense of how modern lighting worked in separate passes but this is just a SUPERB explanation
TL:DR : There is a layer over the hole that forces everything behind it to be shaded like the wall, which they forgot to remove originally when they made the hole. So Link's cell shading disappears ,because it's replaced by the shading meant for the wall. This also applies to other post-processing effects, such as the glow of the bombs.
Your deep-dive videos are incredible. Your videos on Wind-Waker cell-shading and Super Mario Galaxy are what brought me to your channel. I learned a lot about modern graphics from this video and I hope you'll continue to make more videos like this. You explain these things in such a clear and effective way so much that you really help unwrap the magic of modern computer graphics.
I've never seen a more elegant explanation and demonstration of deferred lighting and normal mapping. Concepts I was aware of, but struggled to understand until now. This is one of the best videos on game development I've ever watched. Subscribed.
Man, I have never seen any of your other vids before and this popped into my recommended, and this was INCREDIBLE! The explanation was in great detail, but presented in a way that didn't make it boring or too technical to understand to the average viewer, which is something that unfortunately you don't see too often these days. This obviously took a long time to make because of how polished it is, the choice of music goes well and isn't too intrusive, the custom examples in post-editing that you created just for the video, the comparisons to other games' visual systems, and everything in between! Plus, this is a topic that I haven't seen anywhere else much, so this was totally new to me and so interesting. I'd love more videos like this!
I don't care how long it takes for you to upload a video. These videos are fascinating. Only you can explain how something works in a game in a manner everyone can understand. I hope I can work with 3D some day.
I have no prior knowledge on this, but you're so talented in explaining these things in a simple, but still highly accurate way. That's not something I take for granted, as I suck ass at explaining difficult stuff and know how challenging it can be to make it sound more simple, leaving out most of the very specific jargon and only explaining the most important things. Takes a lot of time.
This video has been one of the most informative looks into modern 3D game engine lighting systems that I have ever seen. Everyone knows how well-engineered and well-developed Zelda BOTW is - with it's many dynamic systems all interacting together in beautiful ways - but taking technical deep dives into it really makes you respect the work they put into this game, even as someone who hasn't played it yet. This is what happens when a modern AAA game studio puts the time and effort into a game that it deserves, and doesn't release it until it's ready and complete.
Now compare this to Pokemon Scarlet and Violet (for those of you who don't know - despite the fact that Pokemon is one of the highest grossing franchises of any type of media, ever made in history - the games continue to be rushed messes; these recent releases are so unplayable performance wise that Nintendo has issued refunds to players disappointed with the game's performance. I just want to highlight that those games are being released years after BOTW did on the same hardware, after devs would have learned more about how to develop for the Switch, both open world games. Really shows you a lot, and this type of thing is why I stopped buying rushed corporate cash grab AAA games)
A video about a complicated topic that takes the time to fully, properly explain it so that I come away understanding what I watched? Incredible video man
This is honestly the first video I have ever watched that talks about video game lighting/rendering and makes the topic understandable and dare I say it, rather intuitive.
shhh i just didn't want to explain a depth buffer. my original script talked about a directional light which is really the 90% case but that just isn't as easy to understand visually so i changed it to a point light and told a white lie about that. but yes, the visualization used a depth buffer in that case. :)
I've actually thought the same thing, lighting/shading can really make or break the quality of something and not just video games. One great example is JoJo's Bizarre Adventure Diamond is Unbreakable, the episode Let's Go Hunting. It's considered the best looking of the episodes and, near the climax of the fight especially, it makes heavy use of shadows due to the setting sun.
There are a few tiny spots in some ruins that can do this. First time I saw the glitch was while I was under a ruined wall near central tower. The sky looked weird.
if you want a fun fact i genuinely forgot where it was while making this video, and wanted to re-shoot some footage there and couldn't find it again lol. now i know the actual answer to the mystery, thanks! i do remember scouting a bit to try and find a rather boring locale, though.
Man, all this information is amazing, but the visuals !! You went above and beyond showing all the render passes and how they are composited together, how did you even manage to get them? Amazing work !
Awesome video :D I studied game development before Deferred Lighting was a mainstay thing so I've kinda always been scared/intimated by it, but this really summarised it quite well, and I got to learn about a fun glitch!
This is seriously incredible!! It's mind-boggling just how complicated lighting can be, even though it's something most people probably take for granted, and you explained it in a compelling, understandable way. Your custom lighting demo was really impressive too. Amazing work! ...Seeing all of this in action also makes me think about how Game Freak really needs to learn some lessons from it all.
Mannnn, I really just wanna shout out the fact that, your videos out of any other, truly ride that ideal line of breaking these super intricate and clever systems down to be understandable and less intimidating, whilst *also* feeling no-holds-barred in terms of just HOW specific and deep you're willing to dive. I've watched more video essays about the mechanics of video games than I can count, but sooooo so many of them either go way too into the woods so it all blows over my head, or not deep enough to satisfy all the questions and details that I'd naturally want covered. The degree to which you're able to create INCREDIBLE supporting visuals and relevant examples of your topic, just to make sure the viewer is not just aware, but *familiar* with how a particular mechanic ticks, it blows my mind. Thank you so much for dedicating your time and passion to this, it really helps not just to satisfy curiosity, but make gamedev as a whole feel a *touch* less daunting.
14:24 _... a pretty major win for performance ..._ This actually needs some clarification. Deferred Rendering *makes a tradeoff between heavy lighting calculations per object to simplified lighting at the expense of HUGE memory bandwidth.* If your scene has thousands of lights we "defer" the lighting calculations (which would be expensive) and batch them up later. Deferred rendering requires a "fat" G-Buffer since the framebuffer no longer just holds RGBA+Depth but albedo, normal, depth, spec+gloss (or roughness+metal) PER pixel. Also, native deferred rendering CAN'T render transparency so you still need to make a traditional forward alpha pass. To have the best of the both worlds we split rendering into tiles, do a light culling pass, do the opaque pass, and do a transparent pass with Forward+.
Lights aren't the main reason we use deferred these days, but I also never said that in the video; tiled deferred is a thing, after all :). I simplified for the video's explanation since I didn't want to explain quad overshading fully, I just said something like "batching being more efficient on the hardware". Memory bandwidth on the Tegra is quite good, and the FB compression helps a ton, too. These days, the lack of overshading tends to outweigh the memory bandwidth costs.
PLEASE do more deep-dives into game rendering. I discovered your channel about a year ago with your Wind Waker video. It's fascinating learning about how much work goes into making a game look good beyond just art style, resolution, and textures, and you do such a great job explaining everything in a way that everyone can understand. I took a programming class once where we had to process image data to make a simple video player, and compared to that, I can only imagine how much more math is happening in these game engines under the hood that you have to "dumb down" for us. The fact that you reverse-engineer the game engine using rips from the exact game you're trying to explain is so helpful in seeing how everything works in real-time (or I guess slower than real time?), and I can't imagine how long that must take you to prepare. Really great work, and I look forward to your next video!
Wow, what an excellent video on modern rendering techniques! Fascinating how much they can do with combining "flat" renders, and explained in such an accessible way. Kudos!
I know this video is old, but let me explain why this happens. Its similar to how any game with multiple regions is made, if there are some limitations to their engine. The reason the cell shading dissapears in this one spot, and it might be that there are more but it's probably really hard to find, is that the devs use geometric objects that are invisible that work as triggers inbetween each wall that is made. So basically, if you are inside of that invisible trigger, which should never happen, but because this is manually made, and a person or more have to make geometry around the whole map making cell shaded triggers as close as possible to those walls, so that this doesn't happen. This again exists as an optimization technique. As anything and everything behind this trigger will not draw cell shaded geometry "be it link, or anything that is cell shaded." The reason this hole in particular is bugged"" as you could say. It was overlooked when some dev or devs were putting this trigger around the map. One more example of this if you wan't to see this optimization in person, is as of a CS:GO battle royale map. Before CS2 And with CS:GO, the game engine had to deal with allot of geometry. So because the map was so big and since the Source engine is not so optimized. Valve Devs had to resort to using the same trigger around the whole map, covering it as to make it so objects behind said trigger won't render onto your screen. This is before Oclussion Culling or Real Time Occlusion Culling was a thing in CS:GO, since it has been introduced in CS2. The drawback of that approach as in here too, is that this is completely manual and a person has to do it. Also everybody makes mistakes, and here we can see an example of that. It's a completely normal occurane, and infact CS:GO's Battle Royale maps had the same problem, but since CS:GO doesn't use any Cell Shading, it only uses this trigger so that objects behind don't render. So you can of course see a problem with that if someone misplaces the trigger and an object that should be visible on your screen is now completely invisible. Cool stuff tho, to see the normal render of link, without the extra effects.
I had just recently discovered your channel thanks to the algorithm recommending me your video's on Galaxy and Wind Waker, and I LOVED THEM! I was a bit disappointed that you hadn't uploaded in a year so seeing this video in my recommendations made me happier than I expected. These kind of video's are just SO damn interesting, I loved every second of it! The way you explain everything is just so clean and understandable for someone who has very little technical knowledge about how video games are made, but would love to learn more about it. There's just something magical about understanding even the smallest things in how a game works that you've put hundreds of hours into. I'm already looking forward to any of your future video's!
I would absolutely love more technical deep dives, you do such a great job at visualizing all these complex systems in a way that makes them approachable to the average watcher. Glad to see you again!
This channel is a gem. Such a lovely video you've managed to create, it's pure pleasure to watch. So much information, presented in a hugely entertaining and educational way.
Being a 3D designer, I spotted that it was at least one of the passes becoming corrupted or overriden. Really neat video: I love the history lesson of lighting, and the breakdown demonstration of normal maps.❤
This is the kind of video that makes it very clear that everything in software is, in fact, made by someone. People very often talk about "X frames per second" without realizing what that means and just what happens when a frame is made, why it happens this way and how it could cause problems. A major obstacle in technical litteracy is deconstructing a complex system into its fundamentals so that people can see that all choices at their core are very natural and logical, and you're very good at showing that. The way you explained things like deconstructing a normal map or recreating a lighting engine in After Effects to explain your point is very impressive and educational, I'm sure it'd be very useful to a class. You may eventually help someone start a career
The majority of this went well over my head, but it was incredibly interesting all the same. While the technical aspects were beyond my understanding, I can still appreciate the writing and production value. Really good work.
I was not expecting this video to be nearly as interesting as it was, thanks for sharing all of these details with us! It’s crazy how all of these tricks were pulled off in this game.
Love to see this kind of content. I’ve always seen glitches as a cool way to get behind the scenes of a game. Whether it be a t-posing model, a dev cube outside the map, or a lighting issue like this one. Replicable glitches show how systems work together by showing them fail to do so
Wow what fantastic breakdown! Brings me back to my younger years making mods on the old Source and Creation engines. Really makes you appreciate how much finesse is involved in making these achievements of artistry.
Oh I KNEW IT! I mean, not exactly, but I knew it was gonna be something invisible covering link when it shouldn't. It reminded me of minecraft of all things, where you can change the game's textures with a texture pack. There used to be a bug where, if you gave a transparent texture to a block that wasn't supposed to be transparent (i.e. give a glass texture to your crafting table) it'd break the renderer and wouldn't properly render the surfaces behind the now-transparent block, allowing you to essentially look through any walls you placed the block on. If you took a moment to understand the rules of this bug, you could actually gain a bit of insight into how the rendering ng engine must be working (it doesn't render any surfaces that are "covered" by a block in order not to waste resources, but there has to be a list of manual exceptions to this covering rule somewhere for transparent blocks like glass and air). This reminded me of that effect somewhat.
also about this bug, it was possible to get a similar effect to this without mods or giving a transparent texture to a block not suposed to be, i think it was related to nether portals under lava or something, someone did a video on this, allowed u to see trough the blocks since the game did not render the inside lava in a portal or something like that. also worked with door and other objects (its patched now no work anymore unless you play in a older version, works on aquaticupdate tho)
Was expecting this to be a relatively simple explanation before I saw the video length, but I learned a lot about render passes and types of filters games use, while it was all easy to follow.
This dude made a 35 minute video on this 🤦♂️
It's damn good too. Learned a lot about modern game renderering.
Coding be like
And it is so good!
@@chrispy5249 I’d make a bowl of Mac and cheese
and it held my attention better than any lecture i went to in school
My mind was blown when you started playing with the lighting on a flat image in real time. I felt like a kid again, seeing something I never could have thought existed, like magic!
Same
Mine too!
Thanks for the video suggestion in the first place, Korri!
The secret of course is that it's not really a "flat" image at all, but a 3-D image encoded as colors for representation purposes. The normal buffer holds all the direction information necessary to reproduce the first light bounce off the object when combined with the material information such as the albedo (surface color) buffer. It's like you are storing a bas relief of the entire visible scene.
The concept of modifying the light in an otherwise static scene is actually a whole field in computer graphics called "relighting" that is dedicated to finding ways to do this in real time for cases beyond the simple first bounce from point sources that's shown in the video.
@@JasperRLZ Wait, did I?? Hahahaha! What a wonderful memory I have 🤣
In 35 minutes you managed to completely explain systems that I have been trying to understand for years. No other video on the topic comes close for beginners, and a worthy follow up the Mario Galaxy video. This is extraordinary
wait a minute? you here?
This is exactly how I felt about this.
Fun fact now that TotK is out: They actually tried fixing the Bridge of Hylia one, but there’s some of it that they missed and you can still see a cel-shadeless Link (although it is a bit more finicky). It seems some changes to Link’s textures were made (aside from just Rauru’s arm) as now you can see thick, black outlines around where Link’s abs would be
damn so link got more jacked?
@@alexgao2587He ain't no twink anymore. 🥶
@@serialdreamer8386Yeah, he's a himbo now.
i don't think they missed it i just think they fixed it only where the decal is fully transparent, fixing it further probably is not possible with their engine
@@serialdreamer8386 he's a twunk now
I definitely feel like lighting is one of those things where if you do it right, people won't be sure you've done anything at all. it feels so natural when well executed it's easy to take for granted all the dedication it took behind the scenes to be that way. even coming into this video i didn't expect how many layers there were to it. fascinating stuff
Sound and music design too. They're not thankless, but they are easy to overlook when done well.
dude I love futurama
People won’t notice when it’s right but when something is off it’s jarring
@@IofSheikah totally, kind of an uncanny valley vibe
Only software engineers would think of making lighting and shading faster by rendering their frame 60 times each frame instead of once
A lot of these passes happen in parallel. The GPU is really good at doing a lot of similar operations in parallel. What the video failed to mention is that the techniques also evolve according to the hardware. In the past, these multi-pass techniques would be severely limited by the amount of parallelism available; it wouldn't necessarily slow down frame rate, but would introduce a lot of delay until the frame is ready for displaying. That would make the game feel like it has input lag, even though it has good frame rate overall; each frame takes a long time to bake in the GPU.
All of these calculations would have to happen even if they were still done separately for each object -- calculating the pixel's facing direction, computing stuff like shadows and fog and wrinkle maps, accumulating all the incoming light, and doing the material shading. It just feels like a lot more when each one of those stages is broken out to a lot of different buffers.
@@JasperRLZ It's like seeing the water is higher when it's poured into a narrower glass. It's the same amount of water, but it's taken in a different shape that may look like more.
@@JasperRLZ it is impressive that Nintendo got a high quality deferred rendering solution working so well on what is essentially mobile class hardware. A lot mobile games go with forward rendering because, for one, differed tends to require a lot more memory (need to store all the various buffers somewhere).
Not to say the forward is limited to mobile. I believe Call of Duty goes with what they call a “Forward+” renderer.
@@TheCrewExpendable The switch uses the Nvidia tegra GPU. It's a true Nvidia graphics chip so it can handle the higher bandwidth. Also it's possible to do deferred rendering on mobile via tiled rendering - basically just breaking up the buffer into smaller sections. A lot of newer games are using this method.
"When you do things right, people won't be sure you've done anything at all."
Perhaps one of my most favourite things of all is seeing how something is done right, and learning to appreciate it all the more. It doesn't just make me appreciate the thing itself, but the people doing them, and I think it's made me appreciate people in general more. The things we're willing to do to entertain others can feel like such an unnecessarily nice gesture.
that philosophy warms my heart.
Nice Futurama reference
Fantastic video. Almost everything you went over there about rendering was unknown entirely to me, despite what i thought i knew about it. when you were going over the fan theories as to why it happened, I figured it would be some kind of mesh in the way with a transparent material over it, And all the leadup to you mentioning what it actually was, all just lined up perfectly as you were explaining exactly how it all fit together.
Did u hack?
@@lederp42 Im 4 133t h4ck3rm4n
@@lederp42 unlisted video, early view and comment, then publishing a day later (we only see the publish date, not upload date)
@@lederp42 Patreon exclusive early viewing.
The section around 16:36 where you demonstrated how a normal map can make a 2D image appear to have 3D lighting literally made me gasp. It seems so obvious having it explained this way, but I'm sure it took a lot of people a lot of work to get it to this point!
Please keep making videos on whatever fascinates you most. You will have an audience :).
Indeed. I've known about Normal Maps for a while, knew what they do and how they look in their raw form but I never knew WHY they look like that or the specific details of how they work. This video explains it incredibly well.
@@Diwasho Maps like that make a lot more sense when you don't think of the channels as representing color! Just like the channels in an ordinary image are data representing color, the channels in a map represent all kinds of details relevant to rendering.
I'm just surpised that people dont know what normal maps are haha
@@snufhuffboy5101 Knowing how many people can't install a Wi-Fi printer will blow your mind then.
SAME !
I don’t think I’ve ever learned as much about a single field within such a short time span. This video has led to me understanding many systems within game dev that I haven’t been able to graft just yet. I really hope you make more videos just like this one, analyzing the reason for something and meanwhile explaining so many other useful and interesting graphics tech solutions and standards. Mad respect! ❤
I've seen a lot of normal maps from my own decades of peeking behind curtains, but never *truly* understood them until this video. I felt totally dumbstruck when you made it sound almost basic- and it's such a cool technique! I'm thoroughly floored by how easy you made all of these incredibly complex concepts sound, and how naturally you explained them. Whatever you deep-dive into next: I can't wait to hear about it.
And take your time- we can't imagine the amount of polish and edge-sanding that goes into making these videos seem so simple ;)
This is probably the best video on introduction to buffers I've seen. After effects scene blew my mind even though I've programed shaders before.
Congrats dude, this seems like it was a ridiculous amount of work, but well worth it!
My best guess for the buffer at 18:54 is probably a roughness buffer, or more likely some kind of specular multiplier/exponent buffer. The second one at 19:00 might be a mask for world weather/rain effects. (Link is likely black because the weather effects on characters is likely handled entirely separately from the world.)
this video gets into all of the fun stuff about the rendering in an easy way to understand for me! and when it finally gets back to the main question, i really felt that "a-ha!" moment when every little detail you mentioned throughout fell into place, it was really satisfying to watch!
Truly the avengers endgame of video game lighting and rendering
what you said about lighting is also true when movies went from black and white to color: because there were no colors to worry about with b&w film, all you had to worry about was the lighting, and even tho those movies have no color many have aged well because film makers were paying attention to that, whereas when color film became available a lot of movies had very flat lighting because film makers became too focused on the colors in their shots and the cinematography suffered. it really wasn’t until movies like The Godfather that we started see really dynamic lighting in color movies like we did with b&w ones.
I feel like a similar transition happened when we started utilizing CGI more as well, and we're only just now figuring it out
I sort-of understood deferred shading before, but this video made it super clear, and the glitch at the core of the video really helped me understand everything. In the demo at 16:37, are you also using the depth buffer? Otherwise it seems like the far wall would be illuminated, and the light wouldn't have the falloff on the ground.
That demo confused me as well, and depth would seem to be the missing piece of information.
Yes, that's correct. I didn't want to take the time to explain a depth buffer. My original video script used a directional light, and those two buffers would indeed be all that's needed for that case, but an animating point light looks a lot better and is easier to understand, so I took a small creative liberty and changed the directional light to a point light, even though I still don't mention the depth buffer. Good eye :)
@@JasperRLZ Ah, that makes sense. I must say the point light did make for a much more compelling visual! Loved the video!
@@JasperRLZ Just for directional lighting? Even for a point light I don't understand how the system would know the far wall is too far away to receive light without looking at the depth buffer.
I meant that directional lighting wouldn't require a depth buffer, since it has no distance falloff. Point lighting requires one.
Whoa! I actually found this lighting bug on my own a long time ago. I didn’t realize it was so significant to people online. Breath of the Wild is so big that it feels cool and rare that I found this organically.
Dude, that was amazing. I can't imagine the amount of time and effort you put into making this, but you nailed it. You made it simple and cohesive enough that even a dummy like me could follow without feeling completely lost, and that's a truly impressive feat. Thanks for all your hard work, and I look forward to the next video.
hahaa i literally typed this comment 5 seconds ago and now i see heaps of comments mentioning this exact thing. really goes to show how good this vid was
16:40 That's a really good explanation and I can't believe I never knew lighting was as efficient as this from the computer's perspective. That you can re-light an entire scene as though you hadn't even taken the picture yet is kind of a mindfuck when you break it down like that. Fascinating!
I just rewatched your Super Mario Galaxy video a few days ago and this was a nice treat. I love your deep dive videos, you manage to explain games in an incredibly visual way to the point where it's crazy you don't have a devkit or something. Keep em up!
I've been looking forward to you showing off more BOTW since your short from last year. Nobody dives as deep or creates visuals like you Jasper. This blew me away! Thank you so much for all the work you put into these videos and your website.
Definitely more of this. Reflections could be an interesting deep dive. You could start with the Build engine (Duke 3D, Shadow Warrior, etc) requiring you to create a hollow room on the other side of a mirror to duplicate geometry into. Then move up to reflection maps, reflection cubes, dynamic reflections, screen space reflections, etc.
This is one of the coolest videos I’ve ever seen… i never thought I’d ever be able to wrap my head around something like lighting and graphics in video games, it always seemed so complicated. Game development is so sick
I went through most of the video going "It's an occlusion pane!" Until you went SUPER indepth into deferred lighting, and I was half correct, but it was super interesting to learn why I was only half right, that something was being occluded, but not in a way an occlusion pane works. Super interesting and I love learning about rendering techniques in games. Great video as per usual :)
@Amy Mason You could say he became one with the Earth!
The fact that all those images are generated and processed around 30 times per a single second is insane
Fascinating. Some concepts are familiar to me, but to put into perspective all the different layers that an engine must render before showing you the final frame is insane. Now to do it 60 times or more each second sounds like a luxury.
Rendering that many frames is easy when they're so simple relative to what it would have to do otherwise.
I'm convinced that in a few years this channel will have measurably helped a generation of graphics programmers excel at their craft
seriously you always do such a great job of explaining complicated concepts without using even a hint of allegory or metaphor. a mess of normals, front buffers, back buffers, vertex shaders, pixel shaders, whatever the fuck else, somehow made accessible with good rhetorical ability
This is genuinely one of the most informative and entertaining videos I have ever found. You have an amazing talent that you should be proud of. Your channel is awesome and I can only hope you grow fast
Completely agree. Was engrossed the entire time and the whole thing was both enlightening and fascinating.
@@kiwaminski Heh heh, "enlightening"
@@theSato Huehehe. Didn't even notice the pun, nice catch
The Return of the King, one of the only people I’ve seen talk about Galaxy’s graphics and I’m so glad to see you upload again!
I love people actually explaining what’s going on behind the scenes instead of giving me half-assed explanations
Lighting is very scary for me because I don’t even know how you put all of that stuff together
This man deserves at least 1 million subs. This is the type of stuff that I find really interesting, but really intimidating so it’s nice to have something that is really easy to understand, with showing tons of visuals and examples, yet still teaches a lot and goes to over half an hour.
I agree with Squag
agreed
Definitely not scared away by the technical details. As a technical artist that dabbles in rendering, it's extremely interesting!
I like seeing the breakdown, because watching your process might actually help me get better at tackling rendering bugs that get thrown at me ha
I feel proud of myself for correctly guessing the basic idea of the problem (overzealous occlusion shading cutting out an expensive step because it thinks the pixels are occluded for some purposes but not others), but I had no idea how complicated it was. Cool video, I always love learning about the thought and effort that goes into something I barely consciously noticed.
I remember finding out that new super Mario bros U is shaded with normal maps. I thought it was really interesting how they baked in the lighting data like that. Now I’m finding out that breath of the Wild simplifies shading by converting every rendered object into a normal map to then apply the lighting onto. What’s even crazier is that we have the capability of rendering the same frame like 20 times, then compiling all the separate information, and that all happens 60 times a second on Nintendo hardware. Takes me back to when I learned how CRT TV’s work.
BOTW targets 30, not 60. It barely hits 30 most of the time too. Deferred shading on switch is incredibly poorly performant since rendering to a pixel more than once basically obliterates the GPU, and you're extremely memory capped so it's hard to store much information in the gbuffer.
You don’t render the same frame 20 times. I doubt even a 4090 doesn’t have the room to do that.
There’s a hardware feature called Multiple Render Target (MRT) allowing one render pass outputting to multiple buffers that stores different geometry data.
Also switch is pretty limited on memory bandwidth. And a full-fledged deferred shading pipeline eats bandwidth like a monster. I’m actually very surprise to see how many Nintendo first party games managed to pull out deferred shading in switch - especially many with physically based materials
This all reminds me of my 8th grade math teacher. He would do this thing pretty often where he would answer someone's question with something seemingly unrelated, but as the explanation goes on he slowly shows ties with what he's explaining and the original question, until eventually the thing he was explaining which at first seemed completely unrelated is now the basis of the exact, concise answer to the student's question. Thanks for the great video :33
It’s been a while but your videos are always gonna be worth it.
I can't IMAGINE how much work this video took to make. Absolutely phenomenal job.
I've known about normals for a while, but this video did such a good job of explaining the how and why, and the after effects demonstration was brilliant.
Also, OH MY GOD YES MORE PLEASE!
This was an amazing video! There was so much dense information made very clear and easy to understand. It felt like it was perfectly paced so that you don’t have to pause/replay to fully understand but it was also moving at a break neck pace. I’d love more videos like these. You spoke of spending so long polishing a game that no one could can notice the hard work, and I know lots of that is going on in this video
This notification popped up right as I got on UA-cam to watch something. Guess I don't have to pick a video this time!
With that and me re-watching the original Wind Waker cell-shading video a few days ago it almost feels like you're spying on me...
Heya! I'm currently doing my final project in university on Breath of the Wild inspired lighting! This video is super helpful with understanding the techniques used in the game. Great work!
To be fair in the village example, realistically forward rendering should be using occlusion culling to prevent the whole village from being rendered as well.
Yeah I think that line in particular undersells the main benefits of deferred lighting, which is mostly that adding more light sources is practically free
Even without occlusion culling, the opaque pass typically draws front to back and can early-out on fragments which are obscured... Buuuuut I think the example works well in a "lies to children" sort of way. For folks who know none of this stuff, it's a close-enough description of one of the main benefits of deferred lighting which is easy to understand.
@@HadynLander Doesn't that mean that the objects have to be sorted before being sent to the gpu? Is the sorting overhead less than what it would've costed to shade obscured pixels? What about large objects like terrain?
@@cylemons8099 nah, you can also do a depth prepass and then only draw pixels at equal depth in the main pass. Sorting would just make that a little more efficient, but a depth prepass makes sorting not necessary.
lot's of people are confused why he'd make such a long video about a single bug, but I think the bug was really just an excuse to learn about the deffered lighting and I'm 100% fine with that. I love learning about this sort of stuff.
Blows my mind to see so many passes to get this game looking like it does. Major props to the artists and the devs that made it possible.
And nice to see you back with another video! Always dig your breakdowns
Man this is basically a "rendering 101" lecture and I am excited to be using this to explaining this stuff to my friends without gamedev experience :D
I absolutely love how in depth your videos are and how much effort you put into making the visuals. It looks like you're using something like RenderDoc to get access to the buffers which is great. But it's absolutely impressive that you went the extra mile and showed the buffers getting written to and what they would look like before and after the decals. Especially on the glowing orb thing. That would've been so easy to just gloss over without going through the effort of getting a visual for it but you did the extra work and it's appreciated. I hope you take pride in this because you absolutely should.
When I clicked on this, I wanted to know why the glitch happened. And then I just wanted to know everything about lighting in games. This video is really good.
Incredible work, I'd love to see more of these! It felt like you were dissecting a luxury car engine and showing how all the pieces work together
I love your technical videos the most.
I'm surprised at how often the technique of building from first principle surprises me. Especially the things I already know. I've already used normal textures in a 2D environment, but seeing them manipulated manually in real-time in a video editor still gave me a youthful joy.
Incredibly well made video, actually made my day. Didn't think I'd be happily sat here watching 40 minutes of rendering techniques, but oh boy, I was fully engaged. Thank you Jasper!
This video taught me so much about stuff I technically knew about but didn't understand. From "what actually is deferred rendering besides something that tends to prevent devs from using MSAA" to "why is transparency crap or non existent in a bunch of newer games I've played", and "what are those material masks for in some PBR assets I've downloaded".
Your videos are so high quality. I'm in a Computer Graphics course right now, so this is absolutely fascinating.
I absolutely love the effort you put into these videos. Not only do you explain why the bug happens. Not only do you say that fixing these bugs would have taken effort by the development team.
No. You break down all of the technical issues in intricate detail. You program/visualize what exactly happens. You even talk about the development process for these games in a way that anyone would understand.
Being a developer myself, this is mighty impressive. Good job!
Live-demonstrating how deferred lighting works using Aftereffects tools blew my mind. I already had a sense of how modern lighting worked in separate passes but this is just a SUPERB explanation
TL:DR : There is a layer over the hole that forces everything behind it to be shaded like the wall, which they forgot to remove originally when they made the hole. So Link's cell shading disappears ,because it's replaced by the shading meant for the wall. This also applies to other post-processing effects, such as the glow of the bombs.
Your deep-dive videos are incredible. Your videos on Wind-Waker cell-shading and Super Mario Galaxy are what brought me to your channel. I learned a lot about modern graphics from this video and I hope you'll continue to make more videos like this. You explain these things in such a clear and effective way so much that you really help unwrap the magic of modern computer graphics.
Hey, you're an incredible video creator man. The amount of effort you put in does not go unnoticed. Well done!
I've never seen a more elegant explanation and demonstration of deferred lighting and normal mapping. Concepts I was aware of, but struggled to understand until now. This is one of the best videos on game development I've ever watched. Subscribed.
Then, when the world needed him most, he came back!
Man, I have never seen any of your other vids before and this popped into my recommended, and this was INCREDIBLE! The explanation was in great detail, but presented in a way that didn't make it boring or too technical to understand to the average viewer, which is something that unfortunately you don't see too often these days. This obviously took a long time to make because of how polished it is, the choice of music goes well and isn't too intrusive, the custom examples in post-editing that you created just for the video, the comparisons to other games' visual systems, and everything in between! Plus, this is a topic that I haven't seen anywhere else much, so this was totally new to me and so interesting. I'd love more videos like this!
I don't care how long it takes for you to upload a video. These videos are fascinating. Only you can explain how something works in a game in a manner everyone can understand. I hope I can work with 3D some day.
I have no prior knowledge on this, but you're so talented in explaining these things in a simple, but still highly accurate way. That's not something I take for granted, as I suck ass at explaining difficult stuff and know how challenging it can be to make it sound more simple, leaving out most of the very specific jargon and only explaining the most important things. Takes a lot of time.
This video has been one of the most informative looks into modern 3D game engine lighting systems that I have ever seen. Everyone knows how well-engineered and well-developed Zelda BOTW is - with it's many dynamic systems all interacting together in beautiful ways - but taking technical deep dives into it really makes you respect the work they put into this game, even as someone who hasn't played it yet. This is what happens when a modern AAA game studio puts the time and effort into a game that it deserves, and doesn't release it until it's ready and complete.
Now compare this to Pokemon Scarlet and Violet (for those of you who don't know - despite the fact that Pokemon is one of the highest grossing franchises of any type of media, ever made in history - the games continue to be rushed messes; these recent releases are so unplayable performance wise that Nintendo has issued refunds to players disappointed with the game's performance. I just want to highlight that those games are being released years after BOTW did on the same hardware, after devs would have learned more about how to develop for the Switch, both open world games. Really shows you a lot, and this type of thing is why I stopped buying rushed corporate cash grab AAA games)
A video about a complicated topic that takes the time to fully, properly explain it so that I come away understanding what I watched? Incredible video man
As Arin Hansen would say: M O A R
DID U HACK?
@@Cliffordlonghead no, it was on his Patreon
This is honestly the first video I have ever watched that talks about video game lighting/rendering and makes the topic understandable and dare I say it, rather intuitive.
Your after effects relighting setup didn't require a depth buffer? How did that point light know to not light up pixels in outside of its radius?
shhh i just didn't want to explain a depth buffer. my original script talked about a directional light which is really the 90% case but that just isn't as easy to understand visually so i changed it to a point light and told a white lie about that. but yes, the visualization used a depth buffer in that case. :)
I've actually thought the same thing, lighting/shading can really make or break the quality of something and not just video games. One great example is JoJo's Bizarre Adventure Diamond is Unbreakable, the episode Let's Go Hunting. It's considered the best looking of the episodes and, near the climax of the fight especially, it makes heavy use of shadows due to the setting sun.
Opposite of clickbait. The video was more interesting than the title made it seem.
There are a few tiny spots in some ruins that can do this. First time I saw the glitch was while I was under a ruined wall near central tower. The sky looked weird.
There's a doorway in one of the highest points of Hyrule Castle where this also occurs
"somewhere completely uninteresting" is just south of Satori Mountain apparently. No, I'm not entirely sure how I knew that.
if you want a fun fact i genuinely forgot where it was while making this video, and wanted to re-shoot some footage there and couldn't find it again lol. now i know the actual answer to the mystery, thanks! i do remember scouting a bit to try and find a rather boring locale, though.
16:27 as someone who enjoys 3D modeling, this puts a smile on my face.
Man, all this information is amazing, but the visuals !! You went above and beyond showing all the render passes and how they are composited together, how did you even manage to get them?
Amazing work !
Awesome video :D
I studied game development before Deferred Lighting was a mainstay thing so I've kinda always been scared/intimated by it, but this really summarised it quite well, and I got to learn about a fun glitch!
This is seriously incredible!! It's mind-boggling just how complicated lighting can be, even though it's something most people probably take for granted, and you explained it in a compelling, understandable way. Your custom lighting demo was really impressive too. Amazing work!
...Seeing all of this in action also makes me think about how Game Freak really needs to learn some lessons from it all.
At some point I completely forgot this video was about zelda and I was like "this is a very nice video about lighting in video games "
Mannnn, I really just wanna shout out the fact that, your videos out of any other, truly ride that ideal line of breaking these super intricate and clever systems down to be understandable and less intimidating, whilst *also* feeling no-holds-barred in terms of just HOW specific and deep you're willing to dive. I've watched more video essays about the mechanics of video games than I can count, but sooooo so many of them either go way too into the woods so it all blows over my head, or not deep enough to satisfy all the questions and details that I'd naturally want covered.
The degree to which you're able to create INCREDIBLE supporting visuals and relevant examples of your topic, just to make sure the viewer is not just aware, but *familiar* with how a particular mechanic ticks, it blows my mind. Thank you so much for dedicating your time and passion to this, it really helps not just to satisfy curiosity, but make gamedev as a whole feel a *touch* less daunting.
14:24 _... a pretty major win for performance ..._
This actually needs some clarification. Deferred Rendering *makes a tradeoff between heavy lighting calculations per object to simplified lighting at the expense of HUGE memory bandwidth.* If your scene has thousands of lights we "defer" the lighting calculations (which would be expensive) and batch them up later. Deferred rendering requires a "fat" G-Buffer since the framebuffer no longer just holds RGBA+Depth but albedo, normal, depth, spec+gloss (or roughness+metal) PER pixel.
Also, native deferred rendering CAN'T render transparency so you still need to make a traditional forward alpha pass.
To have the best of the both worlds we split rendering into tiles, do a light culling pass, do the opaque pass, and do a transparent pass with Forward+.
Lights aren't the main reason we use deferred these days, but I also never said that in the video; tiled deferred is a thing, after all :). I simplified for the video's explanation since I didn't want to explain quad overshading fully, I just said something like "batching being more efficient on the hardware".
Memory bandwidth on the Tegra is quite good, and the FB compression helps a ton, too. These days, the lack of overshading tends to outweigh the memory bandwidth costs.
PLEASE do more deep-dives into game rendering. I discovered your channel about a year ago with your Wind Waker video. It's fascinating learning about how much work goes into making a game look good beyond just art style, resolution, and textures, and you do such a great job explaining everything in a way that everyone can understand.
I took a programming class once where we had to process image data to make a simple video player, and compared to that, I can only imagine how much more math is happening in these game engines under the hood that you have to "dumb down" for us. The fact that you reverse-engineer the game engine using rips from the exact game you're trying to explain is so helpful in seeing how everything works in real-time (or I guess slower than real time?), and I can't imagine how long that must take you to prepare.
Really great work, and I look forward to your next video!
Wow, what an excellent video on modern rendering techniques! Fascinating how much they can do with combining "flat" renders, and explained in such an accessible way. Kudos!
I know this video is old, but let me explain why this happens. Its similar to how any game with multiple regions is made, if there are some limitations to their engine.
The reason the cell shading dissapears in this one spot, and it might be that there are more but it's probably really hard to find, is that the devs use geometric objects that are invisible that work as triggers inbetween each wall that is made. So basically, if you are inside of that invisible trigger, which should never happen, but because this is manually made, and a person or more have to make geometry around the whole map making cell shaded triggers as close as possible to those walls, so that this doesn't happen. This again exists as an optimization technique. As anything and everything behind this trigger will not draw cell shaded geometry "be it link, or anything that is cell shaded."
The reason this hole in particular is bugged"" as you could say. It was overlooked when some dev or devs were putting this trigger around the map.
One more example of this if you wan't to see this optimization in person, is as of a CS:GO battle royale map. Before CS2 And with CS:GO, the game engine had to deal with allot of geometry. So because the map was so big and since the Source engine is not so optimized. Valve Devs had to resort to using the same trigger around the whole map, covering it as to make it so objects behind said trigger won't render onto your screen. This is before Oclussion Culling or Real Time Occlusion Culling was a thing in CS:GO, since it has been introduced in CS2. The drawback of that approach as in here too, is that this is completely manual and a person has to do it. Also everybody makes mistakes, and here we can see an example of that. It's a completely normal occurane, and infact CS:GO's Battle Royale maps had the same problem, but since CS:GO doesn't use any Cell Shading, it only uses this trigger so that objects behind don't render. So you can of course see a problem with that if someone misplaces the trigger and an object that should be visible on your screen is now completely invisible.
Cool stuff tho, to see the normal render of link, without the extra effects.
I had just recently discovered your channel thanks to the algorithm recommending me your video's on Galaxy and Wind Waker, and I LOVED THEM! I was a bit disappointed that you hadn't uploaded in a year so seeing this video in my recommendations made me happier than I expected. These kind of video's are just SO damn interesting, I loved every second of it!
The way you explain everything is just so clean and understandable for someone who has very little technical knowledge about how video games are made, but would love to learn more about it. There's just something magical about understanding even the smallest things in how a game works that you've put hundreds of hours into.
I'm already looking forward to any of your future video's!
I would absolutely love more technical deep dives, you do such a great job at visualizing all these complex systems in a way that makes them approachable to the average watcher. Glad to see you again!
Yes, please more in-depth videos, this one is exactly what I needed to finally understand how deferred lighting works!
This channel is a gem. Such a lovely video you've managed to create, it's pure pleasure to watch. So much information, presented in a hugely entertaining and educational way.
Being a 3D designer, I spotted that it was at least one of the passes becoming corrupted or overriden. Really neat video: I love the history lesson of lighting, and the breakdown demonstration of normal maps.❤
This explanation is absolutely insane! Really professional editing and narrative.
This is the kind of video that makes it very clear that everything in software is, in fact, made by someone. People very often talk about "X frames per second" without realizing what that means and just what happens when a frame is made, why it happens this way and how it could cause problems. A major obstacle in technical litteracy is deconstructing a complex system into its fundamentals so that people can see that all choices at their core are very natural and logical, and you're very good at showing that.
The way you explained things like deconstructing a normal map or recreating a lighting engine in After Effects to explain your point is very impressive and educational, I'm sure it'd be very useful to a class. You may eventually help someone start a career
The majority of this went well over my head, but it was incredibly interesting all the same.
While the technical aspects were beyond my understanding, I can still appreciate the writing and production value. Really good work.
Such an awesome explanation that truly SHINES LIGHT on how much insane amounts of math goes into modern game graphics.
I was not expecting this video to be nearly as interesting as it was, thanks for sharing all of these details with us! It’s crazy how all of these tricks were pulled off in this game.
This was incredible, I had no idea how much went into lighting or even how they even accomplish it. Very educational, I'd love more of this content!!
Love to see this kind of content.
I’ve always seen glitches as a cool way to get behind the scenes of a game. Whether it be a t-posing model, a dev cube outside the map, or a lighting issue like this one. Replicable glitches show how systems work together by showing them fail to do so
Wow what fantastic breakdown! Brings me back to my younger years making mods on the old Source and Creation engines. Really makes you appreciate how much finesse is involved in making these achievements of artistry.
Oh I KNEW IT! I mean, not exactly, but I knew it was gonna be something invisible covering link when it shouldn't. It reminded me of minecraft of all things, where you can change the game's textures with a texture pack. There used to be a bug where, if you gave a transparent texture to a block that wasn't supposed to be transparent (i.e. give a glass texture to your crafting table) it'd break the renderer and wouldn't properly render the surfaces behind the now-transparent block, allowing you to essentially look through any walls you placed the block on. If you took a moment to understand the rules of this bug, you could actually gain a bit of insight into how the rendering ng engine must be working (it doesn't render any surfaces that are "covered" by a block in order not to waste resources, but there has to be a list of manual exceptions to this covering rule somewhere for transparent blocks like glass and air). This reminded me of that effect somewhat.
also about this bug, it was possible to get a similar effect to this without mods or giving a transparent texture to a block not suposed to be, i think it was related to nether portals under lava or something, someone did a video on this, allowed u to see trough the blocks since the game did not render the inside lava in a portal or something like that. also worked with door and other objects (its patched now no work anymore unless you play in a older version, works on aquaticupdate tho)
Please do a whole series on rendering & game-dev techniques in general, I love how clearly you're able to explain difficult stuff :)
Oh man, I’m so glad this showed up in my feed. Incredibly interesting, very entertaining and really neat editing. Instant sub!
Was expecting this to be a relatively simple explanation before I saw the video length, but I learned a lot about render passes and types of filters games use, while it was all easy to follow.
Your editing is insane genuinely. I love the effort put into this so much man
also love the choice of baba is you music!