Is really hard to find this kind of content even on internet. You make it very clear and I freaking love how much a game engine design like ECS take you to think about some important but almost forgotten things like memory chunks and cache access, in this days of Unity, Unreal and other stuff. I really apreciate your work.
I spent a lot of time researching ECS prior to implementing it in my engine a couple of years back. This was an excellent description of the problems ECS tries to solve as well as the challenges in implementing it. Great job! Now, time to get back to studying Vulkan.
Correction at 32:33 > It's actually called data-oriented programming, which is to say that you start to care about how your data is laid out in memory to favour your cpu caches. Being "Data Driven" is to say that you let the data "control" the application rather than the procedures (so logic becomes a part of the data so to speak)
Yes, but get rid of the double quotes and name it by the correct term: data-oriented design. This incorrect naming possibly invites to confuse it with >data-driven programming< which is something completely different[tm]. It's a programming paradigm versus a implementation specific program optimization approach in the case of that thing we both mean and is the topic of the video. The focus is mainly on the data layout. Where and how the logic comes into play is completely up to you and it may be a functional, OOP, dynamic dispatch or similar approach. Well, why argue about labels? Have fun programming :)
No components are data driven design, because the data (e.g. the scene, the prefab, etc) are what define the objects not the C++ developer. The whole memory side is an expansion on top of this which is data-oriented design which is related and often can get confused which is where the ECS approach (e.g. array of specific component types) comes in.
Hey, beginner here, I just wanted to say a huge thankyou to you for making this entire channel, man this is a gold mine. I don't understand anything in this video yet, but I like to listen to this while I sleep, so that I know what I'm going to deal with when I learn enough :) I also wanted to suggest instead of reacting to games/game engines, could you break down some games, maybe talk about how you think a particular mechanic/ interesting feature would have been implemented?
fantastic explanation of ECS. I appreciate these broader conceptual lessons. I'm curious to hear more about how the system is implemented -- e.g. how the challenge of having one system (say physics) need to access multiple component pools is handled. I suppose for your purposes entt handles it, but if you have anything to say on that point I'd appreciate it! This video greatly helped me understand what an ECS is, thank you very much for making this quality informative content, I even took notes lol. Thanks again
this video popped up for me out of plain blue, but I am so happy for having watched all of it. First of all I didn't know that everything within the game is unified into entities. I had a hunch, but didn't know for sure. Surely there are some performance optimization to this system, but good to know that this is the base case. Secondly, composition! Yes, I always thought that it's so much better than hierarchical made-up inheritance in context of more complex systems (which AAA games definitely are)
I'm coming from Business Software development and this was really useful. I noticed after a little while of trying to program a game that the SOLID N-Tier OOP architecture becomes incredibly confusing and complicated when designing even simple games. There are so many permutations of entitites that I was ending up with mega-classes and 5+ of abstraction in my classes. So few game engine tutorials talk about architecture and design.
YESSS ECS, I really wanted to see it. By the time I'll get here this portion will even be complete (since I'm at 40th video), NICE. Thanks a lot Cherno for these videos.
I remember my first game programming book for android had something along the lines of a MeshTexture and TexturedMesh classes, for a Tetris game. At that point I gave up. Then a simple Arduboy (8 bit game system for arduino) game in C had arrays of positions, and arrays of other properties. The "entity" was an index into the array. Funny how today large systems revert to the simple approach.
@@SuperG316 The third edition is relatively new, it's from 2018. Yan's video reminded me about parts of it. Jason Gregory talks about ECS, Job systems. It's just awesome.
EnTT is really damn good. Like, really! The developer behind it is also really friendly and hands-on helpful. I actually built a simple entity compinent system back in 2011. It was a good experience, but yeah, EnTT is amazing. (PS since its EnTTwith capital T’s I guess its pronounced N.T.T. or “entity” if you say it fast, but yeah, I see why you would call it “ent” in the video, to disambiguate). I personally also wrote my own wrapper to abstract it the way I needed. I think tha.t’s a good approach. EnTT gives you a ton of tools, but they may not fit oerfectly into the rest of your engine without some gkue that needs to be wrapped. So very much agree with that choice. I’m not an engine developer, though, just a hobbyist with lots of non-game experience who has spent a bunch of time with EnTT. So my opinions may not be completely relevant :) As an aside, I wish the ECS terminology were different. Component and System are too ambiguous (system isn’t too bad, but component clashes with too many other things in software). Personally I’d like “trait” instead of component: its the traits you assign to your entity that define its behaviour. And.. behaviour instead of system: its the logic to implement a behaviour, based on the traits your entity has. Then I can use “system” for broader things like “rendering system”. But yeah, unfortunately the terminology is already established.
I actually wached whole lot of videos and conferences about ECS, but yours was the best explanation or most satisfying i guess. maybe i forgot previous ones and i just remembered now. BUT in any case, i belive you have too much knowledge about computer an programming and i would love to hear from you these side topics as main topic not like "only mentioned in a larger project". for example if you were to do a playlist on ECS im sure it would be more popular then the game engine series. because other then games, none of the apps are using this kind of architectures even if they need it as much as games do.
great video. Could you make a follow up video on combining the ECS with spatial culling, for example if you only had the CPU to calculate the nearest 1% of the entities in your registry?
Sir, I just want to say: T H A N K Y O U OH my gosh, I have been working on my 2D Unicode-text game engine for about a month now, and I have been building my entity component system exactly as you describe it at 26:33 - I have been HATING it, and desperately searching for a better alternative. It's such a gross solution, I agree! You have enlightened me. This is such a better system, I'm so frickin excited to implement it. This is by far the best video I've watched talking about how to build the architecture of an application - especially a game engine. You are doing god's work here, I'm so thankful of your content, especially this video, you really cover the things most people don't. Your teaching style of pretending to work through the problem logically with the viewer is absolutely the best way to teach, but nobody does it because it's time consuming & you have to know exactly why we have come to the solution we use - so thank YOU for doing it, regardless of that it's so time consuming and so much more difficult - it instills your knowledge in your pupils, AS WELL AS teaching them how you come to that conclusion. Thank you, thank you, thank you - holy fuck, this channel is a godsend. I'm sixteen. I'm a self-taught programmer who's been teaching himself C++ for 3 years now. I have no friends who like programming, and I have no teachers who program. Nobody I know in my small town in Kansas understands what I spend all of my free time doing. I'm luckily really good at tests, and I'm going into an early-entry-to-college program at Fort Hays State University called the KAMS program next year at the age of 17, which will allow me to take college classes on-campus for dual-credit highschool/university credits. This means that I'll be graduating high school with 2 of my 4 years towards my bachelor's degree in computer sciences completed. I have been working my ass off to make sure I can build applications as a job in the future - and I'm relying entirely on online references such as yourself. This type of content at this quality is sorely missing on the internet - trust me, I've been self-teaching for 3 years completely alone. You are what people like me rely upon to make it to where we want to be, and to learn to do what we want to learn to do.. So thank you again.
Well this is crazy. I also grew up in a small town in Kansas and could have gone into KAMS. At the time though I was pretty sure I wanted to go into Economics even though I had been playing around in Unity and with c++ for most of high school. Ultimately I ended not going but still graduated with my Bachelor's in economics and marketing in 2 years. But I hated it and the work. I moved to NYC, went back to school for game dev and I'm much happier programming than I ever would have been working in business. But until I moved to NYC I just assumed all schools lack any form of coding opportunities and that everyone had to be a self teacher. One of the first jobs I got though in NYC was teaching an after school class for programming. It was for kids K-4th. I realized very soon a lot of schools have started teaching programming in some form or another. Though I do think being able to learn new technologies by yourself is frankly more important than anything you'll get at University. Not to say college doesn't have something to offer, but for programming most of the things you need to know you're going to have to learn on your own, it's just the nature of the business. Congrats on getting into KAMS and keep with it.
Dude. Same. But I don't live in kansas and I am not doing college work (running start). That's really weird. Same amount of time for c++ and everything
If I remember correctly EnTT uses dense hashmaps for component storage and then access those components through entityID. This may sound like a good idea, but in reality is very slow (compared to simple arrays). Also remember that components are unsorted! So if you want to loop through entities that use 2-3 types of components you are basically jumping around in memory. I work on a project that simulates hundreds of thousands of agents and I can tell you there is a better way. When you design an entity in general, you structure it as a unique type with some set of components. It is better to make a container that stores specific type of components and link it to the storage unit of that type in your scene, then each entity "type" can use something like std::tuple to store that containers. You can access components with 2 indices (entity typeID and entityID), so you don't need to have any UUID within the entity, or components. Only downside to this solution is that you cannot design your entities in the editor, because you need to define them statically. Also, if you use this design, you don't have to "submit" data to the renderer, because you can sort component containers by material type as they are created.
Awesome vid. How does this combine with the scene graph? Should it combine with the scene graph? I’ve seen some stuff online that using the ECS for the scene graph is an abuse of the ECS and they should be completely separate, but the entities have transform components and meshes, so obviously we should be at least treating that data in two different ways instead of allocating it twice (once for the ECS, then the scene graph). Thanks for any help!
Wow at the moment when I literally was implementing an ecs system into my engine and this came out , I have always felt the shortcomings in my design and here hope cherno will show the better version of it
Was totally expecting a pure ecs from scratch. Disappointing a little by the use of a library instead. Really liked the explanations. I have been a little behind on your game engine series and was kinda tinkering with a project I've had for a while. Turns out this is the 2nd time now something I was doing was on the exact day you put out a video so that's pretty cool. Keep making great videos, I enjoy learning!
holy fucking shit my brain just blew up. i've been writing dodgy code like this for years, thinking it was the only way to implement a high entity-type count and thinking i was dumb because i couldnt keep up. this is HUGE (because of how small it is)
I have question, in Flutter i have to use system to move widgets around, doing it right it is efficient. So a ECS ontop Flutter would be something less efficient since high level programming but all the same efficient. My idea is Nintendo Switch they deliberately built on processor with graphics processor to make it very hard to make simulator since it is so closely built to processor and cache. ALso in flutter there is delay problems one can avoid by putting in 'invisible' movements before 'real on screen' movements, to mitigate lag. I understand the problem of making the 'sprite' dynamics more structural.
Thanks for sharing the knowledge. I have a question regarding ecs especially how to use it to construct the inventory. I wanted to make it like unity3d scene graph but with ecs it’s impossible to do that. Maybe you have suggestions or you can show us how to construct such a complex hierarchies with ecs? Thanks in advanced!
Maybe you should take a look at Milton, a drawing program with an infinite canvas. Makes visualizing stuff like this a lot easier since there is always enough space. Even better, it's free.
Could you please talk about the interface between ECS and our Renderer? I'm having trouble figuring out how the Renderer will extract data from the game simulation.
The components in your entities are pointers, so they can point to a cache-friendly store of each type of component, and you get the benefit of both systems. Contiguousness for quick iteration rendering etc, and discrete entity access in scripts etc if needed. What am I missing?
Encapsulation, probably. The idea of classes whose data is accessible through some alternative data structure is anathema to OOP purists. What you're suggesting is hacking the OOP encapsulation to have your cake and eat it too. An ECS is simply a more formalised way of doing this which ostensibly will have solutions to problems that you'll run into along the way. Like all programming paradigms, nothing is automatically the right way, if you see light at the end of the tunnel then head for it, and ignore the people telling you it's probably an oncoming train. They aren't necessarily working on more information than you have, they just have different priorities.
17:00 “This is already a problem. We have four entities. Can you imagine making a AAA game?” This is how I was introduced to modern engines via (old) unreal and unreal script’s class-based architecture. _Any_ tweak to behavior of something (or *gasp* combining already existing behaviors from two different things) required wholesale refactoring. At least I learned that flavor of pain _before_ discovering unity’s component-based architecture; otherwise I would have never had the patience to mod at all. Yan’s point of thinking this high-level pattern through (to how would it be used to make new features) should be highlighted with a neon sign: the flexibility of interchangeable behavior (modular) is the point of an ECS, and it has the added bonus of being made up of smaller (easier to maintain) atomic components. Atomic and modular is the way to go! [EDIT] The data-driven design explanation is the best!
A question for anyone reading this :) I have implemented my own ECS (in Unity) in pretty much the exact way subscribed in this video. But there's one part I haven't figured out the perfect solution for, which is: How do Entities interact with each other? As an example: Let's say we are making an FPS and an Entity gets shot (or actually his collider collides with a bullet), then this effects many components of this Entity. His health (HealthComponent) will change, an "injured animation" should activate (AnimationComponent), his AI should respond, maybe he screams, etc, etc, etc. Thus, how do we effect all these components when an Entity gets shot?
You could manipulate each of those components individually from the code that handles the bullet. However, if you want a more extensible and flexible solution, add a component "TakeDamage" to the entity, and have the systems that manage those other components react to the new component.
how can u store multiple classes that inherit from a base class ? for exemple if u have a component class (base class ) and then u make Physics Componentn , audioComponent etc... how do u store them ? i mean the technical implementation .. because if u store them in a list of component ( List ) you cant access all their fields and methods ( if a PhysicsComponent "A" in that list then its considered as a simple Component not a PhysicsComponent .
in short-you can implement gameplay. You would make it in a different way than you are used to. The swords data would be contained in components. Maybe there is a "damage" component, or a transform component, doesn't mater. What's important is that the sword itself is only represented by several small modules of data, so that the project remains modular and easy to work with. The sword doesn't have any behaviour, the behaviour can be handled by outside systems. The sword is just a number, marking which components it uses. The way that you normally program is to organise things into separate "objects", this makes sense as it's how our real world works. However, for a computer, it is much easier if the data is packed close together and in logistical positions. Having game Objects filled with scattered behaviour and data not only costs performance but can also be a real head-ache to work with. The problem with ECS is that unique behaviour is not very easy to implement. It's kind of like making the rules for your game, and the data of your game separate in two different arrays.
Towards the end you mention that a system will iterate through all component of a certain type instead of through entities. I understand how that is better for performance, but how do the systems check if this component is from an entity with another certain component? In your example you had 5 being the entity id that links components together but how do you lookup other components that are also from entity 5? It seems like a lot of lookup through an array, which I don't know how it affects performance.
I think the idea is You need to optimize the critical path so stuf that done frequently most of the time. Like the render system go through all the transforms and meshes. Vs tha gamer select a radar dot and the event created look up some data from that entity to be displayed in part of the UI. So seek of a component of specific entity is rare thing. While every system of the ECS is updated very frequently.
Ideally you implement your ECS to support Systems that process two or more component types, since that's usually what you want here. ie. a Model component needs a Position component to draw in your Render System.
The bulk of your critical processing in an ECS should not go from entity->component. That should be a rare case. Instead, most critical systems will fetch the container containing one or more component types directly, completely bypassing the entity. They don't bother with entities at all. Like: for (Motion& m: scene.query()) // do something with 'm' This doesn't involve accessing entities whatsoever, even in the underlying implementation of 'query'. The reason to use something like an entity ID bitset which identifies the types of components an entity has, and allow some way to fetch specific components from a particular entity, is for rare cases... not your critical paths. For example, your engine's editor GUI might want to list all component types available for a given entity in some tree/list GUI control. In that case, you are going from entity to components but that's not a particularly performance-critical case compared to something like a rendering system or physics system which has to loop through components every single frame. Something like a physics system shouldn't care what entity to which a motion component belongs. It only wants the motion components to transform. So those types of systems don't even bother to access the entity at all. They bypass all that and go straight to the container of motion components to iterate through very efficiently.
@@RamHomier Cheers! This is also why we don't store components directly inside entities. It would be more convenient to implement that way, but then we'd have to do a linear-time search for a component (or at least a layer of indirection) starting with an entity in all of our critical loops. The reason we store components outside the entity in their own individual containers is to be able to bypass that entity and just get the entire container of components and loop through them directly.
31:45, unless I am wrong, isn't the cache line start location calculated by removing N least significant bits from the address that you asked for instead of starting on a said address?
One question that has lead me back to inheritance: How do you efficiently deal with an entity's deactivation or destruction, especially if that entity has a lot of components? Currently in my engine I'm using multiple inheritance where each parent is like its own component; the scene stores objects contiguously by type in a tuple of vectors, and the game loop on each operation only iterates in the vectors which content inherits from the needed "component" parent (which vectors is determined at compile time). This way, destroying an object is as simple as erasing it from the vector containing its type. But with an entity-component system, how do I define an entity's destruction? Do I have to iterate all component types vectors looking for the id and removing the components??
Usually they are POD-like structures, so you can just overwrite their components. And since it's all allocated on the same array, you can just free the memory for the whole array.
I already knew the pattern, but a question always comes to mind. Suppose I am actually cache-efficiently iterating over my Poison components, and a certain point a Poison comp. asks if Health comp. exists on entity 0xdeadbeef to update it. I should fetch and update the other component, located in another array. Would this "jump around memory" disrupt cache efficiency? Or maybe what I want is a "join iterator"? Can the cache optimize a simultaneous iteration on two arrays?
Does ECS try to be a "CPU vertex buffer"? I see a similarity between your ECS data layout explanation, and your opengl vertex buffer layout explanation.
I implemented an Entity Component System in the game I am developing on my UA-cam channel. The system in my design has all the logic that operates on all the components registered with the system. So I have an AnimationComponentSystem class that operates on AnimationComponent classes registered with animation component system class. So this is very similar to what you have described in your video.
ECS ( entities, components, systems) entities is just IDS. with comonents behavoirs. systems receive entities and make something with entities components will be like: VisualSystem (entities, args) { for each entity in entities { const position = entity.components.find(PositionComponent) const visual = entity.components.find(VisualComponent) if(position AND visual) SOME_RENDERER.draw(position.x, position.y, visual.width, visual.height, visual.color) } } SoundSystem (entities, args) { for each entity in entities { const sound = entity.components.find(SoundComponent) if(position AND visual) SOME_SOUNDER.play(sound.source, sound.volume) } } PositionComponent { x: number y: number } VisualComponent { color: stirng; width: number height: number } SoundComponent { source: string volume: number } Entity { id: string components: [] } systems = [VisualSystem, SoundSystem] bob = new Entity() bob.components.push(new PositionComponent(100,100)) bob.components.push(new VisualComponent('#f0000', 128, 128) tim= new Entity() tim.components.push(new PositionComponent(200, 100)) tim.components.push(new VisualComponent('#00000f', 128, 128) bird = new Entity() bird.components.push(new PositionComponent(10, 10)) bird.components.push(new VisualComponent('#ffff00', 64, 64) bird.components.push(new SoundComponent('./assets/sounds/bird.mp3', 100) gameloop () => { delta = SOME LOOP TICK for each system in systems { system(entities, { delta } ) } }
Will our Renderer become a system of our Entity Component System? The Renderer feels like it could be "the last system that runs in each frame"......... but I'm not sure that it makes sense to add the Renderer as part of the game state simulation.
Cherno, just a question: in my ECS, systems are not iterating over entities, because it would mean iterating over all of them per system. Instead, every component has an update function, and I pass the systems in as argument (all systems are wrapped in a class), and then each component puts itself into a list in the systems it needs to be in. So for example, the renderer component puts itself in the RenderingEngine (systems are called engines), the PhysicalObject puts itself in the PhysicsEngine. And so on. This means iterating through them only once, and working with a local list later, as after the components are updated, every system is updated. I think it performs better. What's your opinion?
Well most people think components should be raw data, so them having an update function is counter to that. Also, you're calling functions over and over and over while passing in a system, this is trashing your CPU cache each time. Instead, a lot more cache friendly way is to loop over your unordered array of components held by the system in a single function. Something like RenderSystem, which has a link to or holds a list of components that are renderable, you call RenderSystem::Update(const float ts) and Update loops over all the components held in the list. As I said, your method is blowing up your cache each time you call the components update function.
Are you deep copying the objects into each system ("engine" in your case)? Or just referencing/pointing to them? It makes perfect sense to me to avoid iterating over entities in systems and just access a sequence of components directly without the branching and memory overhead, but I can't quite picture how you are injecting your components into their relevant systems in a way that doesn't involve additional layers of indirection (and sporadic memory access patterns)... unless your codebase is such that you never have more than one system accessing a particular type of component.
@@darkengine5931 I use pointers to get the objects there. Actually everthing needs to be a pointer because the entity object can be inherited from to specialise some cases. This system works so far with no performance problems
@@andraskmeczo575 I see. If there are no performance issues in your case with that solution, I think it's all fine and good. Typically I would think that would translate to a lot more cache misses though, as the stride to get from one component through a pointer to the next might be quite variable and range from very small to very large. But you usually need to be processing something like tens to hundreds of thousands to millions of components per frame for that sort of thing to become a non-negligible overhead. In many cases though, I think the performance of ECS is overstated and its flexibility understated. I think the flexibility is going to benefit a much larger range of projects than the performance of cache-friendly contiguous component access.
@@darkengine5931 I see where the struggle can come from. As per se I allocate the entities and components individually, but I'm working on a factory-like system which allocates a lot of the same type, and gives a pointer to one when an allocation occours. This ensures that the entities and components are close together, and while the inherited types need separate allocators (for types that are used regularly, like TerrainNode, GUI elements are used like 5x at most so it doesn't need optimisation. And my system ensures that the inherited types are dealt with together, not mixed up with the regular entities, so in theory, there should be no cache misses, the only problem might be that there should be two caches, one for the entities' block of memory, and the other for the components'.
It took me a lot of time to realize that continuous memory problem and even after that I implemented it using unordered_map which probably saved time if an entity doesn't have that component. This video showed me a big flaw in my ECS, THANKS! for the video. And I'm probably going to switch to enTT instead of writing it.
I have actually a couple questions. In the implementation of the ECS i have seen it uses a Arrays of Components and these arrays sized based on the pool. So lets say you have 50 components and your "1 Million" Entities. That would mean you would have 50 Object Arrays (and some bit vectors) that have the size of 1 Million each. And here is the trouble that I have this: Wouldn't you trade in this case "CPU Speed" for ram usage? Yes null objects are not as bad since they only have the pointer reference but this kind of stuff adds up. My question is: Am i overthinking things like that? Or is this not as bad as i am thinking. Another question. How would you deal with a chunk like system where you could say the same thing as for a an ECS System (give me only the entities i need) This is something that kinda melts my head. This is for me why i am planning to go with the "Worst" Inheritance system, because I am getting what i am doing, but i know that a ECS System would be better, but the issue is that there is sadly not that much resource out there how to use a ECS System with a grid System where you are not forced to iterate over all entities in the world to find what you actually want. The "Whole I just want my components" kind of thinking again. I am sorry for bothering you with these kind of questions. This is just melting my head. Thanks for reading. Sorry if I offended you in any way.
Having 1M entities does not mean all of them have 50 components. So the components arrays are on average much smaller than 1M. And as long as each component only contains what is necessary to represent it (i.e. minimize its memory footprint), it should still be fine in general. Of course if you do have 1M entities and each of them needs all 50 components, maybe you are building your scene wrong. Possibly similar entities could be grouped into one entity, vastly reducing their combined footprint.
@@Wosser1sProductions the arrays still would have to account for the largest Id though. If entity 900000 has a position component the array still would need to be at least 900k big even if it only contains 5 elements. All others contain nullpointers. That was my point of why I am confused with ecs. I even addressed that. And unless you use a dictionary (which has its own performance problems at that scale) you are forced to have some kind of entityid to component array mapper that has to be the exact same size as the biggest entityid. (Writing on phone)
having array per component is bad coz as you said for most of components this array will be just empty with some filled slots somewhere in the middle for example. and since for performance reasons it's better to not have array of pointers but have an array of actual structs - you will waste a lot of memory basically allocating unused components. you can optimize it by introducing "chunks" and "archetypes" (e.g. Unity calls them like that). archetype is a set of components (e.g. archetype1 describes all entities with transform, mesh, rigidbody, and archetype2 describes all entities with transform, mesh, rigidbody, animation). then you have chunks which contain data for the archetype (e.g. for archetype1 it will be 3 fixed arrays of N elements - 1 array of transforms, 1 array of meshes, 1 array of rigidbodies). N is the size of the chunk (and you can play with it to find the best for performance), the bigger chunk size - the better for CPU cache, but more memory wasted bringing you closer to initial "array for components of one type" solution. too small chunk size - better memory management but worse CPU cache friendliness. for example you have chunk size of 256, if you have 200 entities of one given archetype - 1 chunk is created and a bit of space wasted. if you create 200 more - new chunk will be allocated after first one is filled. and ofc you need to cleanup chunks which become unused, so more bookkeeping required than plain 1 array of gigantic size per component type. for inspiration check out Unity's implementation (or at least the idea) of ECS.
@@user-jn9pg3lx7g you still need a translator/mapper from entityid to component index and that mapper (most likely an array for performance reasons) has to as big as the biggest entityid. There is no way around it. And why I said that it would be a null pointer is because I use a language that does not support pointers by default. Could you show me a good implementation of ecs that does not have this exact issue of "wasting memory based on biggest entityId" and does not use dictionaries? Also how would you implement a chunkgrid into the game with ecs. And I am not talking about "memory chunks" I am talking about chunks like minecrafts grid system. Because iterating over every single entity to find something nearby is a bit laggy? Or how would you implement a grid optimization for nonstatic entities?
Hey. I also write a game engine. And I have big problems with understanding how the scene node graph system works. I already read "Game Coding Complete, Fourth Edition by Mike McShaffry" book several times and I just can’t figure it out. Please make a video on this topic.
@@Alex-fr2td Honestly I do not know. :) When developing software there are multiple factors to consider when deciding what goes and what is left out. I would predict that ECS is not the friendliest way of introducing game development to its users. And unless you have a lot of going on in the game and it has to run on weaker hardware, it did not matter much.
@@Alex-fr2td I would like to add that every year it is expected games to offer more and push bar higher... and we are starting to hit the limit of what computers are capable of doing and have to find a way to optimize.
Say I wanted an entity that contained a mesh. Then I wanted another mesh attached to this mesh, under a transform. While I could make 2 entities and put them in a transform hierarchy, wouldn’t it be easier to a link to the parent component in each component and get rid of the idea of an entity altogether? Surely my entities would just be a transform hierarchical structure of components, and the “root” would be a component with no parent? Then during a rendering system update (for example), each mesh would be iterated over using the mesh components list in the scene, then a walk up the tree would be needed to in order to get the full transform of this mesh. The extra work caused by this way of doing it (calculating transforms of the current transform within the context of the “entity” tree) could be negated by “caching” this data in the system as you go? Or have I misunderstood something?
@@bocdev4889 You might be a bit tunnel-visioned on the mesh motion hierarchy and its evaluation unless that's all you need at which point an ECS would be overkill. Typically entities in an ECS store a whole lot more than meshes and/or parent/child links and transformation matrices. The point of associating them to entities is because entities might be associated with all sorts of different component types but we want one single entity unit at the end of the day in particular contexts. Even in the context of motion hierarchies, you might have entities other than meshes that want to be parented. For example, a light source that doesn't store a mesh might want to be parented to a lamp mesh by the level designer, or a particle system, or a light parented to a camera, etc. etc. By having entities separated from components, and different component types from each other, you can still organize multiple component types by associating them to specific entities, but without the entities having to store some indivisible super bundle of data that gets inefficiently and indivisibly loaded into the CPU cache.
Weirdly enough I don't see any comments here about unity using that inefficient ecs system by being one of the most popular game engines. (Ye, Ik about DOTS but it's still not production-ready system so don't remind me of that)
What is your opinion on Jonathan Blow's comments on ECS? I myself think that if you are making an indy game, then you don't need a separate engine and can just go from scratch, but AAA dev teams need consistency, claritx and organisation, which means that ECS becomes necessary IMO, or at least sth like it. Its like how rigorous data base designs are essential in this day and age.
My opinion is it depends on the scale and complexity of the game, and also what your goals of using an ECS is. Eg, if you are writing a AAA game then you probably need a data oriented design and ECS cab help with that. If you’re writing a game with many entities with complex behavior, then ECS can help with managing this as you can compose the behaviors. Similarly, if you expose this to your designers/content creators, then an ECS can allow you to configure the entities in a flexible way. ECS isn’t the only way to do those things, but it seems like a good fit. If you don’t need those things, maybe because your game is smaller, simpler or you are handcoding/scripting all the behaviours (versus giving your designers a way to configure it in a tool), then ECS is totally overkill and may just add complexity. Jonathan Blow is making comparitively simple games with fewer and simpler entities, at least, Braid and The Witness, and he’s coding everything himself so he can create twekaed entities as necessary, so I can see why it wouldn’t be a great fit for him.
If you want to write a game, write a game. If you want to write a game engine, still write a game. Most successful game engines were originally written for some specific game (Unreal Engine for Unreal, Id tech X for Doom Y/Quake Z, ...). Even Unity was done for some game that they never released.
It depend on the need for the type of game and it complexity. For AAA games the bulk of the work force is making high fidelity content , like corridor shooter. On the other end a indie RTS game with many different types of entity need for more advance ECS is higher demand. Even if the content is 2D tile based. AAA titles are render load heavy so a few entity with few types is often not the problem . It could be up to 12 to 200 entity CPU load and cache misses is not the problem but render task is main load. For RTS 2D game the render task is light but the CPU needs to handle 2500 units with bullits missiles and composed units / entities is might be 10000 entities and a AI load for 2500 unit + 250 guided and 1000 unguided misiles.
How to implement the connection between logic and entity and their components? That what is always a hard for me. Especially if we want to do the logic in the data oriented design. Maybe you could share some thoughts about that topic?
If I understand correctly, a crude answer but perhaps helpful if you are stuck in an object-oriented mindset is to start with an object: class Transform { public: // public interface/logic ... private: // data Matrix4 local; Matrix4 world; }; And we move the data stuff into a component: struct TransformComponent { Matrix4 local; Matrix4 world; }; And we move the logic into one or more systems (ex: a physics system). Then you associate the instances of the transform component with whatever applicable entities. That's a sort of roundabout answer to your question if I understood it correctly, but there's not much more to it than that. If OOP is comfortable to you, then you just got to get over the discomfort of moving the logic into a system in a separate place from the data. The main thing you lose is encapsulation including information hiding. In exchange, you get tremendous amounts of decoupling, flexibility, and more breathing room for optimizations. If I misunderstood your question somehow, please let me know and I'll elaborate, but I tend to get the sense that a lot of the mental block that people tend to have with ECS is that they're so stuck in an object-oriented way of thinking about design when the ECS is more of a procedural way of doing things (albeit a very flexible and organized form of procedural programming).
You typically don't. ECS encourages a design mindset closer to duck typing (minus dynamic typing unless you are implementing it in a dynamically-typed language). If something has legs, it can walk. It doesn't matter if it's a robot or a spider or a human. What matters is that it has legs. If you absolutely need to distinguish the "type" of something from another, you do it by attaching a component or storing something in a component that identifies it as being like an enemy vs. a friendly NPC, e.g. If you've ever done generic programming this should be somewhat familiar. A generic sort function doesn't care if you pass it one type of container or another. All it cares is that it's a random-access sequence of any type and has the required stuff needed to sort it. It's not important whether it's a vector or a deque or arraylist or even if it implements any explicit interface in the context of this generic sorting function. All that's important is was it has, not what it is.
"give me a Vertex Buffer, I'm good to go"
Gonna try that one on a date
Is really hard to find this kind of content even on internet. You make it very clear and I freaking love how much a game engine design like ECS take you to think about some important but almost forgotten things like memory chunks and cache access, in this days of Unity, Unreal and other stuff. I really apreciate your work.
Wow! Finally something about ECS :) So happy about this series! Thank you very much! :)
I spent a lot of time researching ECS prior to implementing it in my engine a couple of years back. This was an excellent description of the problems ECS tries to solve as well as the challenges in implementing it. Great job! Now, time to get back to studying Vulkan.
Correction at 32:33 > It's actually called data-oriented programming, which is to say that you start to care about how your data is laid out in memory to favour your cpu caches. Being "Data Driven" is to say that you let the data "control" the application rather than the procedures (so logic becomes a part of the data so to speak)
Did notice it to. Data driven vs hardcoded. Data oriented vs object oriented . Component vs hiarchial .
Data oriented is also support data driven .
Yes, but get rid of the double quotes and name it by the correct term: data-oriented design. This incorrect naming possibly invites to confuse it with >data-driven programming< which is something completely different[tm]. It's a programming paradigm versus a implementation specific program optimization approach in the case of that thing we both mean and is the topic of the video. The focus is mainly on the data layout. Where and how the logic comes into play is completely up to you and it may be a functional, OOP, dynamic dispatch or similar approach.
Well, why argue about labels? Have fun programming :)
Yep, I meant data-oriented design
No components are data driven design, because the data (e.g. the scene, the prefab, etc) are what define the objects not the C++ developer. The whole memory side is an expansion on top of this which is data-oriented design which is related and often can get confused which is where the ECS approach (e.g. array of specific component types) comes in.
@@reductor_ Sure that would be it. This is just a correction of a term used in the video
This is by far the best video I've seen on the subject of ECS.
Thank you! Got a like and a sub!
Hey, beginner here, I just wanted to say a huge thankyou to you for making this entire channel, man this is a gold mine. I don't understand anything in this video yet, but I like to listen to this while I sleep, so that I know what I'm going to deal with when I learn enough :)
I also wanted to suggest instead of reacting to games/game engines, could you break down some games, maybe talk about how you think a particular mechanic/ interesting feature would have been implemented?
This video was unbelievably good! Thank you Cherno! Much love
Yessss! Finally ECS! Thanks you for all this videos! You just doing impossible stuff.
Great video! Love the way you explain things. Very clear.
Please keep them coming. Don't think you're going too long. Interesting topics never seem long. Cheers
fantastic explanation of ECS. I appreciate these broader conceptual lessons. I'm curious to hear more about how the system is implemented -- e.g. how the challenge of having one system (say physics) need to access multiple component pools is handled. I suppose for your purposes entt handles it, but if you have anything to say on that point I'd appreciate it! This video greatly helped me understand what an ECS is, thank you very much for making this quality informative content, I even took notes lol. Thanks again
Lucky me that you uploaded this video, I've been trying to learn this lately
Wow. I was literally about to write a little game in C++/SFML with "entt" for ECS. I love your videos!
EnTT is awesome
I will make my own
this video popped up for me out of plain blue, but I am so happy for having watched all of it. First of all I didn't know that everything within the game is unified into entities. I had a hunch, but didn't know for sure. Surely there are some performance optimization to this system, but good to know that this is the base case. Secondly, composition! Yes, I always thought that it's so much better than hierarchical made-up inheritance in context of more complex systems (which AAA games definitely are)
Huge thank you for this explanation!
YES, I have been waiting for this
I'm coming from Business Software development and this was really useful. I noticed after a little while of trying to program a game that the SOLID N-Tier OOP architecture becomes incredibly confusing and complicated when designing even simple games. There are so many permutations of entitites that I was ending up with mega-classes and 5+ of abstraction in my classes.
So few game engine tutorials talk about architecture and design.
I am using this to simplify my business app right now. And I think it works well in other domains too
Lovely explanation, as usual!
Perfect. Thank you for uploading :)
thanks! you explain ECS very clear.
I prefer this method over inheritance hierarchy. You explained this well, thanks!
Thank you! I really enjoyed this video! It's so easy to digest and quite entertainment also :)
Keep it up Cherno... This is great
YESSS ECS, I really wanted to see it. By the time I'll get here this portion will even be complete (since I'm at 40th video), NICE. Thanks a lot Cherno for these videos.
I remember my first game programming book for android had something along the lines of a MeshTexture and TexturedMesh classes, for a Tetris game. At that point I gave up.
Then a simple Arduboy (8 bit game system for arduino) game in C had arrays of positions, and arrays of other properties. The "entity" was an index into the array. Funny how today large systems revert to the simple approach.
Awesome! Yan, you have probably read it already, but Jason Gregory - Game Engine Architecture, Third Edition. Awesome book.
So there 3th edition nice to know.
I got the Richard Fabian Data oriented book.
@@SuperG316 The third edition is relatively new, it's from 2018. Yan's video reminded me about parts of it. Jason Gregory talks about ECS, Job systems. It's just awesome.
EnTT is really damn good. Like, really! The developer behind it is also really friendly and hands-on helpful. I actually built a simple entity compinent system back in 2011. It was a good experience, but yeah, EnTT is amazing. (PS since its EnTTwith capital T’s I guess its pronounced N.T.T. or “entity” if you say it fast, but yeah, I see why you would call it “ent” in the video, to disambiguate). I personally also wrote my own wrapper to abstract it the way I needed. I think tha.t’s a good approach. EnTT gives you a ton of tools, but they may not fit oerfectly into the rest of your engine without some gkue that needs to be wrapped. So very much agree with that choice.
I’m not an engine developer, though, just a hobbyist with lots of non-game experience who has spent a bunch of time with EnTT. So my opinions may not be completely relevant :)
As an aside, I wish the ECS terminology were different. Component and System are too ambiguous (system isn’t too bad, but component clashes with too many other things in software). Personally I’d like “trait” instead of component: its the traits you assign to your entity that define its behaviour. And.. behaviour instead of system: its the logic to implement a behaviour, based on the traits your entity has. Then I can use “system” for broader things like “rendering system”. But yeah, unfortunately the terminology is already established.
Suggestions for resources for writing an ecs?
I actually wached whole lot of videos and conferences about ECS, but yours was the best explanation or most satisfying i guess. maybe i forgot previous ones and i just remembered now. BUT in any case, i belive you have too much knowledge about computer an programming and i would love to hear from you these side topics as main topic not like "only mentioned in a larger project". for example if you were to do a playlist on ECS im sure it would be more popular then the game engine series. because other then games, none of the apps are using this kind of architectures even if they need it as much as games do.
The rest of the world: Cache
Australians: 𝓒𝓪𝓬𝓱𝓮
in my eyes. the fact ECS are used in games proves the silliness of OOP
This man is exactly what i need inctead of my uni
When you said this video is too long I was like noooo it is about to end :(
Cherno yesss im literally so excited
Definitely watching this... just not at 4am in the morning XD
Thanks for your great video 🙏
Such a good intro to ECS.
That jacket do be looking cozy tho
Hey thank you for telling us about the golden hours I'm using it in every scene
"There are AAA games that have done this" - The guy who worked at EA
great video. Could you make a follow up video on combining the ECS with spatial culling, for example if you only had the CPU to calculate the nearest 1% of the entities in your registry?
Sir, I just want to say:
T H A N K Y O U
OH my gosh, I have been working on my 2D Unicode-text game engine for about a month now, and I have been building my entity component system exactly as you describe it at 26:33 - I have been HATING it, and desperately searching for a better alternative. It's such a gross solution, I agree!
You have enlightened me. This is such a better system, I'm so frickin excited to implement it. This is by far the best video I've watched talking about how to build the architecture of an application - especially a game engine. You are doing god's work here, I'm so thankful of your content, especially this video, you really cover the things most people don't. Your teaching style of pretending to work through the problem logically with the viewer is absolutely the best way to teach, but nobody does it because it's time consuming & you have to know exactly why we have come to the solution we use - so thank YOU for doing it, regardless of that it's so time consuming and so much more difficult - it instills your knowledge in your pupils, AS WELL AS teaching them how you come to that conclusion. Thank you, thank you, thank you - holy fuck, this channel is a godsend.
I'm sixteen. I'm a self-taught programmer who's been teaching himself C++ for 3 years now. I have no friends who like programming, and I have no teachers who program. Nobody I know in my small town in Kansas understands what I spend all of my free time doing. I'm luckily really good at tests, and I'm going into an early-entry-to-college program at Fort Hays State University called the KAMS program next year at the age of 17, which will allow me to take college classes on-campus for dual-credit highschool/university credits. This means that I'll be graduating high school with 2 of my 4 years towards my bachelor's degree in computer sciences completed. I have been working my ass off to make sure I can build applications as a job in the future - and I'm relying entirely on online references such as yourself. This type of content at this quality is sorely missing on the internet - trust me, I've been self-teaching for 3 years completely alone. You are what people like me rely upon to make it to where we want to be, and to learn to do what we want to learn to do.. So thank you again.
Well this is crazy. I also grew up in a small town in Kansas and could have gone into KAMS. At the time though I was pretty sure I wanted to go into Economics even though I had been playing around in Unity and with c++ for most of high school. Ultimately I ended not going but still graduated with my Bachelor's in economics and marketing in 2 years. But I hated it and the work. I moved to NYC, went back to school for game dev and I'm much happier programming than I ever would have been working in business. But until I moved to NYC I just assumed all schools lack any form of coding opportunities and that everyone had to be a self teacher. One of the first jobs I got though in NYC was teaching an after school class for programming. It was for kids K-4th. I realized very soon a lot of schools have started teaching programming in some form or another. Though I do think being able to learn new technologies by yourself is frankly more important than anything you'll get at University. Not to say college doesn't have something to offer, but for programming most of the things you need to know you're going to have to learn on your own, it's just the nature of the business. Congrats on getting into KAMS and keep with it.
Dude. Same. But I don't live in kansas and I am not doing college work (running start). That's really weird. Same amount of time for c++ and everything
YO I am making terminal Flappy bird. So i am here to make literally the same thing. Although I am willing to do ecs separate of graghics
YES! Thank you
If I remember correctly EnTT uses dense hashmaps for component storage and then access those components through entityID. This may sound like a good idea, but in reality is very slow (compared to simple arrays). Also remember that components are unsorted! So if you want to loop through entities that use 2-3 types of components you are basically jumping around in memory. I work on a project that simulates hundreds of thousands of agents and I can tell you there is a better way. When you design an entity in general, you structure it as a unique type with some set of components. It is better to make a container that stores specific type of components and link it to the storage unit of that type in your scene, then each entity "type" can use something like std::tuple to store that containers. You can access components with 2 indices (entity typeID and entityID), so you don't need to have any UUID within the entity, or components. Only downside to this solution is that you cannot design your entities in the editor, because you need to define them statically. Also, if you use this design, you don't have to "submit" data to the renderer, because you can sort component containers by material type as they are created.
Unity DOTS and Unreal Mass do this and call them Archetypes.
amazing video
I see a new video - it's about ECS, one of my favourite topics! - it's 43 minutes long!!! This is my lucky day😂
Agree. Also was waiting for it :)
Awesome vid. How does this combine with the scene graph? Should it combine with the scene graph? I’ve seen some stuff online that using the ECS for the scene graph is an abuse of the ECS and they should be completely separate, but the entities have transform components and meshes, so obviously we should be at least treating that data in two different ways instead of allocating it twice (once for the ECS, then the scene graph). Thanks for any help!
Good vid ended up subbing (hazel might be my new game engine
Wow at the moment when I literally was implementing an ecs system into my engine and this came out , I have always felt the shortcomings in my design and here hope cherno will show the better version of it
GamesWithGabe did very good video of this subject.
Was totally expecting a pure ecs from scratch. Disappointing a little by the use of a library instead. Really liked the explanations. I have been a little behind on your game engine series and was kinda tinkering with a project I've had for a while. Turns out this is the 2nd time now something I was doing was on the exact day you put out a video so that's pretty cool. Keep making great videos, I enjoy learning!
holy fucking shit my brain just blew up. i've been writing dodgy code like this for years, thinking it was the only way to implement a high entity-type count and thinking i was dumb because i couldnt keep up. this is HUGE (because of how small it is)
dont also forget about branch mispredictions that are caused by the if statements
I have question, in Flutter i have to use system to move widgets around, doing it right it is efficient. So a ECS ontop Flutter would be something less efficient since high level programming but all the same efficient. My idea is Nintendo Switch they deliberately built on processor with graphics processor to make it very hard to make simulator since it is so closely built to processor and cache. ALso in flutter there is delay problems one can avoid by putting in 'invisible' movements before 'real on screen' movements, to mitigate lag. I understand the problem of making the 'sprite' dynamics more structural.
YEEEEEEEEEEEEEEEEEEEEEES I've been waiting for this!
Thanks for sharing the knowledge. I have a question regarding ecs especially how to use it to construct the inventory. I wanted to make it like unity3d scene graph but with ecs it’s impossible to do that. Maybe you have suggestions or you can show us how to construct such a complex hierarchies with ecs? Thanks in advanced!
Thanks cherno! i get it!
lmao i know it's not meant this way but it reads like you're annoyed XD.
I'd never heard of entt before, but as it happens the new LEGO:StarWars game used an in house engine called nTT i think.
Maybe you should take a look at Milton, a drawing program with an infinite canvas. Makes visualizing stuff like this a lot easier since there is always enough space. Even better, it's free.
Milton is excellent for this kind of stuff, I use it daily.
good video)
half life's class based hierarchy looking dummy thick doe o-o
Could you please talk about the interface between ECS and our Renderer?
I'm having trouble figuring out how the Renderer will extract data from the game simulation.
The components in your entities are pointers, so they can point to a cache-friendly store of each type of component, and you get the benefit of both systems. Contiguousness for quick iteration rendering etc, and discrete entity access in scripts etc if needed. What am I missing?
Encapsulation, probably. The idea of classes whose data is accessible through some alternative data structure is anathema to OOP purists. What you're suggesting is hacking the OOP encapsulation to have your cake and eat it too. An ECS is simply a more formalised way of doing this which ostensibly will have solutions to problems that you'll run into along the way. Like all programming paradigms, nothing is automatically the right way, if you see light at the end of the tunnel then head for it, and ignore the people telling you it's probably an oncoming train. They aren't necessarily working on more information than you have, they just have different priorities.
He mention other option Entity as ID. Get rid of pointers. Wich give ease of moving and sorting component array and avoid gaps.
17:00 “This is already a problem. We have four entities. Can you imagine making a AAA game?” This is how I was introduced to modern engines via (old) unreal and unreal script’s class-based architecture. _Any_ tweak to behavior of something (or *gasp* combining already existing behaviors from two different things) required wholesale refactoring. At least I learned that flavor of pain _before_ discovering unity’s component-based architecture; otherwise I would have never had the patience to mod at all. Yan’s point of thinking this high-level pattern through (to how would it be used to make new features) should be highlighted with a neon sign: the flexibility of interchangeable behavior (modular) is the point of an ECS, and it has the added bonus of being made up of smaller (easier to maintain) atomic components. Atomic and modular is the way to go! [EDIT] The data-driven design explanation is the best!
A question for anyone reading this :)
I have implemented my own ECS (in Unity) in pretty much the exact way subscribed in this video. But there's one part I haven't figured out the perfect solution for, which is: How do Entities interact with each other? As an example: Let's say we are making an FPS and an Entity gets shot (or actually his collider collides with a bullet), then this effects many components of this Entity. His health (HealthComponent) will change, an "injured animation" should activate (AnimationComponent), his AI should respond, maybe he screams, etc, etc, etc. Thus, how do we effect all these components when an Entity gets shot?
did you find a solution?
You could manipulate each of those components individually from the code that handles the bullet. However, if you want a more extensible and flexible solution, add a component "TakeDamage" to the entity, and have the systems that manage those other components react to the new component.
Guuuuuuud API
You are definitely the best coding channel on youtube!! Love your awesome videos!!!
how can u store multiple classes that inherit from a base class ? for exemple if u have a component class (base class ) and then u make Physics Componentn , audioComponent etc... how do u store them ? i mean the technical implementation .. because if u store them in a list of component ( List ) you cant access all their fields and methods ( if a PhysicsComponent "A" in that list then its considered as a simple Component not a PhysicsComponent .
could you also implement gameplay in ECS? for example, a sword attack would be represented by a component?
No.
Sword is an entity
Attack values on the sword are the components
in short-you can implement gameplay. You would make it in a different way than you are used to. The swords data would be contained in components. Maybe there is a "damage" component, or a transform component, doesn't mater. What's important is that the sword itself is only represented by several small modules of data, so that the project remains modular and easy to work with. The sword doesn't have any behaviour, the behaviour can be handled by outside systems. The sword is just a number, marking which components it uses.
The way that you normally program is to organise things into separate "objects", this makes sense as it's how our real world works. However, for a computer, it is much easier if the data is packed close together and in logistical positions. Having game Objects filled with scattered behaviour and data not only costs performance but can also be a real head-ache to work with.
The problem with ECS is that unique behaviour is not very easy to implement. It's kind of like making the rules for your game, and the data of your game separate in two different arrays.
Towards the end you mention that a system will iterate through all component of a certain type instead of through entities. I understand how that is better for performance, but how do the systems check if this component is from an entity with another certain component? In your example you had 5 being the entity id that links components together but how do you lookup other components that are also from entity 5? It seems like a lot of lookup through an array, which I don't know how it affects performance.
I think the idea is You need to optimize the critical path so stuf that done frequently most of the time. Like the render system go through all the transforms and meshes. Vs tha gamer select a radar dot and the event created look up some data from that entity to be displayed in part of the UI. So seek of a component of specific entity is rare thing. While every system of the ECS is updated very frequently.
Ideally you implement your ECS to support Systems that process two or more component types, since that's usually what you want here. ie. a Model component needs a Position component to draw in your Render System.
The bulk of your critical processing in an ECS should not go from entity->component. That should be a rare case. Instead, most critical systems will fetch the container containing one or more component types directly, completely bypassing the entity. They don't bother with entities at all. Like:
for (Motion& m: scene.query())
// do something with 'm'
This doesn't involve accessing entities whatsoever, even in the underlying implementation of 'query'. The reason to use something like an entity ID bitset which identifies the types of components an entity has, and allow some way to fetch specific components from a particular entity, is for rare cases... not your critical paths. For example, your engine's editor GUI might want to list all component types available for a given entity in some tree/list GUI control. In that case, you are going from entity to components but that's not a particularly performance-critical case compared to something like a rendering system or physics system which has to loop through components every single frame.
Something like a physics system shouldn't care what entity to which a motion component belongs. It only wants the motion components to transform. So those types of systems don't even bother to access the entity at all. They bypass all that and go straight to the container of motion components to iterate through very efficiently.
@@darkengine5931 Incredible answer. You really made me understand something and I appreciate it.
@@RamHomier Cheers! This is also why we don't store components directly inside entities. It would be more convenient to implement that way, but then we'd have to do a linear-time search for a component (or at least a layer of indirection) starting with an entity in all of our critical loops. The reason we store components outside the entity in their own individual containers is to be able to bypass that entity and just get the entire container of components and loop through them directly.
31:45, unless I am wrong, isn't the cache line start location calculated by removing N least significant bits from the address that you asked for instead of starting on a said address?
You're correct, I think they were just simplifying it to get the idea without going into detail.
One question that has lead me back to inheritance:
How do you efficiently deal with an entity's deactivation or destruction, especially if that entity has a lot of components?
Currently in my engine I'm using multiple inheritance where each parent is like its own component; the scene stores objects contiguously by type in a tuple of vectors, and the game loop on each operation only iterates in the vectors which content inherits from the needed "component" parent (which vectors is determined at compile time). This way, destroying an object is as simple as erasing it from the vector containing its type.
But with an entity-component system, how do I define an entity's destruction? Do I have to iterate all component types vectors looking for the id and removing the components??
Usually they are POD-like structures, so you can just overwrite their components. And since it's all allocated on the same array, you can just free the memory for the whole array.
I already knew the pattern, but a question always comes to mind. Suppose I am actually cache-efficiently iterating over my Poison components, and a certain point a Poison comp. asks if Health comp. exists on entity 0xdeadbeef to update it. I should fetch and update the other component, located in another array. Would this "jump around memory" disrupt cache efficiency?
Or maybe what I want is a "join iterator"? Can the cache optimize a simultaneous iteration on two arrays?
Your cache has more than one line (line being a section of memory it grabs from). I'm pretty sure it can handle it.
Does ECS try to be a "CPU vertex buffer"? I see a similarity between your ECS data layout explanation, and your opengl vertex buffer layout explanation.
I implemented an Entity Component System in the game I am developing on my UA-cam channel. The system in my design has all the logic that operates on all the components registered with the system. So I have an AnimationComponentSystem class that operates on AnimationComponent classes registered with animation component system class. So this is very similar to what you have described in your video.
ECS ( entities, components, systems)
entities is just IDS. with comonents behavoirs.
systems receive entities and make something with entities components
will be like:
VisualSystem (entities, args) {
for each entity in entities {
const position = entity.components.find(PositionComponent)
const visual = entity.components.find(VisualComponent)
if(position AND visual)
SOME_RENDERER.draw(position.x, position.y, visual.width, visual.height, visual.color)
}
}
SoundSystem (entities, args) {
for each entity in entities {
const sound = entity.components.find(SoundComponent)
if(position AND visual)
SOME_SOUNDER.play(sound.source, sound.volume)
}
}
PositionComponent {
x: number
y: number
}
VisualComponent {
color: stirng;
width: number
height: number
}
SoundComponent {
source: string
volume: number
}
Entity {
id: string
components: []
}
systems = [VisualSystem, SoundSystem]
bob = new Entity()
bob.components.push(new PositionComponent(100,100))
bob.components.push(new VisualComponent('#f0000', 128, 128)
tim= new Entity()
tim.components.push(new PositionComponent(200, 100))
tim.components.push(new VisualComponent('#00000f', 128, 128)
bird = new Entity()
bird.components.push(new PositionComponent(10, 10))
bird.components.push(new VisualComponent('#ffff00', 64, 64)
bird.components.push(new SoundComponent('./assets/sounds/bird.mp3', 100)
gameloop () => {
delta = SOME LOOP TICK
for each system in systems {
system(entities, { delta } )
}
}
Will our Renderer become a system of our Entity Component System? The Renderer feels like it could be "the last system that runs in each frame"......... but I'm not sure that it makes sense to add the Renderer as part of the game state simulation.
Me: Mischief is a really nice program to dra.....
Cherno: Haha Photoshop go brrrr!!!!
Mischief is dead :(
all im saying is this: mspaint
Can try Leonardo: the painting app
@@eeromutka186 Milton is pretty much the same but newer, saw it on Handmade Hero
Krita anyone?
Cherno, just a question: in my ECS, systems are not iterating over entities, because it would mean iterating over all of them per system. Instead, every component has an update function, and I pass the systems in as argument (all systems are wrapped in a class), and then each component puts itself into a list in the systems it needs to be in. So for example, the renderer component puts itself in the RenderingEngine (systems are called engines), the PhysicalObject puts itself in the PhysicsEngine. And so on. This means iterating through them only once, and working with a local list later, as after the components are updated, every system is updated. I think it performs better. What's your opinion?
Well most people think components should be raw data, so them having an update function is counter to that. Also, you're calling functions over and over and over while passing in a system, this is trashing your CPU cache each time. Instead, a lot more cache friendly way is to loop over your unordered array of components held by the system in a single function.
Something like RenderSystem, which has a link to or holds a list of components that are renderable, you call RenderSystem::Update(const float ts) and Update loops over all the components held in the list. As I said, your method is blowing up your cache each time you call the components update function.
Are you deep copying the objects into each system ("engine" in your case)? Or just referencing/pointing to them? It makes perfect sense to me to avoid iterating over entities in systems and just access a sequence of components directly without the branching and memory overhead, but I can't quite picture how you are injecting your components into their relevant systems in a way that doesn't involve additional layers of indirection (and sporadic memory access patterns)... unless your codebase is such that you never have more than one system accessing a particular type of component.
@@darkengine5931 I use pointers to get the objects there. Actually everthing needs to be a pointer because the entity object can be inherited from to specialise some cases. This system works so far with no performance problems
@@andraskmeczo575 I see. If there are no performance issues in your case with that solution, I think it's all fine and good. Typically I would think that would translate to a lot more cache misses though, as the stride to get from one component through a pointer to the next might be quite variable and range from very small to very large. But you usually need to be processing something like tens to hundreds of thousands to millions of components per frame for that sort of thing to become a non-negligible overhead. In many cases though, I think the performance of ECS is overstated and its flexibility understated. I think the flexibility is going to benefit a much larger range of projects than the performance of cache-friendly contiguous component access.
@@darkengine5931 I see where the struggle can come from. As per se I allocate the entities and components individually, but I'm working on a factory-like system which allocates a lot of the same type, and gives a pointer to one when an allocation occours. This ensures that the entities and components are close together, and while the inherited types need separate allocators (for types that are used regularly, like TerrainNode, GUI elements are used like 5x at most so it doesn't need optimisation. And my system ensures that the inherited types are dealt with together, not mixed up with the regular entities, so in theory, there should be no cache misses, the only problem might be that there should be two caches, one for the entities' block of memory, and the other for the components'.
Yha I have written a Entities system for Unreal and I do have problem that getting and writing to component arrays is kinda slow. Really sucks lol.
I wanna make my ram bus go vroom vroom
Composition over inheritance
Whats the drawing Software you are using ?
It took me a lot of time to realize that continuous memory problem and even after that I implemented it using unordered_map which probably saved time if an entity doesn't have that component.
This video showed me a big flaw in my ECS, THANKS! for the video. And I'm probably going to switch to enTT instead of writing it.
I have actually a couple questions.
In the implementation of the ECS i have seen it uses a Arrays of Components and these arrays sized based on the pool.
So lets say you have 50 components and your "1 Million" Entities.
That would mean you would have 50 Object Arrays (and some bit vectors) that have the size of 1 Million each.
And here is the trouble that I have this: Wouldn't you trade in this case "CPU Speed" for ram usage?
Yes null objects are not as bad since they only have the pointer reference but this kind of stuff adds up.
My question is: Am i overthinking things like that? Or is this not as bad as i am thinking.
Another question. How would you deal with a chunk like system where you could say the same thing as for a an ECS System (give me only the entities i need)
This is something that kinda melts my head.
This is for me why i am planning to go with the "Worst" Inheritance system, because I am getting what i am doing, but i know that a ECS System would be better, but the issue is that there is sadly not that much resource out there how to use a ECS System with a grid System where you are not forced to iterate over all entities in the world to find what you actually want.
The "Whole I just want my components" kind of thinking again.
I am sorry for bothering you with these kind of questions. This is just melting my head.
Thanks for reading. Sorry if I offended you in any way.
Having 1M entities does not mean all of them have 50 components. So the components arrays are on average much smaller than 1M. And as long as each component only contains what is necessary to represent it (i.e. minimize its memory footprint), it should still be fine in general.
Of course if you do have 1M entities and each of them needs all 50 components, maybe you are building your scene wrong. Possibly similar entities could be grouped into one entity, vastly reducing their combined footprint.
@@Wosser1sProductions the arrays still would have to account for the largest Id though.
If entity 900000 has a position component the array still would need to be at least 900k big even if it only contains 5 elements. All others contain nullpointers.
That was my point of why I am confused with ecs.
I even addressed that. And unless you use a dictionary (which has its own performance problems at that scale) you are forced to have some kind of entityid to component array mapper that has to be the exact same size as the biggest entityid.
(Writing on phone)
having array per component is bad coz as you said for most of components this array will be just empty with some filled slots somewhere in the middle for example. and since for performance reasons it's better to not have array of pointers but have an array of actual structs - you will waste a lot of memory basically allocating unused components.
you can optimize it by introducing "chunks" and "archetypes" (e.g. Unity calls them like that). archetype is a set of components (e.g. archetype1 describes all entities with transform, mesh, rigidbody, and archetype2 describes all entities with transform, mesh, rigidbody, animation). then you have chunks which contain data for the archetype (e.g. for archetype1 it will be 3 fixed arrays of N elements - 1 array of transforms, 1 array of meshes, 1 array of rigidbodies). N is the size of the chunk (and you can play with it to find the best for performance), the bigger chunk size - the better for CPU cache, but more memory wasted bringing you closer to initial "array for components of one type" solution. too small chunk size - better memory management but worse CPU cache friendliness. for example you have chunk size of 256, if you have 200 entities of one given archetype - 1 chunk is created and a bit of space wasted. if you create 200 more - new chunk will be allocated after first one is filled. and ofc you need to cleanup chunks which become unused, so more bookkeeping required than plain 1 array of gigantic size per component type.
for inspiration check out Unity's implementation (or at least the idea) of ECS.
@@user-jn9pg3lx7g you still need a translator/mapper from entityid to component index and that mapper (most likely an array for performance reasons) has to as big as the biggest entityid. There is no way around it. And why I said that it would be a null pointer is because I use a language that does not support pointers by default.
Could you show me a good implementation of ecs that does not have this exact issue of "wasting memory based on biggest entityId" and does not use dictionaries?
Also how would you implement a chunkgrid into the game with ecs. And I am not talking about "memory chunks" I am talking about chunks like minecrafts grid system.
Because iterating over every single entity to find something nearby is a bit laggy? Or how would you implement a grid optimization for nonstatic entities?
Hey. I also write a game engine. And I have big problems with understanding how the scene node graph
system works. I already read "Game Coding
Complete, Fourth Edition by Mike McShaffry" book several times and I just can’t figure it out. Please make a video on this topic.
Unity itself using this method in their engine.
lmao that American accent was pretty good
is this something that's commonly done in most games or something new that became the practice only very recently?
It's common and this method is quite old
@@marekpiirikivi9877 why don't big game engines like unreal, cry engine have it? i know unity added ecs beta very recently
@@Alex-fr2td Honestly I do not know. :) When developing software there are multiple factors to consider when deciding what goes and what is left out.
I would predict that ECS is not the friendliest way of introducing game development to its users. And unless you have a lot of going on in the game and it has to run on weaker hardware, it did not matter much.
@@Alex-fr2td I would like to add that every year it is expected games to offer more and push bar higher... and we are starting to hit the limit of what computers are capable of doing and have to find a way to optimize.
Say I wanted an entity that contained a mesh. Then I wanted another mesh attached to this mesh, under a transform. While I could make 2 entities and put them in a transform hierarchy, wouldn’t it be easier to a link to the parent component in each component and get rid of the idea of an entity altogether? Surely my entities would just be a transform hierarchical structure of components, and the “root” would be a component with no parent? Then during a rendering system update (for example), each mesh would be iterated over using the mesh components list in the scene, then a walk up the tree would be needed to in order to get the full transform of this mesh. The extra work caused by this way of doing it (calculating transforms of the current transform within the context of the “entity” tree) could be negated by “caching” this data in the system as you go? Or have I misunderstood something?
also, forgot to say in the post, bloody great videos!
@@bocdev4889 You might be a bit tunnel-visioned on the mesh motion hierarchy and its evaluation unless that's all you need at which point an ECS would be overkill. Typically entities in an ECS store a whole lot more than meshes and/or parent/child links and transformation matrices. The point of associating them to entities is because entities might be associated with all sorts of different component types but we want one single entity unit at the end of the day in particular contexts.
Even in the context of motion hierarchies, you might have entities other than meshes that want to be parented. For example, a light source that doesn't store a mesh might want to be parented to a lamp mesh by the level designer, or a particle system, or a light parented to a camera, etc. etc.
By having entities separated from components, and different component types from each other, you can still organize multiple component types by associating them to specific entities, but without the entities having to store some indivisible super bundle of data that gets inefficiently and indivisibly loaded into the CPU cache.
Weirdly enough I don't see any comments here about unity using that inefficient ecs system by being one of the most popular game engines. (Ye, Ik about DOTS but it's still not production-ready system so don't remind me of that)
32:43 DAYTA 🤓
19:46 #rageQuit :D
please dive in how entt deal with those hard works for us
What is your opinion on Jonathan Blow's comments on ECS? I myself think that if you are making an indy game, then you don't need a separate engine and can just go from scratch, but AAA dev teams need consistency, claritx and organisation, which means that ECS becomes necessary IMO, or at least sth like it. Its like how rigorous data base designs are essential in this day and age.
My opinion is it depends on the scale and complexity of the game, and also what your goals of using an ECS is. Eg, if you are writing a AAA game then you probably need a data oriented design and ECS cab help with that. If you’re writing a game with many entities with complex behavior, then ECS can help with managing this as you can compose the behaviors. Similarly, if you expose this to your designers/content creators, then an ECS can allow you to configure the entities in a flexible way.
ECS isn’t the only way to do those things, but it seems like a good fit. If you don’t need those things, maybe because your game is smaller, simpler or you are handcoding/scripting all the behaviours (versus giving your designers a way to configure it in a tool), then ECS is totally overkill and may just add complexity. Jonathan Blow is making comparitively simple games with fewer and simpler entities, at least, Braid and The Witness, and he’s coding everything himself so he can create twekaed entities as necessary, so I can see why it wouldn’t be a great fit for him.
If you want to write a game, write a game. If you want to write a game engine, still write a game.
Most successful game engines were originally written for some specific game (Unreal Engine for Unreal, Id tech X for Doom Y/Quake Z, ...). Even Unity was done for some game that they never released.
It depend on the need for the type of game and it complexity. For AAA games the bulk of the work force is making high fidelity content , like corridor shooter. On the other end a indie RTS game with many different types of entity need for more advance ECS is higher demand. Even if the content is 2D tile based. AAA titles are render load heavy so a few entity with few types is often not the problem . It could be up to 12 to 200 entity CPU load and cache misses is not the problem but render task is main load. For RTS 2D game the render task is light but the CPU needs to handle 2500 units with bullits missiles and composed units / entities is might be 10000 entities and a AI load for 2500 unit + 250 guided and 1000 unguided misiles.
How to implement the connection between logic and entity and their components? That what is always a hard for me.
Especially if we want to do the logic in the data oriented design. Maybe you could share some
thoughts about that topic?
If I understand correctly, a crude answer but perhaps helpful if you are stuck in an object-oriented mindset is to start with an object:
class Transform
{
public:
// public interface/logic
...
private:
// data
Matrix4 local;
Matrix4 world;
};
And we move the data stuff into a component:
struct TransformComponent
{
Matrix4 local;
Matrix4 world;
};
And we move the logic into one or more systems (ex: a physics system). Then you associate the instances of the transform component with whatever applicable entities.
That's a sort of roundabout answer to your question if I understood it correctly, but there's not much more to it than that. If OOP is comfortable to you, then you just got to get over the discomfort of moving the logic into a system in a separate place from the data. The main thing you lose is encapsulation including information hiding. In exchange, you get tremendous amounts of decoupling, flexibility, and more breathing room for optimizations. If I misunderstood your question somehow, please let me know and I'll elaborate, but I tend to get the sense that a lot of the mental block that people tend to have with ECS is that they're so stuck in an object-oriented way of thinking about design when the ECS is more of a procedural way of doing things (albeit a very flexible and organized form of procedural programming).
😎😎
18:00 In my opinion at this point it was worth to mention about how inheritance is very strong relationship between two classes.
If everything is an entity how about to identify it as starship/player, Enemy, laser , background, explosion?
You typically don't. ECS encourages a design mindset closer to duck typing (minus dynamic typing unless you are implementing it in a dynamically-typed language). If something has legs, it can walk. It doesn't matter if it's a robot or a spider or a human. What matters is that it has legs. If you absolutely need to distinguish the "type" of something from another, you do it by attaching a component or storing something in a component that identifies it as being like an enemy vs. a friendly NPC, e.g.
If you've ever done generic programming this should be somewhat familiar. A generic sort function doesn't care if you pass it one type of container or another. All it cares is that it's a random-access sequence of any type and has the required stuff needed to sort it. It's not important whether it's a vector or a deque or arraylist or even if it implements any explicit interface in the context of this generic sorting function. All that's important is was it has, not what it is.
objects of arrays vs arrays of objects