You may have just single-handedly saved my game here. Thanks for this! I was doing something similar with a huge number of baddies in my project in Unity and after the whole debacle this past week I wasn't really sure which engines would have the tools needed for me to re-implement it. This is at least 1/3 of what I need to re-create from scratch so I'm feeling a lot better than I was even just a few hours ago. Much appreciated!
Thank you James!!! This is also done in Blueprints (visual scripting) which can be up to 10x slower than code in C++/C##, not to even mention ... this is single threaded blueprints, where you can then go ahead and multithread in C++. LOTS of AMAZING tools that do all that work behind the scenes, like MASS AI, are here now with UE5. Your comment I posted on Twitter, it made my night, Thank you very much! And GodSpeed in the Project!
Your idea is definitely a creative approach! The flipbook-like mechanism you're describing is akin to an old-school sprite animation technique, but applied to 3D meshes in a hierarchical instanced static mesh / instanced static mesh setup (lets assume you used HISMs). Keep in mind that every time you change the mesh, the engine would need to re-apply instancing optimizations, which might lead to performance degradation. Here are a few points to consider: Mesh swapping: If each of your meshes has the same vertex count and layout, you could swap the mesh data (C++) each frame to animate it, instead of replacing the entire mesh. This would preserve the instancing benefits of HISMs. Memory management: Static meshes, especially if they are high detail, can add up in memory usage. Be sure you are aware of your memory budget. Material considerations: If your materials are complex, keep in mind that HISMs batch draw calls based on the materials. If each mesh in your flipbook uses a different material, you may lose some of the performance gains from using a HISM. Rigidity: This method doesn't lend itself well to complex or varied animations. If you need characters to do anything other than run (like idle, attack, etc.), you'll need a separate flipbook for each animation. The Animation Budgeting System or crowd simulation plugins may still be a better solution for animating large numbers of characters. Remember to profile your game regularly to ensure you're meeting your performance targets. Unreal's profiling tools can give you detailed information on where your game is spending time each frame, so you can optimize effectively. Complete different approach "Vertex Animation": Vertex animation is a method where you animate a model by manipulating its vertices directly, rather than using a traditional skeletal system. This method can be beneficial for performance because it transfers some of the computation to the GPU, which can handle these operations efficiently. Creating a vertex animation in Unreal Engine involves a multi-step process. Here is a simplified outline: Create the Animation: This is typically done in a 3D modelling tool like Blender or Maya. You will create your animation here, then export each frame as a separate model. Generate a Texture: This step involves taking each frame of your animation, and "baking" the vertex positions into a texture. Each pixel's RGB values will correspond to the XYZ position of a vertex. This process can be quite complicated and may require a custom script in your 3D modelling software. Import into Unreal Engine: You'll then import the model and the texture into Unreal Engine. Create a Material: The material is what actually performs the vertex animation. You'll use the texture you created to offset the vertices of the model. You can do this in the material editor using the World Position Offset input. Apply the Material: Finally, apply the material to your model in the UE editor. Note that this process can be quite complex and requires some understanding of both 3D modelling and shader programming.
If you want variety: In Unreal Engine, you can use a Custom Data pin on an instance of a Hierarchical Instanced Static Mesh Component (HISM) to pass additional data to a material. This data can be used to control the behavior of the material, for instance to vary the animation state for different instances. Here is a broad overview of the steps involved: Create a Material that includes a Customized UVs node: This allows the material to take input from the HISM's custom data. In your case, you would likely use this input to determine the vertex offsets for your running animation. Write the Vertex Shader to animate the vertices: The shader will use the custom UV input to determine how to transform the vertices of the model. For a simple animation like waving or bobbing, this might be a straightforward offset or rotation. For a running animation, you would likely need a more complex set of transformations. The UV input can also be used to vary the animation for different instances, such as offsetting the phase of the running cycle so they don't all move in perfect sync. Set the Custom Data on the HISM instances: This data will be passed to the material and used to control the animation. You can set this data in C++ using the SetCustomDataValueOnInstance function, or in Blueprints using the Set Instance Custom Data node.
Great detail! A lot of your methods sound a little pre Nanite however, believe vertex animations are totally unsupported with Nanite, where this method is fully Nanite. I appreciate the research! Hism and ism both cull now, and with Nanite, ism is king!
To your point, to improve I may pool instances of the animation and Batch update swap them between positions to reduce that mesh change and also provide variety. WPO just got Nanite support so that’s almost vertex animations, but it comes at a cost!
@@aggressivemastery Great idea with the Batch update! I'll probably be working with Unreal Engine's MassEntity later this year. Would be interesting to know how MassEntity would perform in your scenario.
Thank you!! I am glad we have linked up, I havnt posted this to uTubes yet but Tim Sweeney liked and followed me for it, its time for the super hero games! twitter.com/GamedevMicah/status/1578892270677688320?s=20 I will continue to post here! Twitter is my main... (@gamedevmicah @roadlessthegame @aggressivemstry) I also post to reddit (diepepsi)
its best to keep a small number of ISMs, instead of making groups of them. So if you have a single ISM (or collection) ... I was doing 100,000-250,000 without any issues. Its just an ISM count being rendered. ISMs are the most performant way to render meshes in Unreal, so you are kind of asking what Unreal's limits are. And with Nanite, its pretty unlimited!
Is there a way to do this with the 'flip book meshes' chasing the player? For example, lets say the player is walking around in a hilly open field (with elevation changes due to the hills), and all around them hundred of zombies are coming their way. Is there a way to set the the nanite flip book meshes to 'move to player character' before converting to a pawn once theyre close enough? Or will this only work if the nanite flip book meshes are moving in a specific direction(move along xyz) (since as far as I know you cant script static meshes to 'move to' player)?
Sure!! Its all in the amount of work you put in. Here I convert the ISM instance to an ALSv4 AI Agent when near the player, which has static meshes attached to each bone. this way the skeletal mesh is rendered as static meshes, and nanite. You could fix the ISM in place, and move each instance around driven by external calculations (this is what MASS does in the City Sample project for distant AI)
@@aggressivemastery I'm not entirely sure I understand. How could I make it were the ISM moves towards the player, and then when it gets close enough to the player despawns and spawns an actual pawn/character in its place?
@@CommanderColson Hello Colson! Here the ISM is moving and the instances are staying still. If you changed that movement to instead of always being the same direction, be the direction you want it to be, that will start marching them in the right 'way' but not make them face that direction. You could rotate each instance to face the player, or rotate the whole ISM... It a VERY advanced optimization technique for specific crowd needs. So you would convert to ISM at the last possible moment. Maybe even having LOTS of AI near the player, to get the smooth transitions you are expecting. The other option is to move each instance within the ISM, by changing each instances transform information and batch updating the ISM each frame. You could offload that work to other cores as well (to calculate the transforms) then apply it on the game thread. I want to again stress, what I am talking about here is about the... most optimized MOST complex way to do things. So It wont be THAT easy to do or follow, I am sorry! :)
There is also a great way to do this with Niagara and Material Pixel Offsetting via World Position Offset on the shader, spoken about in this video you could use in addition to what I am doing, or in replacement :) ua-cam.com/video/CqXKSyAPWZY/v-deo.html
Not gonna lie, this is not beginner stuff at all. Well explained, but still, far beyond my current understanding. I know it's not the point but I thing you could add some music and sound effects. It doesn't show at high speeds, and the quick gameplay is impressive, but those flipbooks are a bit of.. strange (idle walking around especially) Not bad, but not really fluid either. Even though the game runs smoothly, it looks like the animation is running at lower FPS..
Agreed! lots can be done to improve it, such as more frames baked out, or using 3 different ISMs in the same crowd to give some variety to the footsteps, I also plan for simple ai (turn to look at, move to point) that convert to ALS closer to the player, which would drive ISM instances. But the better it looks, the hard it is to see what the tricks are :)
我是UE5的初学者，最近在研究nanite骨网体，刚看到你的视频，你的方法很有效，也是我最近从奥迪官方的例子中思考的突破，但是在控制骨头包装上最后控制骨骼工作不正常，所以我正在寻找更好的方法， 你的方法我就要在我的项目上测试了，不得不说，当我看到你的游戏终于运行起来的时候，流畅带来的兴奋，真的只有喜欢玩游戏的人才能体验到突破硬件限制的喜悦。
You may have just single-handedly saved my game here. Thanks for this! I was doing something similar with a huge number of baddies in my project in Unity and after the whole debacle this past week I wasn't really sure which engines would have the tools needed for me to re-implement it. This is at least 1/3 of what I need to re-create from scratch so I'm feeling a lot better than I was even just a few hours ago. Much appreciated!
Thank you James!!! This is also done in Blueprints (visual scripting) which can be up to 10x slower than code in C++/C##, not to even mention ... this is single threaded blueprints, where you can then go ahead and multithread in C++. LOTS of AMAZING tools that do all that work behind the scenes, like MASS AI, are here now with UE5.
Your comment I posted on Twitter, it made my night, Thank you very much! And GodSpeed in the Project!
I love videos and discoveries like this, literally mind blowing realizations watching
Thank you Jarry! I posted this to X when you said it, made my night! Cheers!
Excellent work. Very clever solutions to a number of things. Thanks for showing us.
Thanks for that 👍
I liked the dismember part, I'm trying to figure out how I'm going to do dismembering in my project I didn't choose a way yet XD
Wow this is cool! Congrats on the Follow, and good luck on the grant!
Love you Brotha! With Sweeney's help, I could easily port and release Roadless and the Super Hero Teardown games for Xbox and Sony :D
Very cool idea to animate that large crowd
Your idea is definitely a creative approach! The flipbook-like mechanism you're describing is akin to an old-school sprite animation technique, but applied to 3D meshes in a hierarchical instanced static mesh / instanced static mesh setup (lets assume you used HISMs).
Keep in mind that every time you change the mesh, the engine would need to re-apply instancing optimizations, which might lead to performance degradation.
Here are a few points to consider:
Mesh swapping: If each of your meshes has the same vertex count and layout, you could swap the mesh data (C++) each frame to animate it, instead of replacing the entire mesh. This would preserve the instancing benefits of HISMs.
Memory management: Static meshes, especially if they are high detail, can add up in memory usage. Be sure you are aware of your memory budget.
Material considerations: If your materials are complex, keep in mind that HISMs batch draw calls based on the materials. If each mesh in your flipbook uses a different material, you may lose some of the performance gains from using a HISM.
Rigidity: This method doesn't lend itself well to complex or varied animations. If you need characters to do anything other than run (like idle, attack, etc.), you'll need a separate flipbook for each animation.
The Animation Budgeting System or crowd simulation plugins may still be a better solution for animating large numbers of characters.
Remember to profile your game regularly to ensure you're meeting your performance targets. Unreal's profiling tools can give you detailed information on where your game is spending time each frame, so you can optimize effectively.
Complete different approach "Vertex Animation":
Vertex animation is a method where you animate a model by manipulating its vertices directly, rather than using a traditional skeletal system. This method can be beneficial for performance because it transfers some of the computation to the GPU, which can handle these operations efficiently.
Creating a vertex animation in Unreal Engine involves a multi-step process. Here is a simplified outline:
Create the Animation: This is typically done in a 3D modelling tool like Blender or Maya. You will create your animation here, then export each frame as a separate model.
Generate a Texture: This step involves taking each frame of your animation, and "baking" the vertex positions into a texture. Each pixel's RGB values will correspond to the XYZ position of a vertex. This process can be quite complicated and may require a custom script in your 3D modelling software.
Import into Unreal Engine: You'll then import the model and the texture into Unreal Engine.
Create a Material: The material is what actually performs the vertex animation. You'll use the texture you created to offset the vertices of the model. You can do this in the material editor using the World Position Offset input.
Apply the Material: Finally, apply the material to your model in the UE editor.
Note that this process can be quite complex and requires some understanding of both 3D modelling and shader programming.
If you want variety:
In Unreal Engine, you can use a Custom Data pin on an instance of a Hierarchical Instanced Static Mesh Component (HISM) to pass additional data to a material. This data can be used to control the behavior of the material, for instance to vary the animation state for different instances.
Here is a broad overview of the steps involved:
Create a Material that includes a Customized UVs node:
This allows the material to take input from the HISM's custom data.
In your case, you would likely use this input to determine the vertex offsets for your running animation.
Write the Vertex Shader to animate the vertices:
The shader will use the custom UV input to determine how to transform the vertices of the model.
For a simple animation like waving or bobbing, this might be a straightforward offset or rotation. For a running animation, you would likely need a more complex set of transformations.
The UV input can also be used to vary the animation for different instances, such as offsetting the phase of the running cycle so they don't all move in perfect sync.
Set the Custom Data on the HISM instances:
This data will be passed to the material and used to control the animation.
You can set this data in C++ using the SetCustomDataValueOnInstance function, or in Blueprints using the Set Instance Custom Data node.
Great detail! A lot of your methods sound a little pre Nanite however, believe vertex animations are totally unsupported with Nanite, where this method is fully Nanite. I appreciate the research! Hism and ism both cull now, and with Nanite, ism is king!
To your point, to improve I may pool instances of the animation and Batch update swap them between positions to reduce that mesh change and also provide variety. WPO just got Nanite support so that’s almost vertex animations, but it comes at a cost!
@@aggressivemastery To be honest I haven't worked with Nanite yet 😅
@@aggressivemastery Great idea with the Batch update!
I'll probably be working with Unreal Engine's MassEntity later this year. Would be interesting to know how MassEntity would perform in your scenario.
Awesome Tutorial. Thank You for sharing Micah. Would like to know more about your Super Hero Game too.
Thank you!! I am glad we have linked up, I havnt posted this to uTubes yet but Tim Sweeney liked and followed me for it, its time for the super hero games! twitter.com/GamedevMicah/status/1578892270677688320?s=20
I will continue to post here! Twitter is my main... (@gamedevmicah @roadlessthegame @aggressivemstry) I also post to reddit (diepepsi)
@@aggressivemastery We agree. Working on something with hope to garner some attention from my Game Dev Hero as well. Haha.
Bro wtf this is blowing my mind how smart it is!! Thanks for the share!
Thank you!!!
this is going to be super helpful info for me, thank you.
appreciate the tutorial. true to ur handle. well played
very impressive
Thanks for the tutorial! How many static mesh characters can you handle with this technique?
its best to keep a small number of ISMs, instead of making groups of them. So if you have a single ISM (or collection) ... I was doing 100,000-250,000 without any issues. Its just an ISM count being rendered. ISMs are the most performant way to render meshes in Unreal, so you are kind of asking what Unreal's limits are. And with Nanite, its pretty unlimited!
Thank you for the answer !
reminds me of the vertex animation manager plugin
Is there a way to do this with the 'flip book meshes' chasing the player? For example, lets say the player is walking around in a hilly open field (with elevation changes due to the hills), and all around them hundred of zombies are coming their way. Is there a way to set the the nanite flip book meshes to 'move to player character' before converting to a pawn once theyre close enough? Or will this only work if the nanite flip book meshes are moving in a specific direction(move along xyz) (since as far as I know you cant script static meshes to 'move to' player)?
Sure!! Its all in the amount of work you put in. Here I convert the ISM instance to an ALSv4 AI Agent when near the player, which has static meshes attached to each bone. this way the skeletal mesh is rendered as static meshes, and nanite. You could fix the ISM in place, and move each instance around driven by external calculations (this is what MASS does in the City Sample project for distant AI)
@@aggressivemastery I'm not entirely sure I understand. How could I make it were the ISM moves towards the player, and then when it gets close enough to the player despawns and spawns an actual pawn/character in its place?
@@CommanderColson Hello Colson! Here the ISM is moving and the instances are staying still. If you changed that movement to instead of always being the same direction, be the direction you want it to be, that will start marching them in the right 'way' but not make them face that direction. You could rotate each instance to face the player, or rotate the whole ISM... It a VERY advanced optimization technique for specific crowd needs. So you would convert to ISM at the last possible moment. Maybe even having LOTS of AI near the player, to get the smooth transitions you are expecting.
The other option is to move each instance within the ISM, by changing each instances transform information and batch updating the ISM each frame. You could offload that work to other cores as well (to calculate the transforms) then apply it on the game thread.
I want to again stress, what I am talking about here is about the... most optimized MOST complex way to do things. So It wont be THAT easy to do or follow, I am sorry! :)
There is also a great way to do this with Niagara and Material Pixel Offsetting via World Position Offset on the shader, spoken about in this video you could use in addition to what I am doing, or in replacement :) ua-cam.com/video/CqXKSyAPWZY/v-deo.html
Not gonna lie, this is not beginner stuff at all. Well explained, but still, far beyond my current understanding.
I know it's not the point but I thing you could add some music and sound effects.
It doesn't show at high speeds, and the quick gameplay is impressive, but those flipbooks are a bit of.. strange (idle walking around especially) Not bad, but not really fluid either. Even though the game runs smoothly, it looks like the animation is running at lower FPS..
Agreed! lots can be done to improve it, such as more frames baked out, or using 3 different ISMs in the same crowd to give some variety to the footsteps, I also plan for simple ai (turn to look at, move to point) that convert to ALS closer to the player, which would drive ISM instances. But the better it looks, the hard it is to see what the tricks are :)