I think a common way used by many MMORPG games, is to simulate the world on the server (along with its physics), not only just validating actions. That way the client is primarily a screen to view what is being broadcasted to it, and an "action requester" to request actions to be taken by the entities you have permission to on the server. This way, you only need a bare minimum of client side predictions to predict wether or not your requested actions are granted and what you should do. Everything else is parsing and presenting data recieved from the server.
yeah, I can see the delay to request from the server and wait for it to get back working for mmorpgs (or turn based games, etc). Eve also does literal time slowdown don't they? Rocket League, which I mention in the video, does a lot of client side prediction but the server simulation is the source of truth, so if there's a discrepancy the server wins and client updates to reflect that, similar to what you're describing. There are definitely a wide range of games with a wide range of solutions. Not having to pay for servers is something that works well into a workshop IMO, but at the end of the day if the best solutions are "run a server" then I'll be running a server for sure.
It's still super hard to mask network latency if it isn't already included in the whole game design, for instance via animations which mask the latency. Rollback based netcode can be quite tough, but once properly implemented, was a godsend for fighting games in particular. Personally I've measured the latencies to some friends, and packets from Germany to Australia can take 500 ms of latency on average (max was a staggering 1300ms), so even with that such games would be tough to play together
Minecraft spawns a server on the client side, even for single player games. The server runs all the world gen, block state, player position, combat/projectile travel, etc. For larger SMP servers you spin up a dedicated server and the client doesn’t change at all.
I've tried many times to get networking + physics working and it's always a struggle. Many networking crates out there seem promising but always have some weird quirks and/or are unmaintained. Similarly I've had the kind of physics issues you describe. In general it also feels quite hard to hook everything up. Glad to see you're looking into this stuff, your examples and vids are always super helpful!
yeah, physics is a big problem space afaict and building a widely usable feature-rich physics crate is a big undertaking. I'll definitely make more videos as the ecosystem progresses and I'm glad you're enjoying the videos!
Awesome video- I went down this rabbithole trying to use Bevy to make multiplayer version of Asteroids that ran in the browser and using WebSockets I got it working but not really smoothly and then sort of gave up once the invisible part of the iceberg of the problem became apparent
Thanks for sharing your experience here! I've been playing with writing my own physics in Rust to use in browsers but had been hoping to look into libraries later - really helpful to know this will be tricky.
I've been struggling to build a network multiplayer with bevy + rapier for quite some time now and always felt the need to pretty much rewrite the basic components of physics and networking (I'm writing my own client / server with lobby management right now, for instance). I thought that this was due to my lack of understanding of the crates available in the ecosystem (which is true), but it seems though that we also don't have very mature solutions for deterministic multiplayer architectures as of now. Great video, thanks for sharing your experience.
Great video. This is definitely a course I would pay for once you get it up and running. I have gone down the multiplayer rabbit hole multiple times - very complex problem with physics. I still have only ever managed turn based games. I have been messing around with using wasm on v8 cloudflare workers on the edge for finding lobbies, keeping connections etc... Especially for small projects where you can have a small DB on the edge and could keep the entire server game logic there. I would also pay for a course going deeper into turn based games :) . Keep up the great content!
I am using Rapier for 2 month already for my game and I am totally exhausted. The way that rapier uses it's own world is completely wrong for bevy. I can not remove Collider from entity in system => bevy_rapier just don't see this, it watch for Changed. I can not change from RigidBody::Dynamic RigidBody::Fixed/Kinematic in system, it leads to potetially error phone state in the future. I can not use PostUpdate for system and remove some entity inside the system => rapier in next frame will fail. And some other small issues... I will definitely try XPBD, thanks for suggestion! I thought that only one maintaneable physics library is rapier.
You’re not really going to find out of box solutions for these problems, as I would’ve been out of the job a long time ago. Even in more complex games, you don’t really need a full physics simulation rollback. I do find it interesting that you assumed deploying a server too much to take on, but jumped straight into prediction and rollback. Start with basic p2p hard synchronization, and then improve on that.
I mean, if you implement server reconciliation you can always come back to the right path if the physics break, but you would have to consider one of the peers the source of truth
The best way to achieve this would still be an authoritative server/client setup. It can all be done in P2P with only a low power low bandwidth matchmaking server in between. Prediction/rollback is also extremely easy to implement, as long as you can manually tick the physics engine. In game networking, people like to make a huge fuss about "physics determinism", but it hardly matters as long as they're visually very close. It doesn't matter if one computer calculates floats differently from the other, because rollback completely takes care of any discrepancies. Prediction will ensure that both players feel like they're the host. If you do it right, it should be very hard to tell.
Doesn't solve any of the problems mentioned, but it might make server development easier: Have you tried the new spacetimedb or maybe even integrated it with Bevy? Rust + game-dev seem to be first class citizens and realtime updates + persistent state + type safe rpcs seem to be very nice too. Since the server modules are also written in Rust, it might be possible to run the source-of-truth simulation on the server and reuse the same code on the client ...
I haven't tried spacetimedb. I stay pretty far away from anything advertising itself as blockchain and they mention it quite a bit on their homepage so I'd have to do some research first. It also looks like they don't have serverside physics currently, and current games implement collisions in the client, so wouldn't actually solve the problems in the video yet afaict. www.reddit.com/r/rust_gamedev/comments/15n971z/spacetimedb_a_new_database_written_in_rust_that/jvpvuh5/
The latency between clients producing discrepancies reminds me of how gyroscopes have drift in robotics use cases, and they solve that problem by also using magnetometers and accelerometers (with slower measurement rates) and Kalman filters, and combining them into a sensor called the IMU. Maybe some inspiration can be taken here. Also, what if for events like collisions, it’s acceptable to wait for the server or consensus, but otherwise local simulation suffices for free motion? You could wait for the server if a possible collision is within an error tolerance, and receive occasional physics state corrections so the local simulation doesn’t drift off too much.
yeah if I had gone with a server-based approach it probably would've been simpler to sync because the clients would've been treated as "views" into the server. Re-using the p2p communication, but going with a more host-player structure to it could help as well. Downside to that is that one player has 0 ping and the other has a lot more. I was really hoping the physics simulation would be more stable, but it seems that the general opinion is that it won't ever be good enough, so assume it won't be and just go to client/server sooner. Luckily there's an interesting networking crate for client/server (bevy_renet). I was just hoping p2p would work better out of the box.
@@chrisbiscardi I still think it could work by basically separating out synchronization critical events to be handled differently than the local simulation. For Pong, these would be the collisions and occasional motion syncs to keep the drift in check. The sync critical events don’t strictly need to be handled by a server, they can be handled by a small consensus/vote, at least for Pong with only two players.
In the general case with more players, I reckon the vote only has to be between the players involved with the event. So for Pong, each player owns the collisions with their paddle, or something. Maybe the other players then get to veto if anything sus happens, such as the ball miraculously reflecting when the paddle isn’t even there. At that point I suppose you have a constitutional crisis, and maybe then you just have to close the session because there’s a hacker.
I totally disagree, that p2p is better for learning. In fact, I find it better to learn server client immediately, as I find it is better to eat the bigger frog in this case first; I find your choices are trying to go the easy path here, and by that extension, landing in the actually more complicated issues of networking. it might be easier to start with mmo full sync, go over to simple fps sync, then go for prediction and layed back integration, and only finally in the end touch the hot sauce of rollback networking.
> you are trying to take the easy path here quite literally yes, as I said in the video: I'm trying to find an easy path *for other people to learn and build on*. Its fine to disagree but you haven't provided an alternative path here or any resources to accomplish what you're suggesting. You've told me to go do a bunch more work on unrelated games before coming back and doing this again, none of which is guaranteed to be any easier because again, you haven't provided any resources to accomplish what you're suggesting.
![Lp. Сердце Вселенной #60 РОЖДЕНИЕ ЛОЛОЛОШКИ [Финал] • Майнкрафт](http://i.ytimg.com/vi/YoR0pAV9FVQ/mqdefault.jpg)
I think a common way used by many MMORPG games, is to simulate the world on the server (along with its physics), not only just validating actions. That way the client is primarily a screen to view what is being broadcasted to it, and an "action requester" to request actions to be taken by the entities you have permission to on the server.
This way, you only need a bare minimum of client side predictions to predict wether or not your requested actions are granted and what you should do. Everything else is parsing and presenting data recieved from the server.
yeah, I can see the delay to request from the server and wait for it to get back working for mmorpgs (or turn based games, etc). Eve also does literal time slowdown don't they?
Rocket League, which I mention in the video, does a lot of client side prediction but the server simulation is the source of truth, so if there's a discrepancy the server wins and client updates to reflect that, similar to what you're describing.
There are definitely a wide range of games with a wide range of solutions. Not having to pay for servers is something that works well into a workshop IMO, but at the end of the day if the best solutions are "run a server" then I'll be running a server for sure.
It's still super hard to mask network latency if it isn't already included in the whole game design, for instance via animations which mask the latency.
Rollback based netcode can be quite tough, but once properly implemented, was a godsend for fighting games in particular.
Personally I've measured the latencies to some friends, and packets from Germany to Australia can take 500 ms of latency on average (max was a staggering 1300ms), so even with that such games would be tough to play together
Minecraft spawns a server on the client side, even for single player games. The server runs all the world gen, block state, player position, combat/projectile travel, etc. For larger SMP servers you spin up a dedicated server and the client doesn’t change at all.
I've tried many times to get networking + physics working and it's always a struggle. Many networking crates out there seem promising but always have some weird quirks and/or are unmaintained. Similarly I've had the kind of physics issues you describe. In general it also feels quite hard to hook everything up. Glad to see you're looking into this stuff, your examples and vids are always super helpful!
yeah, physics is a big problem space afaict and building a widely usable feature-rich physics crate is a big undertaking. I'll definitely make more videos as the ecosystem progresses and I'm glad you're enjoying the videos!
maybe this would work: calculate physics on one machine and sent results in form of vector and position to others
This is one of those videos that I didn't know I wanted. Fun to hear you deep dive into these multiplayer game problems.
Awesome video- I went down this rabbithole trying to use Bevy to make multiplayer version of Asteroids that ran in the browser and using WebSockets
I got it working but not really smoothly and then sort of gave up once the invisible part of the iceberg of the problem became apparent
Thanks for sharing your experience here! I've been playing with writing my own physics in Rust to use in browsers but had been hoping to look into libraries later - really helpful to know this will be tricky.
I've been struggling to build a network multiplayer with bevy + rapier for quite some time now and always felt the need to pretty much rewrite the basic components of physics and networking (I'm writing my own client / server with lobby management right now, for instance). I thought that this was due to my lack of understanding of the crates available in the ecosystem (which is true), but it seems though that we also don't have very mature solutions for deterministic multiplayer architectures as of now. Great video, thanks for sharing your experience.
Great video. This is definitely a course I would pay for once you get it up and running. I have gone down the multiplayer rabbit hole multiple times - very complex problem with physics. I still have only ever managed turn based games. I have been messing around with using wasm on v8 cloudflare workers on the edge for finding lobbies, keeping connections etc... Especially for small projects where you can have a small DB on the edge and could keep the entire server game logic there. I would also pay for a course going deeper into turn based games :) . Keep up the great content!
I am using Rapier for 2 month already for my game and I am totally exhausted.
The way that rapier uses it's own world is completely wrong for bevy.
I can not remove Collider from entity in system => bevy_rapier just don't see this, it watch for Changed.
I can not change from RigidBody::Dynamic RigidBody::Fixed/Kinematic in system, it leads to potetially error phone state in the future.
I can not use PostUpdate for system and remove some entity inside the system => rapier in next frame will fail.
And some other small issues...
I will definitely try XPBD, thanks for suggestion! I thought that only one maintaneable physics library is rapier.
You’re not really going to find out of box solutions for these problems, as I would’ve been out of the job a long time ago. Even in more complex games, you don’t really need a full physics simulation rollback. I do find it interesting that you assumed deploying a server too much to take on, but jumped straight into prediction and rollback. Start with basic p2p hard synchronization, and then improve on that.
fuck, i've been dreaming of building a physics based multiplayer simulation game with bevy.
This is a year old! There are great networking solutions and new physics libraries that didn't exist then.
@@chrisbiscardiCan you tell, which ones are the most useful currently? All I can think of is renet
@@RomanAvdeevX there's quinnet, lightyear, replicon, matchbox, and some more
@@chrisbiscardi Think you very much! I'll check them out
Love the shoutout to the Fallacies of Distributed Computing.
I mean, if you implement server reconciliation you can always come back to the right path if the physics break, but you would have to consider one of the peers the source of truth
Awesome vids, great channel
The best way to achieve this would still be an authoritative server/client setup. It can all be done in P2P with only a low power low bandwidth matchmaking server in between. Prediction/rollback is also extremely easy to implement, as long as you can manually tick the physics engine.
In game networking, people like to make a huge fuss about "physics determinism", but it hardly matters as long as they're visually very close. It doesn't matter if one computer calculates floats differently from the other, because rollback completely takes care of any discrepancies.
Prediction will ensure that both players feel like they're the host. If you do it right, it should be very hard to tell.
12:06 Have a great Rust of the of the day!
so many good looking crates! I guess the only right choice is to build my own lol
Doesn't solve any of the problems mentioned, but it might make server development easier: Have you tried the new spacetimedb or maybe even integrated it with Bevy? Rust + game-dev seem to be first class citizens and realtime updates + persistent state + type safe rpcs seem to be very nice too. Since the server modules are also written in Rust, it might be possible to run the source-of-truth simulation on the server and reuse the same code on the client ...
I haven't tried spacetimedb. I stay pretty far away from anything advertising itself as blockchain and they mention it quite a bit on their homepage so I'd have to do some research first.
It also looks like they don't have serverside physics currently, and current games implement collisions in the client, so wouldn't actually solve the problems in the video yet afaict. www.reddit.com/r/rust_gamedev/comments/15n971z/spacetimedb_a_new_database_written_in_rust_that/jvpvuh5/
The latency between clients producing discrepancies reminds me of how gyroscopes have drift in robotics use cases, and they solve that problem by also using magnetometers and accelerometers (with slower measurement rates) and Kalman filters, and combining them into a sensor called the IMU. Maybe some inspiration can be taken here.
Also, what if for events like collisions, it’s acceptable to wait for the server or consensus, but otherwise local simulation suffices for free motion? You could wait for the server if a possible collision is within an error tolerance, and receive occasional physics state corrections so the local simulation doesn’t drift off too much.
yeah if I had gone with a server-based approach it probably would've been simpler to sync because the clients would've been treated as "views" into the server. Re-using the p2p communication, but going with a more host-player structure to it could help as well. Downside to that is that one player has 0 ping and the other has a lot more. I was really hoping the physics simulation would be more stable, but it seems that the general opinion is that it won't ever be good enough, so assume it won't be and just go to client/server sooner.
Luckily there's an interesting networking crate for client/server (bevy_renet). I was just hoping p2p would work better out of the box.
@@chrisbiscardi I still think it could work by basically separating out synchronization critical events to be handled differently than the local simulation. For Pong, these would be the collisions and occasional motion syncs to keep the drift in check. The sync critical events don’t strictly need to be handled by a server, they can be handled by a small consensus/vote, at least for Pong with only two players.
In the general case with more players, I reckon the vote only has to be between the players involved with the event. So for Pong, each player owns the collisions with their paddle, or something. Maybe the other players then get to veto if anything sus happens, such as the ball miraculously reflecting when the paddle isn’t even there. At that point I suppose you have a constitutional crisis, and maybe then you just have to close the session because there’s a hacker.
same.
I totally disagree, that p2p is better for learning. In fact, I find it better to learn server client immediately, as I find it is better to eat the bigger frog in this case first;
I find your choices are trying to go the easy path here, and by that extension, landing in the actually more complicated issues of networking.
it might be easier to start with mmo full sync, go over to simple fps sync, then go for prediction and layed back integration, and only finally in the end touch the hot sauce of rollback networking.
> you are trying to take the easy path here
quite literally yes, as I said in the video: I'm trying to find an easy path *for other people to learn and build on*.
Its fine to disagree but you haven't provided an alternative path here or any resources to accomplish what you're suggesting. You've told me to go do a bunch more work on unrelated games before coming back and doing this again, none of which is guaranteed to be any easier because again, you haven't provided any resources to accomplish what you're suggesting.
same
rollback and ECS do not mix well. or you need two different worlds, one for rollback etc and one for graphics.
Bevy's ECS already splits rendering out into its own world. Maybe other implementations differ in that respect?
@@chrisbiscardi Oh it does? Where can I read about it? I was under impression you'd have to do that manually...
I'd start here: bevy-cheatbook.github.io/gpu/intro.html
Thanks, @@chrisbiscardi !
same
same
same
same