We scaled our socket solution with a redis middleware that persists the session id. And it worked great. Very surprised to see no mention of this approach.
This video doesn't show how to scale persistent connections. Who uses websockets with AWS lambda functions? We use websockets to stream data between server and client and lambda functions are stateless and cannot remain up indefinitely. Most applications want to maintain a connection for days, weeks or indefinitely so how does this architecture fix anything in those situations. The gateway can allow you to load balance those persistent connections but that's it.
Websockets are horizontally scalable, you use a message bus where the message is sent to each websocket host also known as a backplane, and you will need to setup sticky sessions with the load balancer. The reason for this is that you don't know what server the client is on, so you send it to all of them. You can also chain websockets together e.g. a websocket server is a client of another websocket server, it's the same concept as the message bus. Also you can downscale due to redistrobution of connection via the load balancer.
Yes, I was also thinking same way. Why can't we use Kafka here...and assign a key to each client so that we will know to which server it can map to...that way we don't have to send it to every server
@jwbonnett suggestion is the correct approach, leverage message brokers and pub/sub for horizontally scaled instances of a server to publish messages for changes it makes which get broadcast by the message broker to the instances subscribing to that event. Those instances will then send those updates its client connections. This is how all clients know about changes by other clients in real-time. Load balancer tracks the client sessions using sticky sessions feature which uses least connection algorithm to load balance the connections to the replicated server instances. This video covers the topic well: ua-cam.com/video/vXJsJ52vwAA/v-deo.html
Websockets maintain state, so if you remove the statefulness of the connection in API gateway how would you handle a case when server wants to send a push request to clients. How does the lambda know which ws connection to try? The http connection is stateless
I want to add 2 points to your video. The rise of anycast based network architectures and edge/serverless edge based architectures make us think differently about websockets and scaling them. I've seen some interesting deployments of self hosted pbx, voip and even office tools like collabra over anycast architectures and they work quite seamlessly. There definitely will be issues/drops during scaling activity but a quick reconnection usually solves the problem. Edge/edge serverless(like cloudflare workers) also have the same ideology as aws api gateway but the added advantage is the session is on the edge. This becomes a game changer for deployments in the above mentioned scenarios. That said, it's not perfect. It has its problems. But the pros outweigh the cons by a big margin for about 70% of the workloads that I deal with.
@@codedamn at a high level, anycast is a faster backbone to your origin server. In aws you can do it with global accelerator. You can setup global accelerator + ec2 or nlb as origin with multiregion deployment. In Google cloud, you can use Google load balancer and that also does the same thing. Throw in a random websocket server and play around with multi region with peak load. There is a noticeable amount of difference. Btw, you have some great content in your channel. Hats off for your effort.
I recently implemented a tic-tak-toe game using websockets with room functionality and I didn't know websockets are hard to scale well thanks for this information, your content is great 👍
I have a friend who has a socket based business that grew 100x faster than they expected. His life was hell scaling that software lol I also had a socket based business that I just recently sold. Only had a few thousand users but i never had a single issue with the sockets.
Your are giving a misleading information (not in purpose, I know) but WS are as scalable as TCP connections are. You should say that node.js is not scalable, instead.
How can maintain the business logics and all sync with different servers. I have already used the redis pub/sub but i can't sync a local objects are sync.
Thanks for the nice video! One question, how API gateway knows to which client the message needs to be delivered? Is there an internal queue per client automatically created by API gateway?
Just a helpful info. AWS Api Gateway needs to respond within 30 sec. If the backend behind the gateway takes more than 30sec then the API Gateway will directly respond with TIMEOUT json.
Hey there great clip. I think another way of doing this (not as scalable as this) would be to have an nginx as a reverse proxi in front of multiple node servers that would distribute the sokets among multiple node servers. This solution would unfortunately require you to change your code quite a bit since you wouldn't be able to guarantee that a socket is connected to the server that got a request, so the app instances would need to negotiate among themselves using a message system (maybe redis) and ask the node instance that has a connection with that particular client to send a message.
Websockets are hard to scale but not impossible to scale. Using a service as recommended here doesn't really solve it, but rather just passes the problem on to AWS. That might or might not be acceptable depending on your business needs.
Hey! Congrats on Winning the codedamn T Shirt Giveaway. Please reply to this comment by mentioning your email for further communication. Or just go on your channel page and set an email address so that we can contact you that way. Or just reply here without using @ gmail . com.
Websockets are scalable. Just use multiple of them. And keep a database for which user group is using which socket. That’s how I solved it in my application.
Websockets are pretty scalable, just for some solutions using them is not best for scaling and some is best for scaling. The only thing you have worry about is tcp sockets.
Yesterday I faced cores issue. UA-cam recommended your video. Not searched in youtube.that time I am not watching tech videos , but youtube recommended. Today websocket also same happened. Currently I am building real time chat application. Thanks for your video
I always used to think if websockets are that of much fast, why use http, but now understood they are not reliable and scalable so http is better, thanks codedamn. Can you please tell us which language is better to us websockets on, php node or python....
You talk a lot about connection limits but I would have thought the main limiting factor would be how much traffic is going back and forth in the open connection.
Thank you so much for this, your videos are really awesome with great video quality and content too. With help of your videos now I am able to build web projects. Thanks a lot!!
Learn WebSockets and other Full Stack Technologies on codedamn: cdm.sh/fullstack
Hey Mehul, can you please show us how to implement this by a project maybe a simple chat app or something.. mostly we all use sockets for chat app ig
We scaled our socket solution with a redis middleware that persists the session id. And it worked great. Very surprised to see no mention of this approach.
This video doesn't show how to scale persistent connections. Who uses websockets with AWS lambda functions? We use websockets to stream data between server and client and lambda functions are stateless and cannot remain up indefinitely.
Most applications want to maintain a connection for days, weeks or indefinitely so how does this architecture fix anything in those situations. The gateway can allow you to load balance those persistent connections but that's it.
Websockets are horizontally scalable, you use a message bus where the message is sent to each websocket host also known as a backplane, and you will need to setup sticky sessions with the load balancer. The reason for this is that you don't know what server the client is on, so you send it to all of them. You can also chain websockets together e.g. a websocket server is a client of another websocket server, it's the same concept as the message bus. Also you can downscale due to redistrobution of connection via the load balancer.
This is the right answer. The solution in this video isn't a good one
I am really confused by this one. Is there any resource from which I can actually read this up?
Yes, I was also thinking same way. Why can't we use Kafka here...and assign a key to each client so that we will know to which server it can map to...that way we don't have to send it to every server
@jwbonnett suggestion is the correct approach, leverage message brokers and pub/sub for horizontally scaled instances of a server to publish messages for changes it makes which get broadcast by the message broker to the instances subscribing to that event. Those instances will then send those updates its client connections. This is how all clients know about changes by other clients in real-time. Load balancer tracks the client sessions using sticky sessions feature which uses least connection algorithm to load balance the connections to the replicated server instances.
This video covers the topic well: ua-cam.com/video/vXJsJ52vwAA/v-deo.html
Websockets maintain state, so if you remove the statefulness of the connection in API gateway how would you handle a case when server wants to send a push request to clients. How does the lambda know which ws connection to try? The http connection is stateless
I want to add 2 points to your video. The rise of anycast based network architectures and edge/serverless edge based architectures make us think differently about websockets and scaling them.
I've seen some interesting deployments of self hosted pbx, voip and even office tools like collabra over anycast architectures and they work quite seamlessly. There definitely will be issues/drops during scaling activity but a quick reconnection usually solves the problem.
Edge/edge serverless(like cloudflare workers) also have the same ideology as aws api gateway but the added advantage is the session is on the edge. This becomes a game changer for deployments in the above mentioned scenarios.
That said, it's not perfect. It has its problems. But the pros outweigh the cons by a big margin for about 70% of the workloads that I deal with.
I don’t know a lot about anycast networks but this sounds very interesting. Thank you for sharing. Do you have any reading materials on this?
@@codedamn at a high level, anycast is a faster backbone to your origin server.
In aws you can do it with global accelerator. You can setup global accelerator + ec2 or nlb as origin with multiregion deployment.
In Google cloud, you can use Google load balancer and that also does the same thing.
Throw in a random websocket server and play around with multi region with peak load. There is a noticeable amount of difference.
Btw, you have some great content in your channel. Hats off for your effort.
Those AWS heads tend to forget that Nginx could balance WebSocket connections with ease.
I recently implemented a tic-tak-toe game using websockets with room functionality and I didn't know websockets are hard to scale well thanks for this information, your content is great 👍
I have a friend who has a socket based business that grew 100x faster than they expected. His life was hell scaling that software lol
I also had a socket based business that I just recently sold. Only had a few thousand users but i never had a single issue with the sockets.
@@reprovedcandy then, what is alternative?
Your are giving a misleading information (not in purpose, I know) but WS are as scalable as TCP connections are.
You should say that node.js is not scalable, instead.
Also, I think what he should say is that websockets are not downscalable which seems to be the only problem he mentions.
Yes the downscale issue seems to be the main concern
JESUS CHRIST. Shut the music down.
How can maintain the business logics and all sync with different servers.
I have already used the redis pub/sub but i can't sync a local objects are sync.
ERLANG VM entered the chat 🤣
sir i am an architrct and i dont agree tht sockets arent horzntly scalable ... simply use backolane to keep connections and memory and sessions .
Thanks for the nice video!
One question, how API gateway knows to which client the message needs to be delivered? Is there an internal queue per client automatically created by API gateway?
Just a helpful info.
AWS Api Gateway needs to respond within 30 sec.
If the backend behind the gateway takes more than 30sec then the API Gateway will directly respond with TIMEOUT json.
Hey there great clip. I think another way of doing this (not as scalable as this) would be to have an nginx as a reverse proxi in front of multiple node servers that would distribute the sokets among multiple node servers. This solution would unfortunately require you to change your code quite a bit since you wouldn't be able to guarantee that a socket is connected to the server that got a request, so the app instances would need to negotiate among themselves using a message system (maybe redis) and ask the node instance that has a connection with that particular client to send a message.
Redis, bingo.
2:21 Until the world runs out of silicon 😂
I ran into the same problem..what I did was run a sister socket concurrently so if one socket fails..the sister socket continues to run.
in the aws solution you mentioned, how would you initiate and send a message/data from server (lambdas in your case) to the client?
Use message broker to publish message centrally to every instance of web socket server
This is not a good solution, it doesn't apply to on prem infrastructure.
Bro you can vertical..the the idea is spread the load over the ports
Serverless will absolutely allow horizontally scaling of web sockets. Each message is dispatched to a serverless function for processing.
great explanation. thanks.
Amazing, as always🔥
As a full stack dev, how should one start learning these important cloud concepts and technologies?
I was watching a WhatsApp system design video and was confused about the gateway. Luckily I found your video. Very well explained. 😊
Another alternative is switching to gRPC which can provide bidirectional communication.
Code Damn is just an amazing channel of an amazing person...I'm really excited for this...
So many minutes to conclude into stupid cloud.
All good but one thing COST.
Your content is always amazing!
Just use long polling
i need this tshirt
Websockets are hard to scale but not impossible to scale. Using a service as recommended here doesn't really solve it, but rather just passes the problem on to AWS. That might or might not be acceptable depending on your business needs.
Great video.
it's amazing can you make live experiment
video, so it will be more helpful
You basically just proved why websockets are horizontally scalable. Amazon isn't using magic, you know. They just have a bunch of computers.
topics like webSockets, server sent events are hard to find and it's a mess in your channel make special playlist for talking about strategies
Really?
Discord scaled 5million concurrent connections on web sockets
Did you not watch the video?
@@codedamn Thanks for the swift reply 🙏
I did.
Can you shed some light how they did it?
There is actually a blog written by discord's cto where he mentioned about this along with why and how they migrated from mongodb to cassandra
@@surajvijay1937 thanks
Marker cap moved from little finger to ring finger @9.57
Api Gateway is underdeveloped at the moment, missing WS features. It also have a 2 hour timeout which is very unfortunate.
codedamn forever 🔥
Hey! Congrats on Winning the codedamn T Shirt Giveaway. Please reply to this comment by mentioning your email for further communication.
Or just go on your channel page and set an email address so that we can contact you that way.
Or just reply here without using @ gmail . com.
Great
Websockets are scalable. Just use multiple of them. And keep a database for which user group is using which socket. That’s how I solved it in my application.
New Java feature introduced in Java 18 as a virtual thread makes websockets very scalable, you can run hundred of thousands thread without a problem.
Websockets are pretty scalable, just for some solutions using them is not best for scaling and some is best for scaling. The only thing you have worry about is tcp sockets.
International Level Content only available in Indian Tech "Codedamn" Channel 🔥🔥🔥
Yesterday I faced cores issue. UA-cam recommended your video. Not searched in youtube.that time I am not watching tech videos , but youtube recommended. Today websocket also same happened. Currently I am building real time chat application. Thanks for your video
Good knowledge. BTW, my T shirt size is L :)
COMMENTED FOR T SHIRT GIVEAWAY!!!! :)
You probably missed the part when the server connects to the client.
I dont want t shirt i want more knowledge 💥💥 thankyou sir
"codedamn" are damn Crazy 🔥
I always used to think if websockets are that of much fast, why use http, but now understood they are not reliable and scalable so http is better, thanks codedamn. Can you please tell us which language is better to us websockets on, php node or python....
If I'm to have a solution that's not cloud based, I would use SSE's for as long as the event period is at least three seconds.
You talk a lot about connection limits but I would have thought the main limiting factor would be how much traffic is going back and forth in the open connection.
Good explanation. Thanks!
doesn't phoenix channels use very horizontally scaleable websockets?
The BEAM erlang vm basically
Miss your videos
Nice. Now give me a T-Shirt
Every time i get notified of your video, i come here as soon as I can
Thank you so much for this, your videos are really awesome with great video quality and content too.
With help of your videos now I am able to build web projects.
Thanks a lot!!
You never said how to scale it??.What if my app gets used by millions of people everyday
Use Erlang
Redis.
Thank you for giving solutions to problems. It was really a good concept to understand and know.
Thanks, i was trying to learn sockets, it helps!
Your information is much appreciated
Please make a video on message broker
Is this related to system design?
awesome explanation
This is great! thank you.
Great video!
Awesome video
thanks
Wow 🔥
Real good explanation 👍
amazing video
Thank you!
ty
🔥🔥
Ajinkya Rahane