I'm going through the System design playlist and just wanna say these are so valuable. Appreciate all the effort that went behind this and for keeping it free!
Thanks stephen! I hear you loud and clear - I've added a CI / Deployment item to my backlog of videos. Hopefully I can have something out soon. Cheerss
From my perspective, latency can not limit throughput directly, but it can delay the amount of data that we trying to sent/receive. For example if we have a server in Asia and we are trying to do a request from South America, even if the server in Asia has a high throughput (e.g. up to 500k requests per second in a client server model) the time that our requests will take to reach the server will not affect the amount of requests the server can handle (still 500k requests per second). For us as a client having a high latency means wait more time to servers to react, on other hand low throughput means requests start being denied at some point or server crash (if the server is not prepared for that) There's some cases where throughput can affect latency, for example when we're doing requests and the server reach their limits and it starts scaling up (vertically or horizontally) so our requests will take more time to be processed since those will need to wait for the scaling process (In modern systems, if we are not talking about scaling in different regions across the globe, usually some milliseconds)
Another great video. Thanks very much. They are always very insightful. I was wondering if you would do a video on instance sizing. Picking the right type of instance for your application and the things to Consider
This is a very nice explanation, i was searching for better understating for a while now. Thank you. Also, i have a question around network bandwidth. how much network bandwidth does cloud provider provides? because if I can increase number of servers and my each request is 1 mb how many concurrent requests can my network handle? Because there should be some limit to the network which connects from client to Load balancer.
Thank you soooo much for the awesome tutorial! It is super clear. Just one small question. If we level up throughput, does it mean that the processing latency could be lower? Thank you for any answer you provide.
Hi Anyi, Increasing throughput (concurrency) can actually INCREASE latency. The reason is because typically to increase concurrency, we increase the amount of requests to a host while keeping the number of hosts fixed. This can result in more CPU usage on higher latency API calls. If you add more hosts, then this shouldn't be a problem. Hope this helps, Daniel
Most clear explanation about latency vs throughput in the internet. Thanks!
Glad it was helpful!
I'm going through the System design playlist and just wanna say these are so valuable. Appreciate all the effort that went behind this and for keeping it free!
Have never seen someone explain latency and throughput in this much depth
Such an elegant demonstration ! I am sincerely grateful to you, Master.
Thank you very much for this clear , pragmatic explanation.
This is great I really enjoy your content. I'd love to see any videos on CI/CD tools deploying to AWS.
Thanks stephen! I hear you loud and clear - I've added a CI / Deployment item to my backlog of videos. Hopefully I can have something out soon.
Cheerss
I would have liked to have heard more about the relationship of latency and throughput and how latency can limit potential throughput.
From my perspective, latency can not limit throughput directly, but it can delay the amount of data that we trying to sent/receive. For example if we have a server in Asia and we are trying to do a request from South America, even if the server in Asia has a high throughput (e.g. up to 500k requests per second in a client server model) the time that our requests will take to reach the server will not affect the amount of requests the server can handle (still 500k requests per second).
For us as a client having a high latency means wait more time to servers to react, on other hand low throughput means requests start being denied at some point or server crash (if the server is not prepared for that)
There's some cases where throughput can affect latency, for example when we're doing requests and the server reach their limits and it starts scaling up (vertically or horizontally) so our requests will take more time to be processed since those will need to wait for the scaling process (In modern systems, if we are not talking about scaling in different regions across the globe, usually some milliseconds)
Gold nugget.
Thanks for taking the time to explain this so well.
Sir you are toooo good, love from India
Regarding Latency/counts chart. The axises should be swapped. Latency should be on the vertical axis as this is a function of counts
Thank you for this clear explanation
You're very welcome Josh!
Another great video. Thanks very much. They are always very insightful. I was wondering if you would do a video on instance sizing. Picking the right type of instance for your application and the things to Consider
Great idea for a video, thank you!
thank you this was a really good source of information
You're very welcome Spas!
THank you soo much. Really enjoyed this talk
You're very welcome abdul!
Thank you Sir.
You're very welcome Raj!
Thank you. This is really helpful
You're very welcome Mahesh!
Thanks, very well explained!!
Really liked this video, thank you :)
Glad you enjoyed it!
Great videos!
This is a very nice explanation, i was searching for better understating for a while now. Thank you.
Also, i have a question around network bandwidth. how much network bandwidth does cloud provider provides? because if I can increase number of servers and my each request is 1 mb how many concurrent requests can my network handle? Because there should be some limit to the network which connects from client to Load balancer.
Thank you soooo much for the awesome tutorial! It is super clear.
Just one small question. If we level up throughput, does it mean that the processing latency could be lower?
Thank you for any answer you provide.
Hi Anyi,
Increasing throughput (concurrency) can actually INCREASE latency. The reason is because typically to increase concurrency, we increase the amount of requests to a host while keeping the number of hosts fixed. This can result in more CPU usage on higher latency API calls. If you add more hosts, then this shouldn't be a problem.
Hope this helps,
Daniel
Thanks, very useful
You're very welcome Kamal!
What’s the difference between Throughput and Bandwidth?
what is the difference between Response time and Latency? It seems your definition of latency is same as the response time.
Its not a hard concept to grasp, the problems arise when you try to measure them properly.
Same here
awesome
thumb up for you nice voice
Thank you!
In your definition of throughput, what is the size of your packet?