Thanks stephen! I hear you loud and clear - I've added a CI / Deployment item to my backlog of videos. Hopefully I can have something out soon. Cheerss
I'm going through the System design playlist and just wanna say these are so valuable. Appreciate all the effort that went behind this and for keeping it free!
From my perspective, latency can not limit throughput directly, but it can delay the amount of data that we trying to sent/receive. For example if we have a server in Asia and we are trying to do a request from South America, even if the server in Asia has a high throughput (e.g. up to 500k requests per second in a client server model) the time that our requests will take to reach the server will not affect the amount of requests the server can handle (still 500k requests per second). For us as a client having a high latency means wait more time to servers to react, on other hand low throughput means requests start being denied at some point or server crash (if the server is not prepared for that) There's some cases where throughput can affect latency, for example when we're doing requests and the server reach their limits and it starts scaling up (vertically or horizontally) so our requests will take more time to be processed since those will need to wait for the scaling process (In modern systems, if we are not talking about scaling in different regions across the globe, usually some milliseconds)
This is a very nice explanation, i was searching for better understating for a while now. Thank you. Also, i have a question around network bandwidth. how much network bandwidth does cloud provider provides? because if I can increase number of servers and my each request is 1 mb how many concurrent requests can my network handle? Because there should be some limit to the network which connects from client to Load balancer.
Another great video. Thanks very much. They are always very insightful. I was wondering if you would do a video on instance sizing. Picking the right type of instance for your application and the things to Consider
Thank you soooo much for the awesome tutorial! It is super clear. Just one small question. If we level up throughput, does it mean that the processing latency could be lower? Thank you for any answer you provide.
Hi Anyi, Increasing throughput (concurrency) can actually INCREASE latency. The reason is because typically to increase concurrency, we increase the amount of requests to a host while keeping the number of hosts fixed. This can result in more CPU usage on higher latency API calls. If you add more hosts, then this shouldn't be a problem. Hope this helps, Daniel
Most clear explanation about latency vs throughput in the internet. Thanks!
Glad it was helpful!
This is great I really enjoy your content. I'd love to see any videos on CI/CD tools deploying to AWS.
Thanks stephen! I hear you loud and clear - I've added a CI / Deployment item to my backlog of videos. Hopefully I can have something out soon.
Cheerss
Its not a hard concept to grasp, the problems arise when you try to measure them properly.
Same here
I'm going through the System design playlist and just wanna say these are so valuable. Appreciate all the effort that went behind this and for keeping it free!
Regarding Latency/counts chart. The axises should be swapped. Latency should be on the vertical axis as this is a function of counts
I would have liked to have heard more about the relationship of latency and throughput and how latency can limit potential throughput.
From my perspective, latency can not limit throughput directly, but it can delay the amount of data that we trying to sent/receive. For example if we have a server in Asia and we are trying to do a request from South America, even if the server in Asia has a high throughput (e.g. up to 500k requests per second in a client server model) the time that our requests will take to reach the server will not affect the amount of requests the server can handle (still 500k requests per second).
For us as a client having a high latency means wait more time to servers to react, on other hand low throughput means requests start being denied at some point or server crash (if the server is not prepared for that)
There's some cases where throughput can affect latency, for example when we're doing requests and the server reach their limits and it starts scaling up (vertically or horizontally) so our requests will take more time to be processed since those will need to wait for the scaling process (In modern systems, if we are not talking about scaling in different regions across the globe, usually some milliseconds)
Thank you very much for this clear , pragmatic explanation.
This is a very nice explanation, i was searching for better understating for a while now. Thank you.
Also, i have a question around network bandwidth. how much network bandwidth does cloud provider provides? because if I can increase number of servers and my each request is 1 mb how many concurrent requests can my network handle? Because there should be some limit to the network which connects from client to Load balancer.
what is the difference between Response time and Latency? It seems your definition of latency is same as the response time.
Such an elegant demonstration ! I am sincerely grateful to you, Master.
Another great video. Thanks very much. They are always very insightful. I was wondering if you would do a video on instance sizing. Picking the right type of instance for your application and the things to Consider
Great idea for a video, thank you!
Gold nugget.
Thank you soooo much for the awesome tutorial! It is super clear.
Just one small question. If we level up throughput, does it mean that the processing latency could be lower?
Thank you for any answer you provide.
Hi Anyi,
Increasing throughput (concurrency) can actually INCREASE latency. The reason is because typically to increase concurrency, we increase the amount of requests to a host while keeping the number of hosts fixed. This can result in more CPU usage on higher latency API calls. If you add more hosts, then this shouldn't be a problem.
Hope this helps,
Daniel
Sir you are toooo good, love from India
Great videos!
What’s the difference between Throughput and Bandwidth?
Really liked this video, thank you :)
Glad you enjoyed it!
In your definition of throughput, what is the size of your packet?
Have never seen someone explain latency and throughput in this much depth
THank you soo much. Really enjoyed this talk
You're very welcome abdul!
thank you this was a really good source of information
You're very welcome Spas!
Thank you for this clear explanation
You're very welcome Josh!
Thank you. This is really helpful
You're very welcome Mahesh!
Thanks, very well explained!!
Thank you Sir.
You're very welcome Raj!
Thanks for taking the time to explain this so well.
awesome
thumb up for you nice voice
Thank you!
Thanks, very useful
You're very welcome Kamal!