My Rocking Systems Design course is out now in Udemy! Limited time discounted link to get the course for $9.99 - www.udemy.com/course/rocking-system-design/?couponCode=LEARNSYSTEMDESIGN
Raj, your videos are incredibly helpful. You are masterful at breaking down complex technical requirements into readily understood concepts. I’ve learned a ton from just a few videos. Thank you for all of your help!
Great video! When explaining when to generate timelines: "It will only create the timelines for the users who have been active in the last 10 hours". I'm assuming that this reduces the compute strain? But what happens when everyone on the US East Coast wakes up, everyone starts logging and looking at their Twitter feed at roughly the same time? The system is busy creating timelines for millions of users at the same time. How does this solution cope with such a situation? Some content suggestion/request for future videos - Compare/contrasts different way to implement a problems using different AWS technology. So for example so how you would implement something using EC2 vs. using Fargate vs. Lambda and contrast the trade-offs. Same for using Redis vs. Elasticache vs DAX, or using SNS vs. SQS, etc...
Hi Raj! excellent video! I have a question about the EC2 and ASG part, what if instead of EC2 we use Step Functions with Lambda and EventBridge? could be an option? I mean having a Step Functions workflow that inserts the tweet on Redis and then to Kinesis. And also, could Redis be replaced with Dynamodb?
Hi Luis, Raj mentions in the video that Redis can be replaced by DynamoDB and if the system needs even better performance than plain DDB, you can use DynamoDB Accelerator (DAX)
Raj thanks a lot for this video. As far as group messaging is concerned, we will use notification service with one queue. As for grouping message is same for every member so our message will pass by SQS and after SQS we will use a notification to every single group member. Is it right please suggest? Rest of the design would be the same.
Hello Raj, thank you for the video. It was very informative. I am curious though. Is Twitter really using a kind of table as you described for the relationships; where a user's name appears in multiple rows to link that user to his or her followers? I had always assumed some kind of graph database based on what I read on data structures and algorithms.
Question: kms gets overloaded with high number of api calls, they cached it with S3 but how does it happen with other services. What is the throughput and how kms cops up with api calls
How would the schema look like if you were to use DynamoDB? The keys need to be unique, so you could not have three rows of Tina unless you had some other partition key. Would you have some random-ish hashkey as the partition key and then have the Tina column be a GSI?
Hey Mahmoud, Redis is considered a NoSQL database, however it has limited capabilities to create keys and query using the keys. The primary reason to use Redis is to have really fast read/write ability.
Why do we do not use EKS or Kubernetes in EC2 before Database rather than use Autoscaling and EC2? It will reduce Cost of Autoscaling and invocation of EC2?
Would API Gateway scale enough to handle twitter-esque scale, or would you need ALB/NLB? The AWS Quotas docs state that AWS Gateway is limited by default to 10,000 RPS, but it can be increased upon request. Still, even at a 10x increase, I'm not sure 100k RPS would be enough.
Great question megabenman! The limits are based on account/region. Beyond raising the limits, you also segregate the APIs on different region/API based on user location. Also CloudFront comes into play to cache certain contents without hitting the backend API. Generally it's a mix of ALB and API Gateway for different APIs. ALBs you can prewarm to handle high spokes for certain events (New Year), but it does cost you more.
My Rocking Systems Design course is out now in Udemy! Limited time discounted link to get the course for $9.99 - www.udemy.com/course/rocking-system-design/?couponCode=LEARNSYSTEMDESIGN
I cant watch it in corporate side of the udemy. is it not available to companies?
Raj, your videos are incredibly helpful. You are masterful at breaking down complex technical requirements into readily understood concepts. I’ve learned a ton from just a few videos. Thank you for all of your help!
Great and simple to understand
Explanation is precise and simple. Please continue with such videos. Thanks for your effor
Thank you Kashmora
youre like the only channel that makes information so digestible i really appreciate it
I appreciate that!
Great video!
When explaining when to generate timelines: "It will only create the timelines for the users who have been active in the last 10 hours". I'm assuming that this reduces the compute strain? But what happens when everyone on the US East Coast wakes up, everyone starts logging and looking at their Twitter feed at roughly the same time? The system is busy creating timelines for millions of users at the same time. How does this solution cope with such a situation?
Some content suggestion/request for future videos - Compare/contrasts different way to implement a problems using different AWS technology. So for example so how you would implement something using EC2 vs. using Fargate vs. Lambda and contrast the trade-offs. Same for using Redis vs. Elasticache vs DAX, or using SNS vs. SQS, etc...
Your biggest fan..keep sharing the knowledge 👍 ...More system design please 😁
More to come!
Raj, this is super simple. Have also bought your course.
Good One ..thanks
Pretty funny you chose Elon Musk as the famous person way prior to him buying the company. Great video btw!
Thanks Moccar!
Thanks Raj!! Let it flow....it's amazing how you turn this complex topics in to simple & understandable
Thanks Rony for the kind words 🙏
Nice video
Great videos from you ! I am learning a lot of AWS !! Thanks a lot
AMAZING
Excellent presentation!
Hi Raj! excellent video! I have a question about the EC2 and ASG part, what if instead of EC2 we use Step Functions with Lambda and EventBridge? could be an option? I mean having a Step Functions workflow that inserts the tweet on Redis and then to Kinesis. And also, could Redis be replaced with Dynamodb?
Hi Luis, Raj mentions in the video that Redis can be replaced by DynamoDB and if the system needs even better performance than plain DDB, you can use DynamoDB Accelerator (DAX)
Great video! TY!
Glad you enjoyed it!
Thanks ! Excellent understanding .
Brilliant Video. Many Thanks Raj
Glad you liked it!
Appreciate the great knowledge sharing course.
Glad it was helpful!
You are amazing
Ty for the kind words Mesanga
Thank you. This is very informative. Please do create more such videos. I am great fan of your videos
Thank you, I will Mahesh
Amazing episode! Thank you.
Excellent thankyou
Nicely explained Raj , Thank you so much
Thanks Shyam for your support as always 🙏
Raj thanks a lot for this video. As far as group messaging is concerned, we will use notification service with one queue. As for grouping message is same for every member so our message will pass by SQS and after SQS we will use a notification to every single group member. Is it right please suggest? Rest of the design would be the same.
Like ur shirt bru👊
Ty bro!
We can use any graph db to store followers relationship if not why?
Hey Raj Thanks for the awesome video... Smashed the like button as you said😃😃👍
Thanks Sharath for your support as always 🙏
Can you please make a video on Bandwidth pricing of AWS and other cloud providers?
brilliant as usual .. Thank You!
Thanks again!
Perfect as usual!
Thank you! Cheers!
Hello Raj, thank you for the video. It was very informative. I am curious though. Is Twitter really using a kind of table as you described for the relationships; where a user's name appears in multiple rows to link that user to his or her followers? I had always assumed some kind of graph database based on what I read on data structures and algorithms.
For mobile app deployment will need separate two builds ? One for apple other android app store right?
Question: kms gets overloaded with high number of api calls, they cached it with S3 but how does it happen with other services. What is the throughput and how kms cops up with api calls
How would the schema look like if you were to use DynamoDB? The keys need to be unique, so you could not have three rows of Tina unless you had some other partition key. Would you have some random-ish hashkey as the partition key and then have the Tina column be a GSI?
Another great comment megabenman. Yes, the table needs to have both partition and sort key to make each record unique.
Thank you , excellent video , very well explained , in simple language . really appreciated your video can we have some video on AWS Glue ETL tool ?
Thanks Mangesh, yes planning on doing an AWS Analytics video in near future!
Hi Raj, great videos, please continue , I like your content, but question: is iy sql or nosql DB?
Hey Mahmoud, Redis is considered a NoSQL database, however it has limited capabilities to create keys and query using the keys. The primary reason to use Redis is to have really fast read/write ability.
Why do we do not use EKS or Kubernetes in EC2 before Database rather than use Autoscaling and EC2? It will reduce Cost of Autoscaling and invocation of EC2?
Would API Gateway scale enough to handle twitter-esque scale, or would you need ALB/NLB? The AWS Quotas docs state that AWS Gateway is limited by default to 10,000 RPS, but it can be increased upon request. Still, even at a 10x increase, I'm not sure 100k RPS would be enough.
Great question megabenman! The limits are based on account/region. Beyond raising the limits, you also segregate the APIs on different region/API based on user location. Also CloudFront comes into play to cache certain contents without hitting the backend API. Generally it's a mix of ALB and API Gateway for different APIs. ALBs you can prewarm to handle high spokes for certain events (New Year), but it does cost you more.
Where can I get that shirt? Employees only? Ive been looking for some more AWS gear because the stuff in the certification store is just...meh
This is from a Serverless hackathon I participated in Imonous, don't think you can buy this one :(
Thank you , assume full stack code ready, deployment of web twitter.com will be on container s but not server less ? As there's database?
Like ur shirt bru👊
Ty Resham 👊