EDIT: Amazon supports 20 GSIs instead of the 4 mentioned in the video. Thanks to anand_ammathil who pointed this in the comments 😁 Glad to see you here again! If you are looking for more system design content, try my course at InterviewReady: interviewready.io/learn/system-design-course Cheers :D
I don't really understand most of the stuff you say because I am just an undergrad but all this stuff is so fascinating....I love the engineering involved in these big applications!
Wow 😮, i used this every day in a project and fetched saved consistently but today i understood the depth 😢 Thanks a lot sirji ❤ Its a gem of a channel
DynamoDb also works on pay on demamd model which cost you for the RCUs and WCUs you made, I think which is great where i don't need to bother what capacity i had to choose and irregular traffic on system.
Few of the issues i had when working with dynamodb is how thier pricing works. They charge based on number of records scanned or written. I had a requirement where my application performs a lot of reads. Now, the problem is dynamo charges based on number of records scanned and not based on number of records matched. As long as you query based on primary key you are fine but the moment you query based on non primary attribute you are screwed, in this case dynamo essentially scans the entire table and consume a table worth of read units.
I think a single table can allow more than 4 GSIs, beacuse i had used 6 in one of application which works on single table design. Thanks for this great content @Gaurav ❤
Hey GKCS, I'm a newbie to sys designs and usually things just bounce off my head or I need to watch them multiple times to gain some sort of assumed sense but the data scheme part of this video was great! I understood almost everything and was enjoying it a lot. I dunno if you did anything different for that section or it was just the visuals, but maybe try doing whatever you did more?
Hey Gaurav, i got confused in caching strategy, you meant when requests rate go down cache will make sure to keep on querying the db so that when there is again increase in traffic it will be ready to handle that much load through cache only ?
Great video as always, currently I'm reading a book "Design data intensive applications " I found it really interesting. Can you suggest some more Programing books for distributed systems and backend engineering
It's a file/object store, fundamental to distributed systems. Google uses Colossus which has a similar function, and is used by nearly every system in their stack.
Google spanner is a consistent datastore that uses timestamps for external consistency. It's also used to store data, but the internal architecture is different.
@Gaurav sir, someone recently said in his video "People talk about consistenct hashing, and say we can create a Ring and blah blah blah, but nobody knows how to implement one, or how it has already been implemented". Can you please please please make a video on this topic and request you to cover any curently availabe impelementation.
bro i didn't understand anything due to my poor fundamental knowledge so can you please tell me what all things i need to learn to make my fundamentals strong and to become a better engineer. I really want to understand these things so that in future i can also come up with smart ideas.
No worries, try this: ua-cam.com/play/PLMCXHnjXnTnvo6alSjVkgxV-VH6EPyvoX.html This is the best begineer's playlist: interviewready.io/learn/system-design-course/building-an-ecommerce-app-1-to-1m/1-what-is-system-design
Have a look at the Google Dremel video on this channel, which is the backbone of Google Bigquery. Snowflake works for similar usecases. You can also have a look at the Apache Spark video to get an idea of different data processing systems.
Hi, thanks for reporting this. Please connect with us at surya@interviewready.io or on WhatsApp at 9920533010, we'll have this resolved as soon as possible.
![20 Whitepapers that changed the world [For Senior Software Engineers]](http://i.ytimg.com/vi/WWGM4hY34pI/mqdefault.jpg)
![20 Whitepapers that changed the world [For Senior Software Engineers]](/img/tr.png)
EDIT: Amazon supports 20 GSIs instead of the 4 mentioned in the video. Thanks to anand_ammathil who pointed this in the comments 😁
Glad to see you here again!
If you are looking for more system design content, try my course at InterviewReady: interviewready.io/learn/system-design-course
Cheers :D
I don't really understand most of the stuff you say because I am just an undergrad but all this stuff is so fascinating....I love the engineering involved in these big applications!
Cheers!
Great to see you back in action.
Please follow it up with Spanner, Cosmos DB, Aurora, and the latest AWS' DSQL. It would be of immense help.
Thanks! Do check out my playlist on research paper breakdowns 😁
I recently came across your small videos (reels). I am loving them. Apparently that's how much time and content one needs to start a conversation.
Glad you like them!
10:55 💰💰
You're a good story teller besides an excellent system design teacher
Thank you :D
Wow 😮, i used this every day in a project and fetched saved consistently but today i understood the depth 😢
Thanks a lot sirji ❤
Its a gem of a channel
Thank you!
The way you walkthrough makes it easier ❤
DynamoDb also works on pay on demamd model which cost you for the RCUs and WCUs you made, I think which is great where i don't need to bother what capacity i had to choose and irregular traffic on system.
Yes that's the option they recommend :)
This guy is the future of teaching SWE online imo, very robust against AI swarming everything
Thanks for this. It would also be interesting to see how other competing NoSQL datastores are architected.
Do checkout the research paper breakdown playlist on this channel, where we have analysed Facebook's graph and Google's timeseries databases 😁
14:15 you can have upto 20 GSIs
I just checked and you are right. Thank for reporting this. I'll make an edit in the pinned comment 😁
Few of the issues i had when working with dynamodb is how thier pricing works. They charge based on number of records scanned or written. I had a requirement where my application performs a lot of reads. Now, the problem is dynamo charges based on number of records scanned and not based on number of records matched. As long as you query based on primary key you are fine but the moment you query based on non primary attribute you are screwed, in this case dynamo essentially scans the entire table and consume a table worth of read units.
Amazing, thanks for your effort.
Thanks :D
I think a single table can allow more than 4 GSIs, beacuse i had used 6 in one of application which works on single table design.
Thanks for this great content @Gaurav ❤
Yes it can allow upto 20. I didn't remember the number correctly, so there is a correction in the pinned comment :)
Hey GKCS, I'm a newbie to sys designs and usually things just bounce off my head or I need to watch them multiple times to gain some sort of assumed sense but the data scheme part of this video was great! I understood almost everything and was enjoying it a lot.
I dunno if you did anything different for that section or it was just the visuals, but maybe try doing whatever you did more?
Thanks, cheers :D
Hey Gaurav, i got confused in caching strategy, you meant when requests rate go down cache will make sure to keep on querying the db so that when there is again increase in traffic it will be ready to handle that much load through cache only ?
This is awesome!
10:57 at this point why to keep a cache at all?
Exactly my point too. My running estimation is that they are using more servers just because they can
Great video as always, currently I'm reading a book "Design data intensive applications " I found it really interesting. Can you suggest some more Programing books for distributed systems and backend engineering
Why does almost every service has some sort of usage of s3
It's a file/object store, fundamental to distributed systems.
Google uses Colossus which has a similar function, and is used by nearly every system in their stack.
How is this compared to google spanner ?
Google spanner is a consistent datastore that uses timestamps for external consistency.
It's also used to store data, but the internal architecture is different.
Hey Gaurav, can you share where can someone find Research Papers related to Designing Systems
interviewready.io/resources
Click the checkbox for staff engineer.
Thank you
@Gaurav sir, someone recently said in his video "People talk about consistenct hashing, and say we can create a Ring and blah blah blah, but nobody knows how to implement one, or how it has already been implemented".
Can you please please please make a video on this topic and request you to cover any curently availabe impelementation.
Try InterviewReady's playlist video here:
interviewready.io/learn/system-design-course/building-an-ecommerce-app-1-to-1m/6-database-like-memory-cache-like-recall?tab=chapters
UA-cam playlists from top engineers (e.g., Gaurav Sen). --- chatgpt suggested your name when i asked for system design tips.. 😃
Cheers :D
bro i didn't understand anything due to my poor fundamental knowledge so can you please tell me what all things i need to learn to make my fundamentals strong and to become a better engineer. I really want to understand these things so that in future i can also come up with smart ideas.
No worries, try this: ua-cam.com/play/PLMCXHnjXnTnvo6alSjVkgxV-VH6EPyvoX.html
This is the best begineer's playlist: interviewready.io/learn/system-design-course/building-an-ecommerce-app-1-to-1m/1-what-is-system-design
@gkcs thanks bro 👍
How to be like you? Plz suggest.
Hey Gaurav ! Can you make a video about snowflake architecture and it is different from others
Have a look at the Google Dremel video on this channel, which is the backbone of Google Bigquery.
Snowflake works for similar usecases. You can also have a look at the Apache Spark video to get an idea of different data processing systems.
Thanks for helping the SWE community
Always!
Please fix my paid subscription
Hi, thanks for reporting this. Please connect with us at surya@interviewready.io or on WhatsApp at 9920533010, we'll have this resolved as soon as possible.
@ this is getting bounced with wrong domain name.