Hi, Firstly I would like to thank you for creating such great videos on System designs. I have just one thing to ask that I can't the reference links in your description for couple of videos. Can you please share across those links, will help me to dig deep in the topics you covered. Thanks in advance
This is great content, thank you so much. could you please do more content on db replication and faults and failure handling like the previous video in the series.
I was searching all over the internet for System design videos that are basic and which I can easily understand. This series helped me a lot in understanding the basic concepts. Nicely explained. Now I'm thinking about why I didn't start searching a little early 🙂. Please add the reference links in your description. I can't find them. Thanks again Yogita.
Hello Yogita, Great informative video as always but was really looking forwards to learn more about the different replication models which you were touching upon briefly throughout the video. You seem to stress a lot on exploring the additional resource links in the description of the video. But unfortunately, they seem to have been missed somehow. This has been the case in past few of the videos too. Could you please update the descriptions of video to have those links added? Thanks in advance.
@sudoCODE, Yogita, thanks for the detailed explanation. Could you please update the links as you mentioned, couldn't find them in the description box. Thank u.
i want to know more on data replication engine,i want you to make a complete vedio for data replication project,how it works....the way you teach is greate. thanks
Hi Yogita, I really liked the series, especially the real-world analogies given at the start. However, I would request you to please add the resources in the description, as they are missing in many videos.
I understand very easily by the way you teach. So please make a in depth video of database replication as it becomes very important when data transaction is very large.
Short and sweet introduction to replication. Though you mention it is short information of big topic, in replication if primary goes down then one of secondary become primary and in case of asynchronous how to make sure that secondary selected out of consistent only.
Hi Yogita, Thanks for providing such good and clear content! I was looking for the links you said (in video) to be their in description but cannot find any here. Is there someother resource i need to check other than the courses ?
I don't know but whenever I see your explanation it feels like my big sister is teaching me.. Although I don't have one 😐... Amazing content can't imagine it's for free..
Hey, I am a bit confused about synchronous replication and how it solves consistency issues in case of high latency between the replicas. You explained that the write is initiated on the master and is broadcasted to all the replicas. The replicas execute the write command and send back acknowledgments. What happens if there is a read during this process? It is quite possible that a read request on the master is going to be different from the read request to one of the replicas. Do we accept that inconsistency? In a write and read-heavy system, this will happen a non-trivial number of times.
i think till the master sends acknowledgement to the writer service for synchronised replication for example updating a order, other services like for example invoice service won't enquire about that particular transaction even if it accessing the inconsistent replicas. They might want to enquire about already consistent transactions. Hope this makes sense may be..
my notes below: Replication: Pros - Fault tolerance, secondary mains, latency reduction, cqrs, Performance improvement, availability. Cons - dissynchronization(replication lag) -> Data inconsistency. Solution for data consistency Read after write consistency To be continued.... #SR7
I was confused with aerospike db for implementing a ratelimiter algorithm to store and access atomic counters in faster way, as it has many read write polices between master and slaves. Finally this video clears my doubts to understand such systems.
hello sudoCode, as you explain very welll but still want to know more details the things, please attach some online linkes or materials for every topics from where we can get the full/details context.... Thanks In Advance :)
Thanks a lot for the nice explanation. Shared the channel link to the coding community sites to acquire the greater number of audience for such an informative topics of system design. Pls do post video on system design example - facebook feed, google serarch, instagram working , uber architecture. Thanks
Hi Yogita, while we do understand the different methods to update the replicas , be it hybrid , sync or async , in case of a transaction driven app , as you said , can be banking , which of these has to be used since I see a glitch in all three in terms of data consistency for example Hybrid - R1 is updated and main DB says write complete , user is accessing R2 which means user wouldn’t get the updated data . So isn’t it like it should first update the replica which user is accessing and all others later ?
Thanks for this amazing video ❤ 🙏 i have a doubt when I insert bulk data in million then slave started 100% cpu utilisation and due to the same my server -read stop working till the operation completed. Pls suggest how I can handle this situation? Thanks in advance 🙏
Nice! So database system will take care of copying data to slaves, it is matter of configuration on database system level....what about routing read n writing request to the respective db node,would that b implemented on the application level via some gateway to filter n send to respective db i.e., read or write???eagerly curious :-))
I liked your video. The explanation is simple and easy to understand but can you please add the links that you mention in your video? I cannot find them
Greate content thanks a lot. But I have a doubt that in all the replication methods (sync or async or hybrid) If one of the replications (write in R1, R2, R3) write fails (suppose in R3 write fails) then how to manage that situation. So in this case Data inconsistency problem might occur. Correct me if I am wrong. Thanks again for producing beautiful content.
Nice explanation!! Also, it would be great if explain a bit about multi-leader replication, and how to handle conflicts in that case. Please keep on making videos!! Thanks for sharing your knowledge ma'am!!
the way you explained synchronous replication feels incomplete. To me, if the replicas are serving the reads (as described a minute before.) then there is a time when some of the replicas have done the write and sent their ack, and others have replicas have not yet done so. At that point the replicas which have not yet acknowledged will be reporting the old value, and those that have acknowledged should return the new one. I suspect there is another round of of packets to circulate to confirm (commit) that the write has succeeded, and only after the second round trip should the master answer that the write has succeeded.
hi really great explanation, it will be nice if you add subtitle for people like me who rly hard to understand spoken English (I'm not native English speaker), it'll really help me, thank you in advance :)
how all replica DBs will be locked for read operation when master is updating using synchronous approach ? that time also read can happen and reader can get wrong data if update is not completed from master. ?
Hi Yogita, nice videos very informative - keep it up! I don't see links to additional resources in description from past few videos - Can you please check it once? I would love to read/explore more on these topics using the links you have in mind. Thanks :)
Hi Yogita, thanks for sharing these wonderful videos. it is really helping us to understand the concepts. One Question though - in every video you mentioned the link for additional content but i never found it. Generally it has details about past and future video, link about your linkedin and hangout... i am not sure if i am looking at the right place. Please suggest.
Hey Yogita, Can you please explain what happens in case of synchronous replication where there is data exchange from both sides at the same time? example: data variable x is updated to 1 at time t1 on primary DB, and at the same time t1,data variable x is updated to 2 on secondary DB. How does DB handle these situations? do database engineers keep a check on race conditions while configuring the replica setup?
This is a classic concurrent event case. In distributed systems there is no way to conclude the exact ordering of concurrent events, so it depends on the consensus algorithm used to reach and agree on a result.
hello sudoCode, as you explain very welll but still want to know more details the things, please attach some online linkes or materials for every topics from where we can get the full/details context.... Thanks In Advance :)
Hello peeps! Hope you like the video. Apologies about the traffic noise in the start. Was recording this late at night in my home town in India. :)
No problem at all.... This video was really helpful and I sincerely encourage you you to please make more videos like these.Thanks a ton!!
Hi, Firstly I would like to thank you for creating such great videos on System designs. I have just one thing to ask that I can't the reference links in your description for couple of videos. Can you please share across those links, will help me to dig deep in the topics you covered. Thanks in advance
This is great content, thank you so much. could you please do more content on db replication and faults and failure handling like the previous video in the series.
Can anyone help me how i can actually implement it, i hve understood the concept but to to implement
I was searching all over the internet for System design videos that are basic and which I can easily understand. This series helped me a lot in understanding the basic concepts. Nicely explained. Now I'm thinking about why I didn't start searching a little early 🙂. Please add the reference links in your description. I can't find them. Thanks again Yogita.
Conceptually you are the best teacher mam...
Thanks Mridul
Hello Yogita,
Great informative video as always but was really looking forwards to learn more about the different replication models which you were touching upon briefly throughout the video. You seem to stress a lot on exploring the additional resource links in the description of the video. But unfortunately, they seem to have been missed somehow. This has been the case in past few of the videos too.
Could you please update the descriptions of video to have those links added?
Thanks in advance.
loving this video series - finding it very helpful!
@sudoCODE, Yogita, thanks for the detailed explanation. Could you please update the links as you mentioned, couldn't find them in the description box. Thank u.
i want to know more on data replication engine,i want you to make a complete vedio for data replication project,how it works....the way you teach is greate.
thanks
Hi Yogita, I really liked the series, especially the real-world analogies given at the start. However, I would request you to please add the resources in the description, as they are missing in many videos.
Sure 😊
Thank you so much, I've watch couple of videos but you are the one that makes me get it!
Thank you so much it is really helpful.
Thank you
I understand very easily by the way you teach. So please make a in depth video of database replication as it becomes very important when data transaction is very large.
Thank you so much 👍🏼🎉⭐🙏
Thanks for the very helpful video.
Really good content mam
Thanks Kalyan
Short and sweet introduction to replication. Though you mention it is short information of big topic, in replication if primary goes down then one of secondary become primary and in case of asynchronous how to make sure that secondary selected out of consistent only.
That depends on implementation level.
You are going great job.
Please make more detailed video database replication .
Sure I will
Hi Yogita, Thanks for providing such good and clear content! I was looking for the links you said (in video) to be their in description but cannot find any here. Is there someother resource i need to check other than the courses ?
+1
I don't know but whenever I see your explanation it feels like my big sister is teaching me.. Although I don't have one 😐... Amazing content can't imagine it's for free..
This made my day 💙
Very good videos. Thanks for the content.
Very useful video..thanks a lot!
Glad it was helpful!
Awesome one, Thanks for sharing.
thanks for knowledge sharing...😊
Hey, I am a bit confused about synchronous replication and how it solves consistency issues in case of high latency between the replicas. You explained that the write is initiated on the master and is broadcasted to all the replicas. The replicas execute the write command and send back acknowledgments. What happens if there is a read during this process? It is quite possible that a read request on the master is going to be different from the read request to one of the replicas. Do we accept that inconsistency? In a write and read-heavy system, this will happen a non-trivial number of times.
i think till the master sends acknowledgement to the writer service for synchronised replication for example updating a order, other services like for example invoice service won't enquire about that particular transaction even if it accessing the inconsistent replicas. They might want to enquire about already consistent transactions. Hope this makes sense may be..
a very good explanation.
Awesome video. Thanks you very much!!
my notes below:
Replication:
Pros - Fault tolerance, secondary mains, latency reduction, cqrs, Performance improvement, availability.
Cons - dissynchronization(replication lag) -> Data inconsistency.
Solution for data consistency
Read after write consistency
To be continued....
#SR7
Superbly made ❤
Very good video with wonderful explanation. ✨✨
I was confused with aerospike db for implementing a ratelimiter algorithm to store and access atomic counters in faster way, as it has many read write polices between master and slaves. Finally this video clears my doubts to understand such systems.
U forgot to add or mention about the latency topic in any video. Pls check and make a video on that as well.
Please explain about eventually consistent
Very insightful mam
Thanks a lot
hello sudoCode, as you explain very welll but still want to know more details the things, please attach some online linkes or materials for every topics from where we can get the full/details context.... Thanks In Advance :)
Thanks a lot for the nice explanation. Shared the channel link to the coding community sites to acquire the greater number of audience for such an informative topics of system design. Pls do post video on system design example
- facebook feed, google serarch, instagram working , uber architecture.
Thanks
That's very kind of you! Thanks a ton.
Clear concept
Hi Yogita, while we do understand the different methods to update the replicas , be it hybrid , sync or async , in case of a transaction driven app , as you said , can be banking , which of these has to be used since I see a glitch in all three in terms of data consistency for example Hybrid - R1 is updated and main DB says write complete , user is accessing R2 which means user wouldn’t get the updated data .
So isn’t it like it should first update the replica which user is accessing and all others later ?
Nice Ma'am!
Yes please, replication primer 👍
No resources in the description
Nice explanation.
Thanks
Thank you very much
hi, thanks for the video Yogita :)
, one doubt is whether the snapshots are also distributed across all the nodes or specific to a node
Thanks for this amazing video ❤ 🙏 i have a doubt when I insert bulk data in million then slave started 100% cpu utilisation and due to the same my server -read stop working till the operation completed. Pls suggest how I can handle this situation?
Thanks in advance 🙏
Neat explanation
Thank you 🌹
Awesome content.
Thanks
Nice! So database system will take care of copying data to slaves, it is matter of configuration on database system level....what about routing read n writing request to the respective db node,would that b implemented on the application level via some gateway to filter n send to respective db i.e., read or write???eagerly curious :-))
That's a great video. Would love to see a lesson on multi-master replication, and how to handle conflicts in those cases.
thank you..😊😊
Thanks for this !!
Thank you so much!
I liked your video. The explanation is simple and easy to understand but can you please add the links that you mention in your video? I cannot find them
Greate content thanks a lot.
But I have a doubt that in all the replication methods (sync or async or hybrid) If one of the replications (write in R1, R2, R3) write fails (suppose in R3 write fails) then how to manage that situation. So in this case Data inconsistency problem might occur.
Correct me if I am wrong.
Thanks again for producing beautiful content.
Nice explanation!!
Also, it would be great if explain a bit about multi-leader replication, and how to handle conflicts in that case.
Please keep on making videos!!
Thanks for sharing your knowledge ma'am!!
Thanka a lot Rupesh. Yes I will definitely make video on leader election, consensus, paxos etc.
the way you explained synchronous replication feels incomplete. To me, if the replicas are serving the reads (as described a minute before.) then there is a time when some of the replicas have done the write and sent their ack, and others have replicas have not yet done so. At that point the replicas which have not yet acknowledged will be reporting the old value, and those that have acknowledged should return the new one.
I suspect there is another round of of packets to circulate to confirm (commit) that the write has succeeded, and only after the second round trip should the master answer that the write has succeeded.
love the video
hi really great explanation, it will be nice if you add subtitle for people like me who rly hard to understand spoken English (I'm not native English speaker), it'll really help me, thank you in advance :)
Very good video ! Thank you
Glad you liked it!
couldn't be more perfect!
how all replica DBs will be locked for read operation when master is updating using synchronous approach ? that time also read can happen and reader can get wrong data if update is not completed from master. ?
great vid
Mam you always say "links in description" for further read. I couldn't find anything in the description.
Awesome content ☺️
Thanks Shani 😊
Streaming is like broadcasting all database in cloud and accessing is available all time as soon as new data uploaded
Excellent Explanation. Keep it up 👍
Great video! Can you make a video on data models? For example Relational/Document/Graph/wide-column? Their applicability, use-cases and examples?
Guess what? I already did :) ua-cam.com/video/O_c7lzNbcKo/v-deo.html
Big ups
Nice video !! A replication primer for sure :)
Hehe. Thanks
Hi, I like the video I am unable to find resources in the description.
Thanks
Nice👌🏻
Excellent !! I dont see any references or links in the description as you mentioned...
let me check the links.
what is row based replication & statement based replication ? Please explain.
Hi Yogita, nice videos very informative - keep it up!
I don't see links to additional resources in description from past few videos - Can you please check it once? I would love to read/explore more on these topics using the links you have in mind. Thanks :)
Will check and try to update.
@@sudocode Hi Yogita, do you have the additional resources? In many videos, they are not there in the description. Thank you for the great content!
You have not added resources in the description. I only see paid courses websites
Please try to create a database replication course.
Can’t promise. Will do if there’s a lot of ask for it.
Hi Yogita, thanks for sharing these wonderful videos. it is really helping us to understand the concepts. One Question though - in every video you mentioned the link for additional content but i never found it. Generally it has details about past and future video, link about your linkedin and hangout... i am not sure if i am looking at the right place. Please suggest.
Hey I am getting same problem . Have you found it
@@AnkurKumar-vx3xv no
What will happen if Replica fail to update? What will be the best approach to handle this ?
There could have been an overall slide for this video on strategy, like you have done for caching video
Hi.. I could not find the link to the docs u mentioned? Could u pls share that
I'm confused between this and CQRS .
Another great Video! :)
Thanks again!
so can you please tell me more about table replica
I'm thinking of created an application service that will act as a db read cache. But pass thought writes.
In synchronous replication only one of replica is in sync with primary db, correct me if wrong.
It depends on the implementation. All replicas can be first synced before committing writes.
Hey Yogita, Can you please explain what happens in case of synchronous replication where there is data exchange from both sides at the same time?
example: data variable x is updated to 1 at time t1 on primary DB, and at the same time t1,data variable x is updated to 2 on secondary DB. How does DB handle these situations? do database engineers keep a check on race conditions while configuring the replica setup?
This is a classic concurrent event case. In distributed systems there is no way to conclude the exact ordering of concurrent events, so it depends on the consensus algorithm used to reach and agree on a result.
If we also able to demo this theories
I am trying to build demos.
Ma'am please share notes of these lecture.
Will try!
I can hear the traffic in the background xD
hmm the location of Ireland seems a bit off..
Yeah. Sorry for that Van 😑
Master slave is a wrong terminology, leader worker is the right one
where is link , are you getting lazy
sorry no hates but she looks like Github LOGO
Thank you it’s an honour 😃
hello sudoCode, as you explain very welll but still want to know more details the things, please attach some online linkes or materials for every topics from where we can get the full/details context.... Thanks In Advance :)