5:13 How is Data Distributed - Replication vs Partitioning, I think in this you are talking about sharding not partitioning. Becuase when we talk about data split into multiple machines that is sharding.
A bit iffy on the WAL based approach (specifically from postgres's pov) mentioned here. I have the following arguments: 1. WALs can be set to any of the 3 levels which doesn't have to be very verbose. 2. it seems counter intuitive to see why WALs would pose a demerit to db replication with different versions since they were actually introduced to solve the problem of physical replication. On the same lines, upgrades also aren't an issue. Also, postgres's logical replication is a combination of WALs and logical decoding.
Databases maintain WAL. so when database comes up again it will follow this log and do the updates. after that consensus will be used to agree on latest data. conflict resolution needs to be done in case of conflicting data.
Great explanation, covers great breadth of replication issues. Good revision for folks who 've already read Designing Data Intensive Applications
A great resource indeed. Haven't watched it completely yet but depth is really awesome. Really glad I've found it. Thanks Rachit :)
Awesome..cleared many doubts and clarifications after watching this video
5:13 How is Data Distributed - Replication vs Partitioning, I think in this you are talking about sharding not partitioning. Becuase when we talk about data split into multiple machines that is sharding.
i think partioning and sharding are not completely mutually exclusive. sharding can be said to be a subset of partioning.
could you please also share the notes you've used in this video?
Thanks for sharing
@rachit would a similar (Leader/follower) architecture also follow for DB syncing across geographies ?
in which video you discussed the data stored in single machine? I am unable locate it, which video is that one?
do you have any notes available?
Does sybase replication have a synchronous/asynchronous option?
Can you explain different in WAL in LSM tree and B-trees??
Already detailed videos for both have been present in the playlist. Check video description
A bit iffy on the WAL based approach (specifically from postgres's pov) mentioned here. I have the following arguments:
1. WALs can be set to any of the 3 levels which doesn't have to be very verbose.
2. it seems counter intuitive to see why WALs would pose a demerit to db replication with different versions since they were actually introduced to solve the problem of physical replication. On the same lines, upgrades also aren't an issue.
Also, postgres's logical replication is a combination of WALs and logical decoding.
Put this video as 1st video in this playlist. You have misplaced it.
You use which programming language the most
JavaScript now
How to deal with the problem where leader goes down before replicating it's writes to the followers?
it's a huge data loss, right?
Databases maintain WAL. so when database comes up again it will follow this log and do the updates. after that consensus will be used to agree on latest data. conflict resolution needs to be done in case of conflicting data.
bhai order thoda upar neeche hai
Just read paxos or raft or zab .....
Someone's reading DDIA
This is real engineering