CAP Theorem - From the First Principles
Вставка
- Опубліковано 14 жов 2023
- System Design for SDE-2 and above: arpitbhayani.me/masterclass
System Design for Beginners: arpitbhayani.me/sys-design
Redis Internals: arpitbhayani.me/redis
Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters.
Sign up and get 40% off - app.codecrafters.io/join?via=...
In the video, I delved into the concepts of CAP theorem and its practical implications using examples from Google's Spanner database blog. CAP theorem, introduced by Eric Brewer, states that distributed data stores can provide only two out of three guarantees: Consistency, Availability, and Partition Tolerance. I explained the differences between consistency in ACID and CAP theorem, highlighting the trade-offs between consistency and availability during network partitions. Google Spanner aims to achieve both consistency and availability through globally distributed data management and specialized hardware like the TrueTime algorithm, challenging traditional CAP theorem assumptions.
Recommended videos and playlists
If you liked this video, you will find the following videos and playlists helpful
System Design: • PostgreSQL connection ...
Designing Microservices: • Advantages of adopting...
Database Engineering: • How nested loop, hash,...
Concurrency In-depth: • How to write efficient...
Research paper dissections: • The Google File System...
Outage Dissections: • Dissecting GitHub Outa...
Hash Table Internals: • Internal Structure of ...
Bittorrent Internals: • Introduction to BitTor...
Things you will find amusing
Knowledge Base: arpitbhayani.me/knowledge-base
Bookshelf: arpitbhayani.me/bookshelf
Papershelf: arpitbhayani.me/papershelf
Other socials
I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff.
LinkedIn: / arpitbhayani
Twitter: / arpit_bhayani
Weekly Newsletter: arpit.substack.com
Thank you for watching and supporting! it means a ton.
I am on a mission to bring out the best engineering stories from around the world and make you all fall in
love with engineering. If you resonate with this then follow along, I always keep it no-fluff. - Наука та технологія
This is legit “Asli Engineering” with top notch content quality in terms of explanation and breaking down the things right from the published paper. You’re actually raising the expectation bar high, and this is what the audience expects from a Google staff SE. Thanks for being a great educator. I hope this reaches out to you, and you keep making such videos.
"@Asli Engineering" What a great explanation on CAP. I request you upload videos on SOLID principles, their actual implementation and understanding required for a production-ready code. Many people just explain definition but never get in depth, on how these principles help in building a strong design of a large base code. Request you to create an extensive video on SOLID principles.
Great explanation. I was asked in one of the Apple interviews, "are you sure CAP theorem is true in today's scenario?" I wasn't aware of any other theorems so I told the traditional definition of CAP theorem (either AP or CP ). But the interviewer was trying to point that it is possible to achieve all three what exactly is availability or consistency or partition tolerance. He told me to read papers where it was possible to achieve all three. So, I guess I came to the right video. Had I known about spanner then, I could have mentioned about it. Nevertheless, its a learning curve for me.
Thanks for resonating 🙌 we all learn through experiences and glad you took it in a positive way. All the very best ✨
Very informative thank you Arpit.
Really good video sess! - I was reading about CAP theorem today too xD
Great video!
You should also discuss PACELC theorem which is more relevant to the new systems
I dont understand one thing. In case we choose consistency in case of partition. We are expecting a downtime. (i.e. missing availability) but then since the system is down now how it is partition tolerant?
at 6:42 - I think it is not necessary to take a downtime of entire system. We are not getting the availability as per the definition of CAP ( because some non-failing nodes are choosing not to respond to read/write requests to maintain consistency as there is a partition)
the system as whole need not have a downtime(outage). We can still have 99.9% kind of high availability by redirecting all reads to the other partition where writes are happening.
I agree. But while explaining it is easy for people to understand and comprehend the word downtime vs availability.
In the context, I meant exactly what you wrote but just to simplify the understanding and help people build intuition I used that term.
But yes. Thank you for pointing it out. This comment will definitely help people understand that you can still serve reads making your system partially available.
Thank you for considering my suggestion of taking CAP theorem as a topic
Thank you for suggesting. Hope this clears up CAP once and for all 😅
Do suggest other topics that you want me to touch upon. Happy to accomodate.
@@AsliEngineeringYou could have a session where you analyse and decode some interesting research papers
@@hrishii.02 this has been in the back of my mind for a long time. Working on something cool around this.
Thanks for the nudge.
apart from your vast knowledge one thing that impresses most is your humility , keep it up dude only sky is the limit for you , blessings for your little one 😄
Hi Arpit just a quick question may be i am totally wrong,In the context of the CAP theorem, when a system prioritizes consistency and availability (CA), it sacrifices partition tolerance. This means that in the event of a network partition, the system might not be able to guarantee the consistency of the data or the availability of the services,so this means every time system will not be consistent and available ,so it means there might be a chance that we will not be able to achieve any of this (CAP)
there is no network if you have ca. you can only have pc and pa if you have it over multiple networks
#AsliEngineering FTW
Hi Arpit, how about CQRS and Event Sourcing as a topic for this series?
Interested as well
How do you add subtitles to your video ?
is there any easy and fast way ?
I recorded this through Reverside - www.riverside.fm/?via=arpit
It is a pretty dope tool for recording and editing podcasts (although I did a single guest video) and added subtitles using that.
@@AsliEngineering do you use the paid version?
@@decadewgame9802 yes. Recently moved to paid tier. This was my first video using the paid tier.
Subtitles and text based video editing are pretty dope. The subtitles you see are part of the paid feature.
Correct me Arpit if am wrong but I think that in reaity AP systems are more present and used than any other combinations of C, A, P.
This is a generalization which might or might not be true. To be honest, this is not how you should look at databases, as in, which is large in numbers.
1. There are large number of databases out there
2. Nobody has gone and analysed all of them
But, if you are referring to a sea of NoSQL databases that have been built in the last few years, then yes. They do prioritise Availability over Consistency.
But in reality, most of them deployed within one data center so the chances of Partitioning happening is surprisingly low making them CA (for practical reasons).
But theoretically and as per their code they would have to choose C or A and a lot of new DBs chose A over C.
@@AsliEngineering thanks Arpit! Got that fact of one data centre aspect.
Bhaiyya use OBS if you are just using it to record data.This looks like apne koi Video conference application mai ye khola hai, boht blurry hai
"Software hai, fatega."
SET GLOBAL sync_binlog = 1;
Until you set this in MySQL it stays a CA system. Your video is confusing. i understood better before going through your video.
Async replicated databases are always prone to high replication lag due to partitioning. if master goes down and we switch this replica to be the master then the data is now not consistent.
Also, it takes some time for replica to become the master so the system is not available during the failover.
A highly available system has zero downtime during failover and it is typically dome with dual writes across nodes.
@@AsliEngineering Thanks for replying. I was super confused yesterday. It will definitely not be a CA system.