Any specific reason why Redis cannot be extended to support other replication formats? Does it make certain assumptions in the code that makes supporting things like leaderless replication impossible?
Hey Jordan, Thank you for the videos. Couple of questions: 1) Are the videos targeted to senior engineers? 2) Are you going to cover API GW, LB, CDNs, DNS in this series?
Hey! Would it be possible to make a video doing a brief overview on all the databases, caches and other things that make up the components to these videos. I know you have individual vids comparing things within a task but idk I think having a general grouping by task and then being able to then look at the tradeoffs within that task is neat.
@@jordanhasnolife5163 I see the googlebot has been scaled out internally and works for over the internet readings: ua-cam.com/video/jeL7NfZ62S8/v-deo.html (Thanks lol. btw if u delete the @ tag ppl don't get the notification that you responded)
Hi Jordan, I have one question. You mentioned "Redis is using single-leader replication". In terms of "single-leader model" and "master-slave model", some online references say their difference lies in "single-leader model" secures strong consistency, and "master-slave model" is more toward eventual consistency. In that sense, should we consider Redis a "master-slave model" because it is with eventual consistency?
Hi Jordan, one doubt that i'm not able to get, what do we mean by in-memory cache. 1) is it a kind of hash-map that persist on server and on restart that cache will get refreshed 2) or is it kind of external cache/db object we use in code and do the operation in DB that is seperate from the service pls help
No, redis doesn't provide ACID transactions. Not at all. What it provides instead is pipelined commands so that other clients can't run anything while the pipeline is not completed. But it's not ACID for two major reasons: 1/ ACID allows concurrent access to the database while the transaction is running, providing up to four levels of isolation for the transaction. But Redis doesn't allow concurrent access during a pipeline, and hence has only a single level of isolation, 2/ Redis doesn't provide any mechanism for data integrity: if one of the commands of a pipeline fails, (e.g. delete a key that doesn't exist, or memory is full), the pipeline stops there, it's not rolled-back to the state it was before the pipeline started, as it is with a proper ACID transaction. EDIT NOTE: by "pipeline" I mean a MULTI / EXEC commands pair.
I disagree with you here for a couple of reasons. 1) ACID makes no assumptions about concurrency. It just means that if you run a transaction, it is done atomically and serializably (no other transaction can interleave with it on the same data). While most implementations of these are multithreaded (because otherwise you'd be blocking the DB), they don't have to be. 2) Redis does allow concurrent access during a pipeline, hence it isn't a transaction. However it doesn't allow concurrent access during a MULTI/EXEC, and at least from my initial glance those seem to all succeed or fail, those satisfying atomicity and serializability.
@hasnolife5163 for 1/ you don't know ACID well enough. Read about the levels of isolations in Wikipedia and you'll see that ACID does make assumptions about concurrency, as some levels allows to see modifications made outside of the transaction and other levels completely isolate from outisde modifications. for 2/ you're right that pipeline and MULTI are not the same thing. In my job we run everything in pipelines for efficiency, so what we call "pipeline" and "transaction" are actually "MULTI/EXEC". But you're wrong about the integrity, a so-called transaction can fail. Here's an extract from the documentation about transactions: "Errors happening after EXEC instead are not handled in a special way: all the other commands will be executed even if some command fails during the transaction."
@@unperrier5998 that says after exec but what about between multi and exec? As for your second statement, acid is a made up term, in the same way that the CAP theorem is useless. What matters to me when I hear the word "transaction" is that I don't have to worry about race conditions. Execution things in an atomic and serializable manner does that. Note that following line of this doc on voltdb, which runs on a single thread: docs.voltdb.com/UsingVoltDB/ChapOverview.php#:~:text=VoltDB%20is%20a%20fully%20ACID,rollbacks%20within%20their%20own%20application.
@@jordanhasnolife5163 exec and multi go together: multi basically accumulate commands and exec runs the accumulated commands all at once. ACID is a well defined concept, just like the CAP theorem is. They're both very useful to model systems. About VoltDB, it's a different implementation than Redis: VoltDB implements a undo log.
Hey Khaal! Will try to do something like that but honestly it's a pretty similar design to tinyurl for the analytics part and pretty similar to social media for the blogging part!
Nice content Jordan! Keep this up!
studying for my resume, but dude the intro was so real I do the same thing lmao
Nothing like a good rosterbation sesh
thank you, very informative
Any specific reason why Redis cannot be extended to support other replication formats? Does it make certain assumptions in the code that makes supporting things like leaderless replication impossible?
I'm not sure if there's a specific reason for it but if you look at their official docs they only seem to mention single leader replication
Hey Jordan,
Thank you for the videos. Couple of questions:
1) Are the videos targeted to senior engineers?
2) Are you going to cover API GW, LB, CDNs, DNS in this series?
Hey Danny!
1) I think the knowledge is useful for any engineer at any level, it's frequently useful for me in my own career
2) Yes
Hey! Would it be possible to make a video doing a brief overview on all the databases, caches and other things that make up the components to these videos. I know you have individual vids comparing things within a task but idk I think having a general grouping by task and then being able to then look at the tradeoffs within that task is neat.
Hey Andrew! I already have this! ua-cam.com/video/6GebEqt6Ynk/v-deo.htmlsi=CDeQs5ZQdnLjmP7k
@@jordanhasnolife5163 I see the googlebot has been scaled out internally and works for over the internet readings: ua-cam.com/video/jeL7NfZ62S8/v-deo.html (Thanks lol. btw if u delete the @ tag ppl don't get the notification that you responded)
Good content! Is there any way to get an hold of your notes?
Hey - I have my old ones in my channel description, the new ones should be uploaded within a month or two!
Hi Jordan, I have one question. You mentioned "Redis is using single-leader replication". In terms of "single-leader model" and "master-slave model", some online references say their difference lies in "single-leader model" secures strong consistency, and "master-slave model" is more toward eventual consistency. In that sense, should we consider Redis a "master-slave model" because it is with eventual consistency?
They're the same thing
@@jordanhasnolife5163 Thank you!
Hi Jordan,
one doubt that i'm not able to get, what do we mean by in-memory cache.
1) is it a kind of hash-map that persist on server and on restart that cache will get refreshed
2) or is it kind of external cache/db object we use in code and do the operation in DB that is seperate from the service
pls help
It's number 2!
got it! thank you @@jordanhasnolife5163
No, redis doesn't provide ACID transactions. Not at all.
What it provides instead is pipelined commands so that other clients can't run anything while the pipeline is not completed.
But it's not ACID for two major reasons:
1/ ACID allows concurrent access to the database while the transaction is running, providing up to four levels of isolation for the transaction. But Redis doesn't allow concurrent access during a pipeline, and hence has only a single level of isolation,
2/ Redis doesn't provide any mechanism for data integrity: if one of the commands of a pipeline fails, (e.g. delete a key that doesn't exist, or memory is full), the pipeline stops there, it's not rolled-back to the state it was before the pipeline started, as it is with a proper ACID transaction.
EDIT NOTE: by "pipeline" I mean a MULTI / EXEC commands pair.
I disagree with you here for a couple of reasons.
1) ACID makes no assumptions about concurrency. It just means that if you run a transaction, it is done atomically and serializably (no other transaction can interleave with it on the same data). While most implementations of these are multithreaded (because otherwise you'd be blocking the DB), they don't have to be.
2) Redis does allow concurrent access during a pipeline, hence it isn't a transaction. However it doesn't allow concurrent access during a MULTI/EXEC, and at least from my initial glance those seem to all succeed or fail, those satisfying atomicity and serializability.
@hasnolife5163 for 1/ you don't know ACID well enough. Read about the levels of isolations in Wikipedia and you'll see that ACID does make assumptions about concurrency, as some levels allows to see modifications made outside of the transaction and other levels completely isolate from outisde modifications.
for 2/ you're right that pipeline and MULTI are not the same thing. In my job we run everything in pipelines for efficiency, so what we call "pipeline" and "transaction" are actually "MULTI/EXEC". But you're wrong about the integrity, a so-called transaction can fail. Here's an extract from the documentation about transactions:
"Errors happening after EXEC instead are not handled in a special way: all the other commands will be executed even if some command fails during the transaction."
@@unperrier5998 that says after exec but what about between multi and exec?
As for your second statement, acid is a made up term, in the same way that the CAP theorem is useless. What matters to me when I hear the word "transaction" is that I don't have to worry about race conditions. Execution things in an atomic and serializable manner does that.
Note that following line of this doc on voltdb, which runs on a single thread:
docs.voltdb.com/UsingVoltDB/ChapOverview.php#:~:text=VoltDB%20is%20a%20fully%20ACID,rollbacks%20within%20their%20own%20application.
@@jordanhasnolife5163 exec and multi go together: multi basically accumulate commands and exec runs the accumulated commands all at once.
ACID is a well defined concept, just like the CAP theorem is. They're both very useful to model systems.
About VoltDB, it's a different implementation than Redis: VoltDB implements a undo log.
you design systems with scalability in mind. i design systems based only on matching logo colors. we are not the same.
Gigachad mentality
Absolutely based
Hi Jordan... Can you do a Sys Design for Blogging site with analytics in mind.
Hey Khaal! Will try to do something like that but honestly it's a pretty similar design to tinyurl for the analytics part and pretty similar to social media for the blogging part!
Maybe moreso a newspaper for the blogging part, use a CDN
TC at current job?
Not enough to make girls love me
any progress on starting the feet pics or onlyfans content
Yeah I've been thinking about becoming a hand model instead
You need to type stuff or practice writing. It's just as bad or worse than a doctor
It has improved