At 16:38, in the last point about non-relational DB towards the bottom right, read it as - "Different types of non relational DBs are used for different purposes". Thanks. Happy learning :)
In a video when she told.."these are your homework." Like a primary class student my hand was extended unconsciously to grab a piece paper to write the home-tasks on it. Later, blissfully she retraced her order like a kind-hearted class-teacher who sometimes grants much loved breaks. Bless you, madam. You have a good flair for teaching.
Hey, thank you so much for being our source of knowledge on SDE. For a self-taught dev like me it is extremely helpful and understandable source before getting into heavy books. One small requests, I'm lazying my way out of this, just watching the videos and grasping the concepts. I can already tell I wont be able to retain all the information. Do you mind making notes or ppt of your points? I don't mind paying for this course from Udemy or something if notes are available.
I started this playlist today and I am finding this to be better than a lot of standard materials that were previously suggested to me. I'll surely recommend this channel when asked. Thanks for the quality content!
Wow.. What a video... With lot of information and effective communication.. Thank you for sharing knowledge in youtube.. And the best part is you never asked for likes/shares..it will be informed decision from user.. All the best.. Keep doing like this , you ll be top youtuber..
Consistency - State of the database before and after the transaction should be maintained. This is different from the consistency in CAP theorem and it is not read consistency as you mentioned... Correct me if I'm wrong
You're right. It's different. In acid, C means before or after transactions, the database integrity will be maintained. For example, your data will be always following constraints you put in there
Hi Yogita thanks for the video, there is a minor review at 12:55 - 12:57 it should be heavy writes I think and not heavy reads while you speak. Just a suggestion 🙂
@yogita - Can you please double check your description of Column DBs. As per Martin Kleppman's book, Column DBs are mainly suitable for data warehouses where large amounts of data needs to be processed but useful information resides in only a few columns. However in your description you highlighted that Column DBs are suited for heavy writes and less reads. I am a little confused. Could you please clarify. Having said that, I am really impressed with your videos. You and your team have organized, simplified and presented the content in a highly structured way and easy to understand manner. Thank you very much for your work. I especially liked your video on consistent hashing, and also your analogy of types of DBs with types of rooms in buildings (in another video).
Thanks for pointing this out. I checked too and columnar DBs are suitable for heavy reads and now I can't trust the entire course :/ I have trust issues.
Hi Mam, 1'st of all thank you very much for all this informative video that you are creating. Really really helpful. I am from NON IT Background and still i can understand all of this. Regarding Quiz : We will use search database like S3 or Buckets Beacuse it helps in storing the Audio/Video data.
I'm eagerly waiting for the next parts in this series. I have an assignment submission this week and your videos are extremely helpful. Keep creating! I hope you can upload the remaining parts FAST though :3 Thank you
Thanks Simran. Yes we will try to increase the frequency. We just don't want to compromise with quality that's why it's taking time. All the best with your assignment 😉🙂
This series on system design I am hoping is going to be amazing and till now I found it amazing. Thanks for sharing your valuable content and knowledge with others, you are helping others to become a better Software Engineer. Can you please suggest a book on system design?
First of all,I would like to say concept wise your video is quite good.One suggestion please slow down your pace.Video may be long,but if it is good,then I don't think any body will complain.But based on my experience ,I would like to appreciate you for your splendid video.Wish you all the best for your coming video.Thanks and Good Job!!!
Thanks a lot for such detailed and articulate feedback Ajay. I am working on my speed and hope can address this soon 😃. By the time, please consider watching the lectures on reduced speed so that you don't miss anything important in there. Happy learning 😀
ap padate bht strict teacher jaise ho but end me smile bdi cute si dedete ho .... but thanks for using realy crisp and concrete videos. may be one more point. Relational DB are for faster writes and OLTP and non releation are for faste reads and OLAP.
At 12:52 for Column DBs you are saying column DBs when there's a requirement for heavy reads, at 13:35 video says Column DBs do not support huge number of reads, but they do support special kind of reads, I am a bit confused here.
@@sudocode Can u suggest how to learn system design as I have experience in machine learning and deep learning and little bit of data science in python.
Really nice video, you have explained all kinds of DB with examples, applications and Pro-Con list in just 18 minutes 👍 Regarding Quiz : I think for system like instagram where million of users are continuously posting, liking, commenting, uploading data in that case heavy write operation would be required. To read these frequent events join would be complex and slower. Also, Document Store can not be used as fixed structure is required for instagram to have organized read. I would request to talk about quiz answer in next video as it helps us to understand how to approach such questions.
Will definitely talk about quiz answers and instagram design, may be not in next video but soon enough. Make sure your subscribe to get to know about it. ;) Thanks for the good words.
Excellent explanation of DBs. Can different types of DBs be used for the same system in the backend? e.g. A device that stores user data and at the same time makes use of data sent by ioT device(fitbit) to run analytics in order to recommend a diet which dynamically changes depending upon the calories burnt. Can such a system make use of Relattional DB to store user data and Column DBs to store iOT data and changing preferences of user in the backend?
New MongoDB does have document field constraints and schema support now. Also transaction (ACID) support is available at document level (not at the fields level)
I guess Column based No-sql db is used for instagram / image sharing platform .and since there can be so many results after we open search bar , so I think search db is also used.
just try to answer quiz question please correct me if i am wrong columns DB would be the right choice because as you explained in the video that this kind db support heavy writes and special query and both the thing we need in systems like instagram. Thanks in advance waiting for the next video🙌.
The lectures are awesome and really nice chanel to improve knowledge but if you change the presentation ppts will do better to understand and visualize
Elastic search are used by some companies as their primary datastore as well. If there are not much updates for example, in a e-commerce use case, elasticsearch is sufficient. Also, elasticsearch also ensures durability. Is there any reason why some companies do not use elastic search as their primary datastore?
@@sudocode Hello Di , Thank you for the Awesome content. Learning about System Design is so much fun now, before this, I was completely lost in the ocean of so many system design videos. Thank you again!! Some days back I had an interview and in last round, interviewer asked me about Book My show system design and I was kind of blank, and Now I am slowly getting a feel, how to go ahead explaining approach, listing down requirements, talking about Database I will use, then the n tier architecture and so on. A big thank you again.
Not really. If fintech wants to store some analytical data or time series data or images, it might have to go with other data stores. Try to stereotype type of data, volume and how it will be written/read, that will give you pointers to choose right set of data stores.
@@sudocode Cool, but when I mentioned FinTech, I was talking about table that holds financial transactions (not the entire department, my mistake in framing my question).
I think your explanation of Consistency for ACID is wrong. It should be - All constraints / validations should pass for a successful transaction. It is not same as Consistency referred in CAP theorem.
A database being column based do not make it fast for read or write automatically. That's just the logical data structure a database exposed to developers, like an API. You won't expect an api being json formatted to be fast for read or write immediately, would you? There are some column based databases are fast for writes (I think that's why she made this conclusion), but they're not fast because of the logical data structure being column based. Hbase for example, a column based DB and it's fast for writes. However, the reason it is fast to write is because it is using memory buffer to store the data, and use a write ahead log that's also fast to write to avoid crashes. It flush to disk only every now and then. That's why its writes can be fast. Not because it is column based.
Thanks for sharing your knowledge. You are right. It’s the underlying storage structures that make column based or other data stores optimised for heavier reads or writes.
This content doesn't touch some of the key topics. You haven't explained the factors that should be considered while choosing a DB. People need to know concepts like CAP theorem, horizontal scaling and various tradeoffs beyond what has been explained in this video in order to make a decision on DB choice in system design.
I don't agree with the information you have shared for Column DB, a lot of people are watching and trusting you ma'am, please do a proper research before making videos. I hope you will take this feedback positively. Thank You. Column DD are not made for write intensive load because the data needs to be inserted in each column one by one.
Thanks for catching that. It can be an errata. If not, I will publish a correction. Videos are well researched but sometimes the delivery can contain errors. 🙂
blog.pythian.com/cassandra-use-cases/ Please check this link. I have explained similar use case. You can share the time stamp you think a wrong claim has been made and also a resource to back up your hypothesis. Columnar databases like cassandra support use cases where writes are very high as compared to reads.
At 16:38, in the last point about non-relational DB towards the bottom right, read it as - "Different types of non relational DBs are used for different purposes". Thanks. Happy learning :)
Mam can we use posgress json data type for unstructured data
In a video when she told.."these are your homework." Like a primary class student my hand was extended unconsciously to grab a piece paper to write the home-tasks on it.
Later, blissfully she retraced her order like a kind-hearted class-teacher who sometimes grants much loved breaks.
Bless you, madam. You have a good flair for teaching.
This video is GOLD! Save her at any cost.
Hey, thank you so much for being our source of knowledge on SDE. For a self-taught dev like me it is extremely helpful and understandable source before getting into heavy books. One small requests, I'm lazying my way out of this, just watching the videos and grasping the concepts. I can already tell I wont be able to retain all the information. Do you mind making notes or ppt of your points? I don't mind paying for this course from Udemy or something if notes are available.
I started this playlist today and I am finding this to be better than a lot of standard materials that were previously suggested to me. I'll surely recommend this channel when asked. Thanks for the quality content!
This channel is so good! thank you!
Thanks 👍
Done Thanks for sharing your knowledge
Types of non relational db
Key value store
Document db
Column db
Search db
This can be the best explanation I have seen for the differences between different kinds of databases. Awesome video.
Thanks Luis
mam you are looking so pretty in this video..
@@sudocode
Wow.. What a video... With lot of information and effective communication.. Thank you for sharing knowledge in youtube.. And the best part is you never asked for likes/shares..it will be informed decision from user.. All the best.. Keep doing like this , you ll be top youtuber..
God Has Blessed You Sister. Thank you for the videos especially for starting with absolute basics. It truly helps.😊
Consistency - State of the database before and after the transaction should be maintained.
This is different from the consistency in CAP theorem and it is not read consistency as you mentioned... Correct me if I'm wrong
You're right. It's different. In acid, C means before or after transactions, the database integrity will be maintained. For example, your data will be always following constraints you put in there
Hi Yogita thanks for the video, there is a minor review at 12:55 - 12:57 it should be heavy writes I think and not heavy reads while you speak. Just a suggestion 🙂
Thanks so much! You are perfect! Everything was explained so simply and logically, just appreciated!
Hi Yogita ,
Is this HLD ??
Can I take this video as reference to prepare HLD?
@yogita - Can you please double check your description of Column DBs. As per Martin Kleppman's book, Column DBs are mainly suitable for data warehouses where large amounts of data needs to be processed but useful information resides in only a few columns. However in your description you highlighted that Column DBs are suited for heavy writes and less reads. I am a little confused. Could you please clarify.
Having said that, I am really impressed with your videos. You and your team have organized, simplified and presented the content in a highly structured way and easy to understand manner. Thank you very much for your work. I especially liked your video on consistent hashing, and also your analogy of types of DBs with types of rooms in buildings (in another video).
Thanks for pointing this out. I checked too and columnar DBs are suitable for heavy reads and now I can't trust the entire course :/ I have trust issues.
@@versatilecraft good issue to have in fundamentals knowledge.
You have saved my life! Very clear explanation! Thank you, teacher! :)
I'm from Brazil
Hi Mam, 1'st of all thank you very much for all this informative video that you are creating. Really really helpful. I am from NON IT Background and still i can understand all of this.
Regarding Quiz : We will use search database like S3 or Buckets Beacuse it helps in storing the Audio/Video data.
where is the quiz though?
@@kushagra0347 its in the description, hard to find but its just before "Recommendations"
I'm eagerly waiting for the next parts in this series. I have an assignment submission this week and your videos are extremely helpful. Keep creating! I hope you can upload the remaining parts FAST though :3 Thank you
Thanks Simran. Yes we will try to increase the frequency. We just don't want to compromise with quality that's why it's taking time. All the best with your assignment 😉🙂
In the beginning I was NOT sure But you delivered a well balance information. Thank you.
This series on system design I am hoping is going to be amazing and till now I found it amazing. Thanks for sharing your valuable content and knowledge with others, you are helping others to become a better Software Engineer. Can you please suggest a book on system design?
One of the books is mentioned i description. Also, planning a video for the same.
First of all,I would like to say concept wise your video is quite good.One suggestion please slow down your pace.Video may be long,but if it is good,then I don't think any body will complain.But based on my experience ,I would like to appreciate you for your splendid video.Wish you all the best for your coming video.Thanks and Good Job!!!
Thanks a lot for such detailed and articulate feedback Ajay. I am working on my speed and hope can address this soon 😃. By the time, please consider watching the lectures on reduced speed so that you don't miss anything important in there. Happy learning 😀
Great explanation with a simplied manner. I request you to make another series for advanced topics on system design.
Clear and crisp explanation. Loved it!
ap padate bht strict teacher jaise ho but end me smile bdi cute si dedete ho .... but thanks for using realy crisp and concrete videos. may be one more point. Relational DB are for faster writes and OLTP and non releation are for faste reads and OLAP.
At 12:52 for Column DBs you are saying column DBs when there's a requirement for heavy reads, at 13:35 video says Column DBs do not support huge number of reads, but they do support special kind of reads, I am a bit confused here.
Loved the way u r explaining things with very simple examples which have resemblance with real world entities. Very nice teaching.Thnks a lot.
Thanks a lot for your appreciation Samiran 😊
@@sudocode Can u suggest how to learn system design as I have experience in machine learning and deep learning and little bit of data science in python.
Clean and crystal clear explaination, keep it up with this consistency.
Ur explaination is very nice and easy
Really nice video, you have explained all kinds of DB with examples, applications and Pro-Con list in just 18 minutes 👍
Regarding Quiz : I think for system like instagram where million of users are continuously posting, liking, commenting, uploading data in that case heavy write operation would be required. To read these frequent events join would be complex and slower. Also, Document Store can not be used as fixed structure is required for instagram to have organized read. I would request to talk about quiz answer in next video as it helps us to understand how to approach such questions.
Will definitely talk about quiz answers and instagram design, may be not in next video but soon enough. Make sure your subscribe to get to know about it. ;) Thanks for the good words.
Thank you 👍
I really like that this quality of content which is highly structured is freely available on the internet on System designs. #50k
Can you please revisit your statement on columnar db's not being good with reads... IN MY HUMBLE OPINION, they are better at reads than writes
Good explanation. Note: Scylla is a C++ rewrite of Cassandra.
Superb! Really appreciate this concise introduction.
Thanks Rafat 👍
nice analogy of homes and databases
Glad you liked it Soumyadip
Fantastic video - thankyou.
Clear, Concise, Correct info. Thank you and thanks youtube for enabling people like you.
This series is informative.
Excellent explanation of DBs.
Can different types of DBs be used for the same system in the backend?
e.g. A device that stores user data and at the same time makes use of data sent by ioT device(fitbit) to run analytics in order to recommend a diet which dynamically changes depending upon the calories burnt. Can such a system make use of Relattional DB to store user data and Column DBs to store iOT data and changing preferences of user in the backend?
A system may/ will different databases for different user journey
New MongoDB does have document field constraints and schema support now. Also transaction (ACID) support is available at document level (not at the fields level)
Very decent content. Really like this as a refresher course. Personally prefer the no non-sense approach to concepts. Thanks for this!
Great to hear!
Good and useful. But you could have helped us to digest the columner db how it looks like. But thanks much
I guess Column based No-sql db is used for instagram / image sharing platform .and since there can be so many results after we open search bar , so I think search db is also used.
Hello maam..!! Very impressed with your teaching style and knowledge of the subject... things are getting clearer now.....
Great explanation
Simply awesome!!
Columnar db is used for heavy read or for heavy write usecase got confused?
just try to answer quiz question please correct me if i am wrong
columns DB would be the right choice because as you explained in the video that this kind db support heavy writes and special query and both the thing we need in systems like instagram.
Thanks in advance waiting for the next video🙌.
Your video are really awesome ... subscribed your channel...Thanks
Welcome to the fam. Thanks 😊
Its great to watch thses contents. Thank u very much.
Nice explanation
Wonderful explanation 😁☺️
OMG Excellent! just like that
Excellent, thank you.
Nice job .. i am subscribing to it.
Just
Awesome, Keep it up good work Thanks a lot
The lectures are awesome and really nice chanel to improve knowledge but if you change the presentation ppts will do better to understand and visualize
your videos are really amazing!!!
Please keep posting regularly.
India needs more people like you.
Thank you, I will
Very informative.. Thanks sudoCode.. Keep uploading ✌️
Thank you so much!
what u meant by heavy read and writes in document db databases
cool video)
Does KV stores support ACID properties?
Not really.
Some of them have started supporting transactions though
you teach very well ! thanks
what database you suggest for ecommerce website like myntra???
Depends on use case. User data can be kept in relational database, logs can be kept in non relational and so on.
I thought document and key-value are same type databases .i.e. NOSQL like mongoDB. What's the difference then?
Superb videos
Thanks
Elastic search are used by some companies as their primary datastore as well. If there are not much updates for example, in a e-commerce use case, elasticsearch is sufficient. Also, elasticsearch also ensures durability. Is there any reason why some companies do not use elastic search as their primary datastore?
elastic search is expensive and is used only for searching use cases
love you ma'am 😊😊
Thanks Priyanka
in reational DB clustering is very difficult
My Answer to the Quiz
For an image sharing system like instagram, I will prefer to use a database like Amazon S3, or GCP Buckets.
That's one of the most effective ways for image storage and distribution on global scale. :)
@@sudocode Hello Di , Thank you for the Awesome content. Learning about System Design is so much fun now, before this, I was completely lost in the ocean of so many system design videos. Thank you again!! Some days back I had an interview and in last round, interviewer asked me about Book My show system design and I was kind of blank, and Now I am slowly getting a feel, how to go ahead explaining approach, listing down requirements, talking about Database I will use, then the n tier architecture and so on. A big thank you again.
Great Work !!
Hi! Did you learn this in your job or you learnt ut out of curiosity?
Both :)
can we just stereotype(at high scale) like
sql serves FinTech purposes and
no sql serves all other purposes??
Not really. If fintech wants to store some analytical data or time series data or images, it might have to go with other data stores. Try to stereotype type of data, volume and how it will be written/read, that will give you pointers to choose right set of data stores.
@@sudocode Cool, but when I mentioned FinTech, I was talking about table that holds financial transactions (not the entire department, my mistake in framing my question).
What about graph databases?
I wish u taught us in college rather than abhay katiyaar sir 😀
Please suggest mongodb Resources
Good work sister! Keep it up :)
I think your explanation of Consistency for ACID is wrong. It should be - All constraints / validations should pass for a successful transaction. It is not same as Consistency referred in CAP theorem.
A database being column based do not make it fast for read or write automatically. That's just the logical data structure a database exposed to developers, like an API. You won't expect an api being json formatted to be fast for read or write immediately, would you? There are some column based databases are fast for writes (I think that's why she made this conclusion), but they're not fast because of the logical data structure being column based.
Hbase for example, a column based DB and it's fast for writes. However, the reason it is fast to write is because it is using memory buffer to store the data, and use a write ahead log that's also fast to write to avoid crashes. It flush to disk only every now and then. That's why its writes can be fast. Not because it is column based.
Thanks for sharing your knowledge. You are right. It’s the underlying storage structures that make column based or other data stores optimised for heavier reads or writes.
neo4j can be added
Yogita - How to join your slack community, it's not getting searched by your name or sudocode
👍👍👍👍👍👍
I love the concept of u teaching in the same getup every time , u r gonna leave a long Lasting impact on your viewers 😎
Thanks for such kind words Anchal. 😀
I am 100th like
nosql looks like json basically
This content doesn't touch some of the key topics. You haven't explained the factors that should be considered while choosing a DB. People need to know concepts like CAP theorem, horizontal scaling and various tradeoffs beyond what has been explained in this video in order to make a decision on DB choice in system design.
damn she's good
I don't agree with the information you have shared for Column DB, a lot of people are watching and trusting you ma'am, please do a proper research before making videos. I hope you will take this feedback positively. Thank You. Column DD are not made for write intensive load because the data needs to be inserted in each column one by one.
Thanks for catching that. It can be an errata. If not, I will publish a correction. Videos are well researched but sometimes the delivery can contain errors. 🙂
blog.pythian.com/cassandra-use-cases/
Please check this link. I have explained similar use case. You can share the time stamp you think a wrong claim has been made and also a resource to back up your hypothesis. Columnar databases like cassandra support use cases where writes are very high as compared to reads.