Let me know if you guys want more system design content - I've been wanting to do more and have a lot of ideas, so hopefully you all are interested! Let me know, and feel free to drop a like 👍 =) 🚀 neetcode.io/ - Get lifetime access to every course I ever create!
Design a system that pushes notifications with image and/or video to a mobile device using local storage only. No AWS or other external servers allowed. Once you have that done, let Eufy know, bc they f’d up big time 😂
Neetcode! I've been applying to jobs for 7 months now since my graduation and struggled to get interviews as my resume was fairly unimpressive. I kept telling myself that I just needed to make sure that I killed any interview I DID get and your channel is what helped me to do just that. Got an offer today that blew my expectations out of the water. Thanks for all you do for all of us and keep it up!
Awesome Video! Thank you for going into detail and explaining what every part of an example systems architecture is accomplishing and why it's useful. One little nitpick -> (22:00 ) you say UA-cam is using TCP which is (for quite some time now) not true anymore. If you look closely at the "client-protocol" used to send the video/audio data (you show that at 18:29 ) you will see that UA-cam is using QUIC (HTTP3) which is built on top of UDP instead of TCP. It includes most of the functionality of TCP based HTTP protocol but is technically not using TCP. Just wanted to point that out in case anyone was wondering... Great video though, keep up the neet work!
I think you've missed most points in the watching part. User can change the resolution for that we need multiple videos. Also, different OS will have different file formats like iOS, and android. so if there are 4 resolutions with 3 formats, 4*3=12. Each video should be stored as 12 different videos. Also, we need to store videos in chunks in the cloud storage. We will use main Dash File which contains all the video's information like chunk number, url to that chunk. So, if user skips to the 4th chunk, we'll check the dash file to get the url for the 4th chunk and will make the request for that particular chunk having specific resolution and format.
Bro! Genuinely such eye opening content. Thank you! I am no where near solving problems like this yet but it's really helpful getting exposed to all the realistic considerations involved in designing large scale systems.
The fact that YT isn’t streaming a data source in complete but making HTTP requests to load chunks of video/audio and taping them at the client end BLEW MY MIND !!! my whole life seems like a lie , lol Great video, Thanks mate
such beauty looking at all the different high level trade offs and considerations going into designing an application that I have been using daily for the past 15 years
These types of videos are such a blessings since it goes deep into a particular usecase or several design descisions which would be rather difficult to gain experience. I can try to learn as many frameworks as possible, do as many projects as possible, but if I dont have such real-world (or close to real-world) lessons like these I might be making mistakes that aren't so visible to the naked eyes.
seriously your explanations of complex topics are amazing, i’m so glad i found you! I was going cross eyed looking at graphs on Leetcode and their explanations were not helpful at all. Officially subscribed, thank you so so so much!
You can easily have a user service and store all of the users information including the uploader there, hence you don't need to have it stored alongside the video but just have a userId as reference. This way you have decoupled your systems and follow a more domain driven design and if the user data changes you don't need to go through 1000 of videos just to update a profile picture. At the beginning of every system design interview/document you'd better write down your domains in order to see what are the different services you need before jumping into the design.
2:14- Reliablity is defined as ability of a system to work as per specification inspite of extenal (high loads) or internal (faults) factors. If our non-functional requirement is videos to not disappear after upload , we are essentially looking for the system to be durable (not reliable).
If a video takes 1 minute to encode, a worker can encode about 60*24 = 1500 videos a day. 50 million divided by 1500 is about 35000. You're making the math much more complicated than it needs to be in the section on the encoding workers.
On the profile picture updating async, you could always solve that by referencing a latest.png and then only updating the object store to change latest.png to reference to the newest image and hence the changes would propagate much faster than updating 1000x video documents
For things like the user profile photo, I wouldn't think you'd store that in each video document. The document should contain a reference to the profile photo. Each user can only have one profile photo, so this is fine.
Putting huge video files in a queue is too much and very costly and slow but instead you should have it in S3 and then just populate an event to Kafka/RabbitMQ/etc which is consumed by the encoding service that in turn knows it has to fetch video with id: XYZ and start encoding it.
@@vyshnavramesh9305 You can utilize multiple threads and splitting them into chunks, however putting video files into a messaging queue is still a bad practice and using a technology for wrong reasons where it doesn't fit. Message brokers are not there to transfer BLOBs.
There’s a lot of issues in this video. Durability instead of reliability, no explanation for how videos get from the object store to the CDN, NoSQL not being the best option for metadata, caching videos at the app server being a bad idea in general when you have a CDN, apparently saving profile picture URLs in the video metadata instead of just making a separate request for user data, etc.
Although normalisation isn't expected in NoSQL, it's expected for you to normalize at a collection level in document databases like Mongo. It just makes it easy to deal with points you mentioned, like profile pictures and what not
The only video on youtube which actually assumes that "Design UA-cam" is a fresh question that was ever thrown at you in an interview and you followed a top-down approach to present a possible design.
16:35 Lesson learned: If you have demonstrated you know your stuff, saying "I don't know" actually makes you seem more competent than trying to sneak past addressing something.
For the NoSQL user profile picture topic. Instead updating the picture in every document could it not make sense to store profile picture as a userid.jpg and overwriting this userid.jpg with changes hence the reference always pointing to the same destination and thus never requiring updating of references in multiple documents in case of picture updating?
Awesome video ! Your explaination is easy to understand and it's detailed enough for a video like this. Would be great if you do more videos about system design
Even in sport live streaming, to the end users, they are mostly over TCP. TCP gets your congestion control, error code checking, and guarantee ordering delivery. With live stream, sure it sucks when you are 10 seconds behind or when you see the "10x speed up" or "fuzzy screen" when there is a slow network, but ultimately as a viewer we still want to see most of the live video to be played. The client side can throw error when there is a consistent connection error, or skip ahead when it finds itself way behind. With UDP you really don't get those benefits. TCP reusing connection also helps. In practice, almost no one will use udp. There are other proprietary protocols out there, but most of them depend on tcp.
Once a video is encoded and stored in the object storage (encoded), does the original video need to remain on the raw object storage? If it can be removed, then what is the use case of the raw object storage? Can raw object storage be removed from the design such that uploaded videos go directly to the message queue? Also, I thought sharding can only be implemented for NoSQL databases. If SQL databases can implement sharding, then they can also horizontally scale and I don't see a downside to using them over NoSQL databses?
Message queues aren't generally optimised for holding large blobs of binary data like that - a raw video could easily be 1GB+. Also it would probably have to be placed in the queue in a single operation, while object stores usually offer multipart upload. The queue would probably just hold metadata like videoId, timestamp, videoType etc.
System design interviews seem to be totally phony. Looks like they might have originated at Google or something and then all other companies started copying this (humans arent smart so they just go with the flow; thats how human culture works). No one ever does system design in their software engineering jobs. There's always teams of people doing this for months. Whatever we can come up in 30 minutes is absolutely useless (was UA-cam really designed in 30 minutes by a single person? no), so whats the point of testing something that is useless and something that we will never use in our real jobs? It makes ZERO sense. If something requires some kind of GRE/SAT preparation for something is not practical knowledge and people are trying to "train" for something that is just not used in their real jobs, thats another red flag. If they're trying to test intelligence or problem solving, they should stick to those kinds of questions. This system design stuff is just total BS.
Considering they get most of the information from previous assessments, problem solving and technical rounds, this is the place where one can have a true SWE to SWE conversation. What you mentioned about how products are being ideated over weeks or months is true, and that's the point of system design interviews. Your idea will never become an actual product, but rather, this is a good way to test the real-time problem solving ability of a candidate. In theory, there are no wrong answers in system design interviews. Your solution might be flawless or it could have different flaws, but just the fact that you're able to adjust to changing micro-requirements and continue building is a good sign of a promising candidate.
It's not about coming up with the ideal design. It's more about seeing if the interviewer understands various concepts and technologies + can communicate them well.
Hi thanks for the content. One question if in the real interview you where are exactly this question and the answer was the content of the video, how would have the candidate do on the interview?
Hey there! Thanks for the awesome video, just one question, wouldn't be better to send to the queue, encode and then store, so we only store the encoded video instead of the raw footage? At least in this case where we do not need the raw video anytime after
11:00 if a user updates his profile picture you don’t need to update it anywhere is the video metadata iff you have chose to name the profile picture as [uid].jpg in this manner you will always have the current profile picture
Ok, Hi, now I dont know if you may or may not reply to this, but I finally finished watching this video and would like to appreciate how informative this whole thing is. Now I do have a some questions if thats ok (I deeply apologize). So for starters, 50m uploads a day, is that true? Also, why divide by 100,000? You're also saying that within a min, that the encoder may need time to "encode" one batch of vids, but more may keep coming and coming and our need for more encoders increases? Does it keep growing or does it stop at some point. Now, you're also saying that the actual video is stored within a object store while its meta-data (like title, description, user) is within a database (could it be either sql or nosql)? Are the database/object store within their own server? also, could you define in layman's terms what is a app server, queue, cdn, and load balancer is and mention where they might be? Furthermore, why exactly do we need cache here? Lastly, you said that when we skip ahead within a YT vid, we're making http requests to load more chunks, right? If thats the case, does the same thing occur when we just let the vid play like normal? As I mentioned, I hope you do reply and I have to say how very insightful this whole thing is. Many Thanks.
Hey can’t we use two db here ? to tackle the problem where you said if it you upload a video and you don’t get to see the newest video . I think we can use two db here nosql and sql , sql we can use for only reads and implement caching on the sql db
The justification for using NoSQL was kind of lacking.Relational database do just fine with high read throughput. I don't think it's necessarily a bad choice, I'd say either SQL or NoSQL would be fine.
Hello Neet. I have a question. in the object oriented design interview course of yours do you have any roadmap planned like what future case studies you will be covering??
@NeetCode Thank you so much for such great contents! Could you please create a system design video on how to design a feature flag/toggle system similar to CloudBees or GitLab ones.
hi, Thanks, amazing video. seems like the they writing into the DB should be done only (or also) after the video is ready for watch - that would be after the encoder finished. no?
I have a question people.. Why do we have to update the videos collection when the user updates a profile picture? Since the videos are referenced with the user via the id..and if the user updates their profile picture, only that user document should be updated? And since the video collection has the user via the id..and the next time a request for a video goes in, it will fetch the user via id and the updated profile picture will already be there in the users document. Why have you updated all the videos documents of that user asynchronously.. isn't this a bad practice..can anyone explain to me..I think I am missing something here..
@neetcode, any plans of launching pricing based on PPP so that it becomes consecutively affordable across all countries as well apart from US? Tried to send mail as well for the same, got no response
Yes. We really like your videos and content. But purchasing power difference is only making me to stop buying the pro. As a just grad student it's Ton of money other countries.
Neetcode - why we decided to use NOSQL for storing metadata rather than a relational DB and also how can we handle the use case you briefly touched upon where we don't want to upload the data again in case of a network partition during file upload?
Nosql used since uploads are frequent mandating sharding. But I don't think Mongodb is suitable to UA-cam since it isn't good for Availability. There is a downtime when the master is down to elect the next master from slaves. I would prefer Cassandra due to its high availability (leaderless). But Cassandra isn't good when there are read queries on multiple attributes. Hence I would couple Cassandra with Elasticsearch inturn bringing in Fuzzysearch. If it were Netflix, I would have sticked onto a simple Mysql since writes are not frequent and provides enough querying. If fuzzy search required, I would still couple it with ES. (Would never use an ES as a primary db since unreliable).
I made a observation that when you reply to a comment on UA-cam. By default the person's name is added in input and it is highlighted. But when we type the persons name it is not highlighted and considered as text. Does someone know how this functionality works. P.S - I'm trying this things out on UA-cam phone app.
Hello! Thanks for the videos, they are very helpful and very good explanations and questions. 11:04, you talk about a load balancer, this was confusing for me, because the diagram is more for architectural design than application design (microservices, bff, kubernetes), is the LB mentioned as a BFF that manages authentication and requests, or is Kubernetes LB more part of microservices? Thank you!
Is it really that better to use NoSQL and have a need to update potentially unlimitted number of records instead of having one user record and update it? I don't see how it is improvement over using relational DB. Yeah, it avoids join but I don't think that it is a big performance hit.
Yes NoSQL is a bad idea here. The joins are better to get all user data when needed, rewrites are actually somewhat frequent if you actually see how users make immediate changes multiple times on upload, the metadata content fields are going to constantly change and NoSQL could be a very expensive mass update vs SQL, storing profile pic urls in the video metadata is strange when a join could very easily solve this issue, it’s also potentially a many to many relationship… SQL just better here.
The one thing that stumps me that gets asked all the time are things about updating the table of videos needed to be processed without losing the queue of videos that are to be processed. Any resources about improving the reliability of the encoding processor pipeline?
I'm not sure where you got the 100k, but the actual number you'd be looking for is 86400 - 24 * 60 * 60. But that doesn't make for easy math. Just saying though.
What's up with this NoSQL scales and SQL doesn't scale? I know those interviews are weird, but I can't say something like that even if that's required to pass...
Let me know if you guys want more system design content - I've been wanting to do more and have a lot of ideas, so hopefully you all are interested! Let me know, and feel free to drop a like 👍 =)
🚀 neetcode.io/ - Get lifetime access to every course I ever create!
Design a system that pushes notifications with image and/or video to a mobile device using local storage only. No AWS or other external servers allowed. Once you have that done, let Eufy know, bc they f’d up big time 😂
Thanks for the video! Could you recommend any good resources for learning System Design at a level like this?
Thanks again
More systems design content sounds great and if you want to do it, then you should!
Commented this on an old video but the api for neetcode is not working. I am unable to sign in with both Google and github.
GIVE ME MORE
Thank you Mr beast for system design 😊
Neetcode! I've been applying to jobs for 7 months now since my graduation and struggled to get interviews as my resume was fairly unimpressive. I kept telling myself that I just needed to make sure that I killed any interview I DID get and your channel is what helped me to do just that. Got an offer today that blew my expectations out of the water. Thanks for all you do for all of us and keep it up!
Congratulations, I'm really happy for you! 🎉
are u fired
Hello Alex, what company you got the offer from??
@@VY-zt3ph Infineon Technologies, a German semiconductor company.
@@aelam02 u live in Germany brother??
Awesome Video! Thank you for going into detail and explaining what every part of an example systems architecture is accomplishing and why it's useful. One little nitpick -> (22:00 ) you say UA-cam is using TCP which is (for quite some time now) not true anymore. If you look closely at the "client-protocol" used to send the video/audio data (you show that at 18:29 ) you will see that UA-cam is using QUIC (HTTP3) which is built on top of UDP instead of TCP. It includes most of the functionality of TCP based HTTP protocol but is technically not using TCP.
Just wanted to point that out in case anyone was wondering...
Great video though, keep up the neet work!
I think you've missed most points in the watching part. User can change the resolution for that we need multiple videos. Also, different OS will have different file formats like iOS, and android. so if there are 4 resolutions with 3 formats, 4*3=12. Each video should be stored as 12 different videos. Also, we need to store videos in chunks in the cloud storage. We will use main Dash File which contains all the video's information like chunk number, url to that chunk. So, if user skips to the 4th chunk, we'll check the dash file to get the url for the 4th chunk and will make the request for that particular chunk having specific resolution and format.
We want more system design video...I love all your videos ❤️
Bro! Genuinely such eye opening content. Thank you! I am no where near solving problems like this yet but it's really helpful getting exposed to all the realistic considerations involved in designing large scale systems.
The fact that YT isn’t streaming a data source in complete but making HTTP requests to load chunks of video/audio and taping them at the client end
BLEW MY MIND !!! my whole life seems like a lie , lol
Great video, Thanks mate
such beauty looking at all the different high level trade offs and considerations going into designing an application that I have been using daily for the past 15 years
First video recently that I haven't skipped for one second. Great work!
These types of videos are such a blessings since it goes deep into a particular usecase or several design descisions which would be rather difficult to gain experience. I can try to learn as many frameworks as possible, do as many projects as possible, but if I dont have such real-world (or close to real-world) lessons like these I might be making mistakes that aren't so visible to the naked eyes.
💯
Yes, we need it. Hope to have other content appear in the course, such as machine learning, data analysis.
seriously your explanations of complex topics are amazing, i’m so glad i found you! I was going cross eyed looking at graphs on Leetcode and their explanations were not helpful at all. Officially subscribed, thank you so so so much!
You can easily have a user service and store all of the users information including the uploader there, hence you don't need to have it stored alongside the video but just have a userId as reference. This way you have decoupled your systems and follow a more domain driven design and if the user data changes you don't need to go through 1000 of videos just to update a profile picture. At the beginning of every system design interview/document you'd better write down your domains in order to see what are the different services you need before jumping into the design.
Yes, it can be cached as well.
i would prefer a user system as well
Getting into system design and one of the best vids I've seen so far, great in depth detail
2:14- Reliablity is defined as ability of a system to work as per specification inspite of extenal (high loads) or internal (faults) factors. If our non-functional requirement is videos to not disappear after upload , we are essentially looking for the system to be durable (not reliable).
good point
If a video takes 1 minute to encode, a worker can encode about 60*24 = 1500 videos a day. 50 million divided by 1500 is about 35000. You're making the math much more complicated than it needs to be in the section on the encoding workers.
On the profile picture updating async, you could always solve that by referencing a latest.png and then only updating the object store to change latest.png to reference to the newest image and hence the changes would propagate much faster than updating 1000x video documents
For things like the user profile photo, I wouldn't think you'd store that in each video document. The document should contain a reference to the profile photo. Each user can only have one profile photo, so this is fine.
hasn't the author said that? profile picture is stored in S3, the DB just references it🤔
First i thought why is MrBeast is teaching system design lmaooo
Putting huge video files in a queue is too much and very costly and slow but instead you should have it in S3 and then just populate an event to Kafka/RabbitMQ/etc which is consumed by the encoding service that in turn knows it has to fetch video with id: XYZ and start encoding it.
What if the file is fed into a pipeline and split into chunks in the first step and so parallely processed?
@@vyshnavramesh9305 You can utilize multiple threads and splitting them into chunks, however putting video files into a messaging queue is still a bad practice and using a technology for wrong reasons where it doesn't fit. Message brokers are not there to transfer BLOBs.
There’s a lot of issues in this video. Durability instead of reliability, no explanation for how videos get from the object store to the CDN, NoSQL not being the best option for metadata, caching videos at the app server being a bad idea in general when you have a CDN, apparently saving profile picture URLs in the video metadata instead of just making a separate request for user data, etc.
Although normalisation isn't expected in NoSQL, it's expected for you to normalize at a collection level in document databases like Mongo. It just makes it easy to deal with points you mentioned, like profile pictures and what not
The only video on youtube which actually assumes that "Design UA-cam" is a fresh question that was ever thrown at you in an interview and you followed a top-down approach to present a possible design.
you can scale num of encoding workers based on queue backlog length
16:35 Lesson learned: If you have demonstrated you know your stuff, saying "I don't know" actually makes you seem more competent than trying to sneak past addressing something.
FYI - Vitess is also used by Slack. Great throughput and amazing latency!
For the NoSQL user profile picture topic. Instead updating the picture in every document could it not make sense to store profile picture as a userid.jpg and overwriting this userid.jpg with changes hence the reference always pointing to the same destination and thus never requiring updating of references in multiple documents in case of picture updating?
Awesome video ! Your explaination is easy to understand and it's detailed enough for a video like this. Would be great if you do more videos about system design
Even in sport live streaming, to the end users, they are mostly over TCP. TCP gets your congestion control, error code checking, and guarantee ordering delivery. With live stream, sure it sucks when you are 10 seconds behind or when you see the "10x speed up" or "fuzzy screen" when there is a slow network, but ultimately as a viewer we still want to see most of the live video to be played. The client side can throw error when there is a consistent connection error, or skip ahead when it finds itself way behind. With UDP you really don't get those benefits. TCP reusing connection also helps. In practice, almost no one will use udp. There are other proprietary protocols out there, but most of them depend on tcp.
Please more system design content.🙇
At split second I thought that mr Beast is actually a software engineer
Once a video is encoded and stored in the object storage (encoded), does the original video need to remain on the raw object storage? If it can be removed, then what is the use case of the raw object storage? Can raw object storage be removed from the design such that uploaded videos go directly to the message queue?
Also, I thought sharding can only be implemented for NoSQL databases. If SQL databases can implement sharding, then they can also horizontally scale and I don't see a downside to using them over NoSQL databses?
Message queues aren't generally optimised for holding large blobs of binary data like that - a raw video could easily be 1GB+. Also it would probably have to be placed in the queue in a single operation, while object stores usually offer multipart upload.
The queue would probably just hold metadata like videoId, timestamp, videoType etc.
System design interviews seem to be totally phony. Looks like they might have originated at Google or something and then all other companies started copying this (humans arent smart so they just go with the flow; thats how human culture works). No one ever does system design in their software engineering jobs. There's always teams of people doing this for months. Whatever we can come up in 30 minutes is absolutely useless (was UA-cam really designed in 30 minutes by a single person? no), so whats the point of testing something that is useless and something that we will never use in our real jobs? It makes ZERO sense. If something requires some kind of GRE/SAT preparation for something is not practical knowledge and people are trying to "train" for something that is just not used in their real jobs, thats another red flag. If they're trying to test intelligence or problem solving, they should stick to those kinds of questions. This system design stuff is just total BS.
It's a fun conversation starter though
Considering they get most of the information from previous assessments, problem solving and technical rounds, this is the place where one can have a true SWE to SWE conversation.
What you mentioned about how products are being ideated over weeks or months is true, and that's the point of system design interviews. Your idea will never become an actual product, but rather, this is a good way to test the real-time problem solving ability of a candidate. In theory, there are no wrong answers in system design interviews. Your solution might be flawless or it could have different flaws, but just the fact that you're able to adjust to changing micro-requirements and continue building is a good sign of a promising candidate.
You cant design anything in 30 mins it is way to check how do people think and how much they can provide context and how they deal with trade off
It's not about coming up with the ideal design. It's more about seeing if the interviewer understands various concepts and technologies + can communicate them well.
It's definitely a skill you must acquire outside of real life software engineering. Almost like studying for an exam
Why dont we skip the first object storage? Like the app server directly adds the video to the queue?
Very helpful. Interesting details about UA-cam using MySQL with Vitess
Thx bro, have only few hours before SD interview, vary usefull)
amazing explanations! thank you so much for your time
Hi Neetcode. Thank you for your work. Would you mind sharing/telling us what whiteboarding app you use please? Thank you!
i was abt to ask same thing! Great vid!
Hi thanks for the content. One question if in the real interview you where are exactly this question and the answer was the content of the video, how would have the candidate do on the interview?
Hey there! Thanks for the awesome video, just one question, wouldn't be better to send to the queue, encode and then store, so we only store the encoded video instead of the raw footage? At least in this case where we do not need the raw video anytime after
How does youtube handle the quality control of a video?
11:00 if a user updates his profile picture you don’t need to update it anywhere is the video metadata iff you have chose to name the profile picture as [uid].jpg in this manner you will always have the current profile picture
Since we have to favor availability over consistency, should we use some other database like Cassandra or DynamoDB than MongoDB?
Ok, Hi, now I dont know if you may or may not reply to this, but I finally finished watching this video and would like to appreciate how informative this whole thing is. Now I do have a some questions if thats ok (I deeply apologize). So for starters, 50m uploads a day, is that true? Also, why divide by 100,000? You're also saying that within a min, that the encoder may need time to "encode" one batch of vids, but more may keep coming and coming and our need for more encoders increases? Does it keep growing or does it stop at some point. Now, you're also saying that the actual video is stored within a object store while its meta-data (like title, description, user) is within a database (could it be either sql or nosql)? Are the database/object store within their own server? also, could you define in layman's terms what is a app server, queue, cdn, and load balancer is and mention where they might be? Furthermore, why exactly do we need cache here? Lastly, you said that when we skip ahead within a YT vid, we're making http requests to load more chunks, right? If thats the case, does the same thing occur when we just let the vid play like normal? As I mentioned, I hope you do reply and I have to say how very insightful this whole thing is. Many Thanks.
You definitely killing it! Love this content!
Hey can’t we use two db here ? to tackle the problem where you said if it you upload a video and you don’t get to see the newest video .
I think we can use two db here nosql and sql , sql we can use for only reads and implement caching on the sql db
this is awesome!!
btw, what app/website are you using to write this notes?
The justification for using NoSQL was kind of lacking.Relational database do just fine with high read throughput. I don't think it's necessarily a bad choice, I'd say either SQL or NoSQL would be fine.
Hello Neet. I have a question. in the object oriented design interview course of yours do you have any roadmap planned like what future case studies you will be covering??
Hi, thank you so much for sharing amazing contents!
Hi NC, please what tools do you use for making your videos?
Hardware and Software tools
Why would you use in-memory over distributed caching for your LRO?
I can't wrap my head around that choice.
What tool/software do you use for writing/drawing in your videos?
@NeetCode Thank you so much for such great contents! Could you please create a system design video on how to design a feature flag/toggle system similar to CloudBees or GitLab ones.
Finally an english tutorial
nice thank you so much, thanks for sharing the knowledge
Love it! Design Leetcode next please :P
Can we ask for more of a system design stuff from you? Please!
Loved the conciseness of the video
You are a great teacher. Thanks!
hi, Thanks, amazing video.
seems like the they writing into the DB should be done only (or also) after the video is ready for watch - that would be after the encoder finished.
no?
Thanks for amazing design throughout
Should you in a design interview cover how many app servers you might need based on the DAU
I first thought Mr.Beast going to teach us youtube system
But then I read the channel name 😅
I have a question people..
Why do we have to update the videos collection when the user updates a profile picture?
Since the videos are referenced with the user via the id..and if the user updates their profile picture, only that user document should be updated?
And since the video collection has the user via the id..and the next time a request for a video goes in, it will fetch the user via id and the updated profile picture will already be there in the users document.
Why have you updated all the videos documents of that user asynchronously.. isn't this a bad practice..can anyone explain to me..I think I am missing something here..
UA-cam just suggested me this video!
More sys design! Thanks!
ce faci frt
Stau tu
10:34 Why do we need to update all video documents if we just updated user image. User id will still stay the same
@neetcode, any plans of launching pricing based on PPP so that it becomes consecutively affordable across all countries as well apart from US? Tried to send mail as well for the same, got no response
Sorry, I might've missed your email - feel free to send it again.
Yes. We really like your videos and content. But purchasing power difference is only making me to stop buying the pro. As a just grad student it's Ton of money other countries.
@@sarathchandrareddy3033 Feel free to email me!
@@NeetCode Here in India, its a significant amount of money when converted into Indian Rupees. I really like your videos.
Neetcode - why we decided to use NOSQL for storing metadata rather than a relational DB and also how can we handle the use case you briefly touched upon where we don't want to upload the data again in case of a network partition during file upload?
Nosql used since uploads are frequent mandating sharding. But I don't think Mongodb is suitable to UA-cam since it isn't good for Availability. There is a downtime when the master is down to elect the next master from slaves. I would prefer Cassandra due to its high availability (leaderless). But Cassandra isn't good when there are read queries on multiple attributes. Hence I would couple Cassandra with Elasticsearch inturn bringing in Fuzzysearch.
If it were Netflix, I would have sticked onto a simple Mysql since writes are not frequent and provides enough querying. If fuzzy search required, I would still couple it with ES. (Would never use an ES as a primary db since unreliable).
I made a observation that when you reply to a comment on UA-cam. By default the person's name is added in input and it is highlighted. But when we type the persons name it is not highlighted and considered as text. Does someone know how this functionality works.
P.S - I'm trying this things out on UA-cam phone app.
Why would you use an LRU cache when the most viewed videos are the most recent? You'd want the opposite of that.
LRU means the least-recently used will be the first to be evicted - which I think is what you're also saying, unless I'm misunderstanding?
I guess, video upload should be 5million instead of 50 million. Ratio is 1:100 for uploading and watching users are 5 billion
Mr. Beast in the thumbnail OP!
Great video. I like the high level stuff.
Great video, thank you!
Hello! Thanks for the videos, they are very helpful and very good explanations and questions. 11:04, you talk about a load balancer, this was confusing for me, because the diagram is more for architectural design than application design (microservices, bff, kubernetes), is the LB mentioned as a BFF that manages authentication and requests, or is Kubernetes LB more part of microservices? Thank you!
Is it really that better to use NoSQL and have a need to update potentially unlimitted number of records instead of having one user record and update it? I don't see how it is improvement over using relational DB. Yeah, it avoids join but I don't think that it is a big performance hit.
Yes NoSQL is a bad idea here. The joins are better to get all user data when needed, rewrites are actually somewhat frequent if you actually see how users make immediate changes multiple times on upload, the metadata content fields are going to constantly change and NoSQL could be a very expensive mass update vs SQL, storing profile pic urls in the video metadata is strange when a join could very easily solve this issue, it’s also potentially a many to many relationship… SQL just better here.
The one thing that stumps me that gets asked all the time are things about updating the table of videos needed to be processed without losing the queue of videos that are to be processed. Any resources about improving the reliability of the encoding processor pipeline?
You can use distributed message queue with persistence feature like Apache Kafka.
Could you pls design a notification system and ad click event generation system.
Why no arrows from metadata to encoded object stores? How metadata go to CDN?
10:57 what if we store the only video metadata in the nosql while other data in sql?
your very knowledgeable man.. you just inspire me to keep studying tbh . gonna buy the course soon thanks for the video bro 👌🏽
First comment, lots of love and respect. I solved lots of questions just from your explanations.
Why'd he pick MongoDB? He didn't really make a distinction about why a NoSQL DB is advantageous.
what books do I need to read for this any suggestions? what exactly is the name of the topic i need to research?
so queue is a place for raw videos waiting their turn to be encoded by the encoding service?
Where is the API gateway in relation to all this... Specifically in relation to the CDN?
I'm not sure where you got the 100k, but the actual number you'd be looking for is 86400 - 24 * 60 * 60. But that doesn't make for easy math. Just saying though.
What's up with this NoSQL scales and SQL doesn't scale? I know those interviews are weird, but I can't say something like that even if that's required to pass...
Awesome content ❤
Why are you using noSQL and not SQL?
Appropriate thumbnail as Mr Beast dominates YT
Very informative!
so Mr. Beast was a software engineer so amazed by youtube architecture that he started youtubing
Bro was using an ad blocker on UA-cam. 😎
Great Video!
Why is mr. Beast on the thumbnail? 😂 great vid btw
very good video