I do not recommend Exponent. I signed up and paid for Exponent's system design course - it is not worth it. It's very generic and the material very shallow. This video is proof :)
I feel like this isn’t realistic. The interviewer doesn’t really push back with any alternatives and the interviewee just hand waves over these topics.
@@pavel5515 I’m not too worried about this point, since you can add the sharding mechanism in the load balancers of the DB servers. Or is there something else you’re worried about? Genuinely curious.
This. In an actual interview she would have interrupted him at least a dozen of times to ask him why he was choosing those specific components and data structures. I wish the interviews I've been doing were like this one.
@@stirfriedmushupork Load balancers mainly manage the request forwarding, whereas distributing the data and syncronization between the DB instances are completely different.
most of the system design was focused on storing videos which interviewer clearly mentionwd is not the focus. main requirement is around user activities and recommendation engine.
Netflix supports 6-8 bitrates for videos via the DASH protocol. They can switch bitrates every 2-5s. They keep up to 5m of video buffered in the playback device. The size of the videos varies from 256 kbps and increases by a factor of sqrt(2) for each higher bitrate video, i.e. 256 kbps - 3 mbps (1080p), or 15 mbps (4k).
2 роки тому+51
Very interesting! What I'd also talk about - sharding user metadata via region, having different PostgreSQL instances and Redis instances available in different regions serving these specific regions.
in this case this not only a db sharding problem. the host need to be grouped by different regions to serve the traffic for different traffic. otherwise if sharding db by regions while their harware stay in the same region doesn't help the network latency while introduce potential hotshot
but then, i don't know how to route th user's traffic to different regions of server effciently probably need gateway, but then users' traffic have to goes to a centered gateway server firstly.
I am not satisfied with this content. The explanations can be a bit more detailed. For example, the answer to the question "Why should we use Postgres instead of NoSQL" was not thorough. And the interviewer could ask more detailed questions like, "Why Postgres amongst other RDBMS?", "Why do you think the shard size should be around 1 to 10 TB", "How many replicas should we have and why?", "Shouldn't we use a NoSQL storing for user activity to feed the recommendation engine, because the write rate will be very high? " etc. The interview should be challenging the interviewee so that we can learn more.
I’d agree. She says “That makes sense” to a lot of answers that just ended abruptly without actually answering the question, like the one you pointed out.
The choice to pick POSTGRES for User Data was not entirely correct especially given such a system's main requierment : scalability. Netflix moved away from SQL to NOSQL a long time ago: - Consistency is not as important as availability : Netflix is not a banking application so this is a huge factor. ACID is also not as important. - Normalization is not that complex in systems like Netflix. They're not OLTP apps like banking apps. NOSQL scale immensely with static models that dont change much - If models DO requier that rare change -> SQL involves some sort of downtime. NOSQL has a distinct advantage here. (This was cited as one of the reasons for moving away from SQL)
After watching this video I read up on it and apparently instagram uses postgres for user data and they work at a very high scale. postgres can be tweaked to support sharding and duplication. however I'm still not sure how that is helpful since you won't be able to run relational queries across shards efficiently. I guess they shard it in a way that allows most queries to be on the same shards.
You say Netflix moved away, from SQL that means hundreds of smart people in netflix designed it first as an SQL, and moved after few years (if I understand correctly). So it is not bad that one smart guy designs it as SQL in 45 minutes interview.
@@BartoszRybacki0 You assume the initial decision was made at a time when NOSQL was viable and by people who actually knew how to scale systems (and knew they'd need to scale).
@@plopplop. I just say that there are many variables and many possible solutions. And I just say that it is very good design for one hour interview. And during interview the process is more important than a goal. You just want to see if the candidate thinks about scale, cost, reliability and other *ilities while choosing solution. Btw: Even Google wanted to have SQL so much that they created spanner.
Quick advise : Using CDNs to serve your premium content can compromise the security of your platform. Remember CDNs are typical for static public content that you don’t care to keep confidential and that they don’t come with authentication out of the box but you have to arrange some knobs to authenticate and authorize.
@@apurvachatterji5852 definetly, Id like to know more about the security compromises.If i understood him correct, he is trying to say that someone who is not a subscriber can see the videos in cdn. that could be definetly addressed in the design. I agree. if Im not right, or there are more compromises, please advice.
If I remember correctly there is DRM to solve that. All video content are encrypted when delivered to the user. Then there is key server that users' devices fetch the decryption key from. Only authorized user can have the key.
Honest you should be more worried about the user stealing the content then AWS or other CDN provider stealing your video content. Why risk your CDN service when you can register an account for 20$ and just watch/screen record the damn thing.
exactly a sql database is for high consistency but why would you need high consistency with netflix? Who cares if every user can see the release of a new series at the exact release time? You sacrifice high availability and speed for that? nosql is for high availability and horizontally scaled which is what you need for netflix as it needs to scale big. I'd say choosing a sql db is an immediate fail as nosql seems too obvious for me.
I believe you do not totally understand the process of a system design interview. It is not all about accurate solutions, no one expects 100% and producing grade solutions here.
Yeah, when it got to that part, I started looking at comments to see if this was an example of a passing or failing system design interview. The interviewer sounds sharp, but I'm not always following the interviewee.
I'm also confused why blob storage would've had challenges with user-generated videos (UA-cam/Facebook). He was mentioning about indexing/searching, but I got lost there. Moreover, I don't see how storing videos in RDBMS is much of an option either. Maintainability and serving of videos using RDBMS is just terrible compared to CDN and/or a file storage (preferably distributed)
Quite a beginner level system design interview, could've covered things like how to ensure fast videos globally, how to handle sudden peak when popular shows are launched, how to ensure high availability
Ya this was bad... I mean for video storage you're obviously going to have some kind of ingestion engine where you're getting massive file size originals from studios and then you're encoding probably 20+ different codecs and qualities for compatibility and then using a CDN to push popular stuff closer to users to reduce latency and improve availability. This is like what I would expect from junior level developer who probably shouldn't be designing systems anyway.
@@williambecker5811 He literally talked about a CDN in the end and just because you don't know EVERYTHING about how to build an actual movie platform doesn't mean this is a bad response. It's people like you that make it impossible to move up in software engineering. You believe if the candidate doesn't already come with a dictionary of information already in their head then they are useless. This is a horrible mentality and will only lead you to hire people who are just good and memorization as opposed to critical thinking.
@@michaelbatey5861 Sounds like you're probably just bad at system design and can't land these interviews. Nothing wrong with only hiring top talent. Companies that can attract top talent prefer false negatives over false positives.
@@williambecker5811 The guy you replied to said nothing about him being bad at system design but you made that assumption to feel good about yourself. You sounded more like a false positive to me.
@@williambecker5811 Your attitude is one of the most toxic elements of software engineering, you shouldn't have to enforce your superiority complex on others to feel good about your job
Hi Kevin! Practicing explaining your thought process to friends as you design something will be a great place to start as it gets you're comfortable with what you'll be doing in the interview. If you would like more feedback or practice your interviews, we offer peer mock interviews as well as professional coaching at Exponent, so do check us out if you are interested! Link: www.tryexponent.com
At the beginning of the interview, didn’t the interviewer specifically ask “only” for the design of a recommendation engine? Seems like the interviewee was attempting to design Netflix as a whole - considering he was talking about BLOB storage for videos, etc.
The static content is so static it could even be served from web servers, which you could distribute worldwide via CDN for faster browsing in remote countries. There is no need for transactions since videos listings are created hidden off-line and then materialized at a certain time and date, and later de-materialized at a much later date. Also when there is a popular video it can easily get 10M views in 48 hours. In this case if there is only 1 video per CDN POP you could easily overload the storage servers serving the videos - therefore - when a user goes to play a video that is in very high demand the servers for that video are near capacity, the next serving process should replicate the video to new storage in the same POP of the CDN in order to scale that particular video to meet performance demands. You can do this because most CDNs support customized programs that run in each POP to serve out the content that is requested and this code can initiate the duplication and then hand off the video playback once the duplication is well underway. Netflix started out using AWS to serve its vieeos but they quickly realized the CDN was going to determine profitability (same probably as UA-cam had until 2015 - it wasn't profitable) so they moved onto their own CDN in the late 2010's.
I don't like the approach of how this design went down tbh. I feel like it emphasized a lot on the data storage part and in the process it used so many inconvenient assumptions. I feel like the right approach would be collecting the functional requirements, then improving them with the non-functional and then work more on the replication & the horizontal scaling. I feel like the data storage part wouldn't be my first priority when answering this question (unless It's explicitly stressed upon by the interviewer.
I don't like how he structured the problem. No API descriptions and no explanation about the DB columns. Also, sharding based on users without any hashing is a bad idea since it will cause imbalances among the shards.
You are wrong. There's no imbalance if user ids are random. I think you assumed user ids are numeric and incremental. But if you are generating UUIDs, then it's more or less the same to use hashing or range based sharding.
I do not recommend Exponent. I signed up and paid for Exponent's system design course - it is not worth it. It's very generic and the material very shallow. This video is proof :)
I am blown away by the lack of details in the design part, the only decent part was the initial requirements gathering & estimation, and even for that part, there was no estimation for QPS, If I were the interviewer, for sure this candidate would be rejected.
I think the interviewee actually expect candidates to explain how to collect data for building a recommending system, as that's the core of value Netflix system provides. Everything else is just generic content distribution CDN work
One thing I'm thinking about... does caching metadata and video static data help that much with low latency? Cuz I thought the main latency comes from the video content itself so talking about video chunking and definition is important.
I think it would help with loading the initial Ui the user sees, so when they log on they instantly would see things like the top list, continue watching list, etc. But you're right in that it wouldn't help with loading the actual content. I think in this case that's okay because it would be super expensive to cache large video files, especially if they're in 4k, also this is just my assumption but users probably expect the video to take a second or two to load since they need a second to put the controller down and get comfortable. Probably what we could do instead is shard the content storage by region so that relevant content loads faster for users in the same areas, for example Korean users would likely only have access to Korean licensed content, so we can store that data separately.
Exactly this is why I came to watch this video, to see people talking about actual internals of streaming content and design a system with that specifics in mind. They ended up designing the same cliched so-called scalable backend application without event mentioning once how the actual content would travel from blob storage to user, and how would scale be handled there.
When they got to caching I don't believe it's accurate or correct, and she just continues to agree with everything he says as if it's perfect and right... Maybe you could cache some content of a previous screen. Maybe that should be client side. But caching a user's series they're watching doesn't make much sense to me in terms of it only being for that user and not knowing when the next time they may log in even. Caching something like "top 10 videos last week" that every user is seeing would be much more valuable and make more sense.
This design is pretty high level. Unfortunately, I don't think this is enough to pass a System Design interview. There were a few videos I thought went into decent depth from this channel, but this one is not it. For those who are studying System Design, I recommend checking out Pratiksha Bakrola. Her videos go in-depth in the system designs and breaking things up at a microservice level.
I would expect the interviewer to be active and ask technical questions here and there. The interviewer isn't really playing a role in the interview - which is something really important if the purpose here is to practice interviews. Feels strange.
I think he was not looking at the window which shows the "Interviewer". Thats the whole reason he was able to concentrate and focus on the system design.
When a trade-off is in the way it often means we have options. I would usually try all options and let reality reveal its truth. Otherwise, as we all probably experienced in our careers as an engineer at some stage, a hole will be created based on our "correct" assumptions. Our customers, teammates, business, and new joiners will suffer from something we could have prevented when we have the opportunity to experiment.
@19.45 when explaining the choice of Postgres it’s not clear , should discuss more about availability just easy to query shouldn’t be a closure to a very important section about choosing a database
Netflix actually encrypts videos with a time-dependent session key which expires after 24h or 48h or something I have experienced it many times in the past where you have a paused video and you try to unpause the video and it fails because the session key has expired. I have heard that they actually statically encrypt the keys and encrypt new versions every day or something like that which can be a big win for performance.
I don't think, this is a good episode. We didn't discuss about the steps required in upload and streaming actions. We didn't discuss about chunkify the videos and API designs.
Should have added the point of s3 intelligence tiering..Because not all videos gonna be accessed frequently.So less frequent videos should be put on to the storage class which is cost effective$. Because as a solution architect or software architect I think it is the primary thing to work on:)
Since this user data gathering is being done for future video recommendations to the user, don't we need to join the video meta data with user activity? We have sharded user activity DB on the basis of userID. How are we going to extract data like, say, this user likes to watch sci-fi movies, from this DB? Shouldn't we be emitting events of user activity in a stream that gets enriched with such video meta data instead of simply inserting user activity in DB like this?
it's not a bad way to start, it can use up some time and might be good to do thruput calcs as well. it's good to come up with baseline and max data expectations and you can design to those numbers.
guess it depends, for video systems, storage capacity might be a bottleneck. For e-commerce, db qps may be the bottleneck. And bottlenecks need to be quantified.
Calculating numbers (number of users, active users, number of writes, number of reads - what about in 5x, 10x peaks?) are important when it comes to scalability and can actually be helpful to determine the design. Is your system powerful enough to scale with increased users and large storage requirements? Do you need to sharding or to partition the database? Do you need more than a single cache server? Do you need a load balancer? Do you need queues? Remember that scalability matters in system design, otherwise you can design the most basic diagram and be right. Calculated numbers can help you determine how/what kind of database to use (RDBMS, No-SQL, document), caching strategies, etc. A RDBMS can reasonably have 10,000 reads per second. What happens if you require 20,000 reads per second? What do you do? What do you do if the caching server exceeds its storage limit of 500GB? Numbers can help you determine design choices.
His explanation of choosing S3 over rdbms for video storage is not correct. S3 is actually highly scalable and fast (with video chunking and sophisticated indexing). It makes no sense to choose S3 just because there is a top level amount of video content. Traditional database is essentially not designed for storing binary large objects and can be very slow and expensive to scale for video content.
Is S3 an example of Blob storage though? When I google about it, there are articles which state the differences between S3 (a distributed object storage service) and a Blob store.
Hey suvajitchakrabarty! You can use S3 as a traditional blob storage, but S3 can also be used for other purposes (like the ones you have probably read in the articles)
Never did sharding on postgres, it looks terrible according to documentation. It requires the creation of a partition for the table and next connecting the remote table with postgres-fdw. But what to do if we lose a connected server, how to automate linking shards?
Overall very helpful, but this felt too rehearsed. The interviewer did not challenge anything the candidate said and went along with every single assumption he made. He basically got to design a system he wanted and not what the interviewer wanted. And throughout the process, she never questioned his choices and asked a "what about this" type of follow up question. I think for authenticity and for people watching these videos to actually pass these interviews, there should be more back and forth between the candidate and interviewer.
Its quite vague..they don't discuss access pattern or any other specific attribute why we prefer A over B amd just say that this makes sense. Well..it doesn't make sense to me
great video but I don't particularly understand the video metadata cache. What would that schema look like/what would actually be cached here? You mentioned, for a given user, you would cache their most frequently watched video... so would the cache be on the userId level?
feels weird, why would somebody watching video update the static content here ? can we do that in netflix or hulu or whatever. we can like and dislike but caching is useless there
Excellent job!!! One thought - it would have been much more cleaner to look it these different components were put under the respective serving layer (OSI). And also, options to consider, pluses, minuses etc. Thoughts???
I have one doubt, how do we handle Streaming issues. Like user internet quality drops to a certain level. how in runtime we can solve that issue? Could anybody recommend some content which talks about this issue?
Store the videos in a DFS with SFTP to allow for CDNs for higher availability/lower latency? I'm hearing some say this would compromise security though....
Hey MrRetroboyish! Storing videos in a Distributed File System (DFS) and utilizing a Content Delivery Network (CDN) can significantly improve availability and reduce latency. However, you are right in stating that this could introduce extra security vulnerabilities. By implementing stringent access controls, encryption for data at rest and in transit, and opting for reputable CDN providers with strong security features (to name a few), we can mitigate such vulnerabilities. Hope this helps!
it was great! But I would have preferred NoSQL document based database (MongoDB), for User data/metadata, using Postgres would be a poor choice, and if we want some analytics to be done then Cassandra.
Sort of unrelated to the video, but does anyone know what app they are using to jot down notes and create basic blocks to denote services? During a remote interview, are we allowed to use any app? I've had to previously use things like a Google Drive Word file, and it was very painful.. lol
By PM if you mean Product Manager. Then for tha role product related interviews happen not system design. Exponent already has PM role specific mock interviews, do check them
Well yes.... its a system design job role so you are expected to atleast be able to give a *generic* figure as to what the system requiremnts are. Sure you cant be accurate but good/experienced engineers should be able to home in on the magnitude of data that you are working with. If you are worried about this aspect, brush up on the main data tyoe sizes (chars, int, longs etx) and please feel free to being a caluclator if you need 😉
Yes - system design interviews care about scalability and trade-offs. Why choose X over Y? If we didn't care about reasonable estimates, then you could just say you design a system with a database that has 1000TB storage that allows 1000000 writes per second. This is just silly. You have to keep the system design reasonably realistic and enough to be able to scale if writes/reads and storage increased by 10X. Personally, calculating numbers can make it easier to determine how you'll design the system. if you calculate you'll have 5000 reads per second based on activity, then a RDBMS is a good choice and if you only need 5GB of storage, you probably don't care about sharding or partitioning the database. With these interviews, it looks poor on you if you try to ignore scalability - such as if you say "Let's say we have 100 active users at any given time", that's a terrible number as it requires just a basic design. You should use large reasonable numbers such as millions of users.
Guy's thinking process is not bad, but his skill of drawing the diagrams is very poor... From where the original call comes, where goes, what does is not shown on the diagram. Chache for some reason calls APIs etc...
All these mock interviews on this channel are so useless for anyone preparing for a system design interview In most cases it feels like the interviewees are L3s who read a couple of articles on system design and throw a bunch of terms without knowing what they mean In most cases what you can learn from these videos is what NOT to do lol
Can you improve the quality of your videos? It's 2023. The video quality is unacceptable. There are tools that let users in a live stream record their portion of the video, and upload it online. The sound quality is also sub par. I understand if you're a small time UA-camr doing this for the benefit of users, that's fine, but you're trying to make money off of this thing. Do better.
Don't leave your system design interview to chance. Sign up for Exponent's system design interview course today: bit.ly/3PfSq3B
I do not recommend Exponent. I signed up and paid for Exponent's system design course - it is not worth it. It's very generic and the material very shallow. This video is proof :)
I feel like this isn’t realistic. The interviewer doesn’t really push back with any alternatives and the interviewee just hand waves over these topics.
@@pavel5515 I’m not too worried about this point, since you can add the sharding mechanism in the load balancers of the DB servers. Or is there something else you’re worried about? Genuinely curious.
This. In an actual interview she would have interrupted him at least a dozen of times to ask him why he was choosing those specific components and data structures. I wish the interviews I've been doing were like this one.
Not of highest quality, but appreciate the info.
@@stirfriedmushupork Load balancers mainly manage the request forwarding, whereas distributing the data and syncronization between the DB instances are completely different.
True, interviewee sais we need SQL because of complex queries and interviewer just accepts it. What complex queries? can you give any example?
most of the system design was focused on storing videos which interviewer clearly mentionwd is not the focus. main requirement is around user activities and recommendation engine.
Netflix supports 6-8 bitrates for videos via the DASH protocol. They can switch bitrates every 2-5s. They keep up to 5m of video buffered in the playback device. The size of the videos varies from 256 kbps and increases by a factor of sqrt(2) for each higher bitrate video, i.e. 256 kbps - 3 mbps (1080p), or 15 mbps (4k).
Very interesting! What I'd also talk about - sharding user metadata via region, having different PostgreSQL instances and Redis instances available in different regions serving these specific regions.
in this case this not only a db sharding problem. the host need to be grouped by different regions to serve the traffic for different traffic. otherwise if sharding db by regions while their harware stay in the same region doesn't help the network latency while introduce potential hotshot
but then, i don't know how to route th user's traffic to different regions of server effciently probably need gateway, but then users' traffic have to goes to a centered gateway server firstly.
@@jasusyb2734 Good point! Setting DNS geographically should help.
I am not satisfied with this content. The explanations can be a bit more detailed. For example, the answer to the question "Why should we use Postgres instead of NoSQL" was not thorough. And the interviewer could ask more detailed questions like, "Why Postgres amongst other RDBMS?", "Why do you think the shard size should be around 1 to 10 TB", "How many replicas should we have and why?", "Shouldn't we use a NoSQL storing for user activity to feed the recommendation engine, because the write rate will be very high? " etc.
The interview should be challenging the interviewee so that we can learn more.
User event data should be included in this design and recommendation should be based on that. The interview went sideways.
Yeah, I was falling asleep to this design.
I’d agree. She says “That makes sense” to a lot of answers that just ended abruptly without actually answering the question, like the one you pointed out.
Yeah I think this video isn't very good - compared to the Design a Parking Garage video which was good.
I agree. I was expecting more on this.
The choice to pick POSTGRES for User Data was not entirely correct especially given such a system's main requierment : scalability. Netflix moved away from SQL to NOSQL a long time ago:
- Consistency is not as important as availability : Netflix is not a banking application so this is a huge factor. ACID is also not as important.
- Normalization is not that complex in systems like Netflix. They're not OLTP apps like banking apps. NOSQL scale immensely with static models that dont change much
- If models DO requier that rare change -> SQL involves some sort of downtime. NOSQL has a distinct advantage here. (This was cited as one of the reasons for moving away from SQL)
Good points. I also wonder why he went with sql vs non sql (actually used)
After watching this video I read up on it and apparently instagram uses postgres for user data and they work at a very high scale. postgres can be tweaked to support sharding and duplication. however I'm still not sure how that is helpful since you won't be able to run relational queries across shards efficiently. I guess they shard it in a way that allows most queries to be on the same shards.
You say Netflix moved away, from SQL that means hundreds of smart people in netflix designed it first as an SQL, and moved after few years (if I understand correctly). So it is not bad that one smart guy designs it as SQL in 45 minutes interview.
@@BartoszRybacki0 You assume the initial decision was made at a time when NOSQL was viable and by people who actually knew how to scale systems (and knew they'd need to scale).
@@plopplop. I just say that there are many variables and many possible solutions. And I just say that it is very good design for one hour interview. And during interview the process is more important than a goal. You just want to see if the candidate thinks about scale, cost, reliability and other *ilities while choosing solution.
Btw: Even Google wanted to have SQL so much that they created spanner.
Quick advise : Using CDNs to serve your premium content can compromise the security of your platform. Remember CDNs are typical for static public content that you don’t care to keep confidential and that they don’t come with authentication out of the box but you have to arrange some knobs to authenticate and authorize.
could you speak to a bit more detail or pointers about what tools are commonly used for auth to some cdn content I want to keep confidential
I was under the impression that CDNs rather mitigate some kinds of security concerns..
@@apurvachatterji5852 definetly, Id like to know more about the security compromises.If i understood him correct, he is trying to say that someone who is not a subscriber can see the videos in cdn. that could be definetly addressed in the design. I agree. if Im not right, or there are more compromises, please advice.
If I remember correctly there is DRM to solve that. All video content are encrypted when delivered to the user. Then there is key server that users' devices fetch the decryption key from. Only authorized user can have the key.
Honest you should be more worried about the user stealing the content then AWS or other CDN provider stealing your video content. Why risk your CDN service when you can register an account for 20$ and just watch/screen record the damn thing.
NoSQL would have been perfect for this use case…the user metadata does not always need to be consistent, it can be eventually consistent.
exactly a sql database is for high consistency but why would you need high consistency with netflix? Who cares if every user can see the release of a new series at the exact release time? You sacrifice high availability and speed for that? nosql is for high availability and horizontally scaled which is what you need for netflix as it needs to scale big. I'd say choosing a sql db is an immediate fail as nosql seems too obvious for me.
I believe you do not totally understand the process of a system design interview. It is not all about accurate solutions, no one expects 100% and producing grade solutions here.
completely missed : user watches for 30 minutes, stops and logs off, comes back after few days and want to resume from where he left off :)
I found the reasoning for using a blob store vs. any source very weak. He could have said it's a distributed storage system.
Yeah, when it got to that part, I started looking at comments to see if this was an example of a passing or failing system design interview. The interviewer sounds sharp, but I'm not always following the interviewee.
Why would Blob store have problem with increasing content? That's the part that was not clear.
I'm also confused why blob storage would've had challenges with user-generated videos (UA-cam/Facebook). He was mentioning about indexing/searching, but I got lost there. Moreover, I don't see how storing videos in RDBMS is much of an option either. Maintainability and serving of videos using RDBMS is just terrible compared to CDN and/or a file storage (preferably distributed)
I didn't bother finishing the video once he started talking about RDBMS for video storage. RDBMS would not have even come across my mind.
@@foreverergouzi you should have he said cloud file storage for actual video and something else for meta data not actual video on RDBMS
Quite a beginner level system design interview, could've covered things like how to ensure fast videos globally, how to handle sudden peak when popular shows are launched, how to ensure high availability
Ya this was bad... I mean for video storage you're obviously going to have some kind of ingestion engine where you're getting massive file size originals from studios and then you're encoding probably 20+ different codecs and qualities for compatibility and then using a CDN to push popular stuff closer to users to reduce latency and improve availability. This is like what I would expect from junior level developer who probably shouldn't be designing systems anyway.
@@williambecker5811 He literally talked about a CDN in the end and just because you don't know EVERYTHING about how to build an actual movie platform doesn't mean this is a bad response. It's people like you that make it impossible to move up in software engineering. You believe if the candidate doesn't already come with a dictionary of information already in their head then they are useless. This is a horrible mentality and will only lead you to hire people who are just good and memorization as opposed to critical thinking.
@@michaelbatey5861 Sounds like you're probably just bad at system design and can't land these interviews. Nothing wrong with only hiring top talent. Companies that can attract top talent prefer false negatives over false positives.
@@williambecker5811 The guy you replied to said nothing about him being bad at system design but you made that assumption to feel good about yourself. You sounded more like a false positive to me.
@@williambecker5811 Your attitude is one of the most toxic elements of software engineering, you shouldn't have to enforce your superiority complex on others to feel good about your job
The static video data can be placed on a CDN too.
Heck it could even potentially be part of the file headers/metadata.
How is this interviewee so calm and not nervous?
Hi Kevin! Practicing explaining your thought process to friends as you design something will be a great place to start as it gets you're comfortable with what you'll be doing in the interview. If you would like more feedback or practice your interviews, we offer peer mock interviews as well as professional coaching at Exponent, so do check us out if you are interested!
Link: www.tryexponent.com
At the beginning of the interview, didn’t the interviewer specifically ask “only” for the design of a recommendation engine?
Seems like the interviewee was attempting to design Netflix as a whole - considering he was talking about BLOB storage for videos, etc.
The interview just agrees on everything, how it's an interview mocking? Could use my cat for this role 🐱
The static content is so static it could even be served from web servers, which you could distribute worldwide via CDN for faster browsing in remote countries. There is no need for transactions since videos listings are created hidden off-line and then materialized at a certain time and date, and later de-materialized at a much later date. Also when there is a popular video it can easily get 10M views in 48 hours. In this case if there is only 1 video per CDN POP you could easily overload the storage servers serving the videos - therefore - when a user goes to play a video that is in very high demand the servers for that video are near capacity, the next serving process should replicate the video to new storage in the same POP of the CDN in order to scale that particular video to meet performance demands. You can do this because most CDNs support customized programs that run in each POP to serve out the content that is requested and this code can initiate the duplication and then hand off the video playback once the duplication is well underway. Netflix started out using AWS to serve its vieeos but they quickly realized the CDN was going to determine profitability (same probably as UA-cam had until 2015 - it wasn't profitable) so they moved onto their own CDN in the late 2010's.
Great interview, but the unmissable positive energy exuding from both participants of this interview is what really did it for me.
Exactly.
I don't like the approach of how this design went down tbh. I feel like it emphasized a lot on the data storage part and in the process it used so many inconvenient assumptions. I feel like the right approach would be collecting the functional requirements, then improving them with the non-functional and then work more on the replication & the horizontal scaling.
I feel like the data storage part wouldn't be my first priority when answering this question (unless It's explicitly stressed upon by the interviewer.
I don't like how he structured the problem. No API descriptions and no explanation about the DB columns. Also, sharding based on users without any hashing is a bad idea since it will cause imbalances among the shards.
Same
You are wrong. There's no imbalance if user ids are random. I think you assumed user ids are numeric and incremental. But if you are generating UUIDs, then it's more or less the same to use hashing or range based sharding.
I do not recommend Exponent. I signed up and paid for Exponent's system design course - it is not worth it. It's very generic and the material very shallow. This video is proof :)
Good to see some free content. Most of the channels have commercialised everything so much that learning is now just for rich
x1.25 is the perfect speed for this video
Or even 2x lol
I am blown away by the lack of details in the design part, the only decent part was the initial requirements gathering & estimation, and even for that part, there was no estimation for QPS, If I were the interviewer, for sure this candidate would be rejected.
You killed this tho, really enjoyed hearing the thought process surrounding the data storage
It's called envelope estimation.
I think the interviewee actually expect candidates to explain how to collect data for building a recommending system, as that's the core of value Netflix system provides. Everything else is just generic content distribution CDN work
One thing I'm thinking about... does caching metadata and video static data help that much with low latency? Cuz I thought the main latency comes from the video content itself so talking about video chunking and definition is important.
I think it would help with loading the initial Ui the user sees, so when they log on they instantly would see things like the top list, continue watching list, etc. But you're right in that it wouldn't help with loading the actual content. I think in this case that's okay because it would be super expensive to cache large video files, especially if they're in 4k, also this is just my assumption but users probably expect the video to take a second or two to load since they need a second to put the controller down and get comfortable. Probably what we could do instead is shard the content storage by region so that relevant content loads faster for users in the same areas, for example Korean users would likely only have access to Korean licensed content, so we can store that data separately.
Exactly this is why I came to watch this video, to see people talking about actual internals of streaming content and design a system with that specifics in mind. They ended up designing the same cliched so-called scalable backend application without event mentioning once how the actual content would travel from blob storage to user, and how would scale be handled there.
When they got to caching I don't believe it's accurate or correct, and she just continues to agree with everything he says as if it's perfect and right...
Maybe you could cache some content of a previous screen. Maybe that should be client side. But caching a user's series they're watching doesn't make much sense to me in terms of it only being for that user and not knowing when the next time they may log in even. Caching something like "top 10 videos last week" that every user is seeing would be much more valuable and make more sense.
This design is pretty high level. Unfortunately, I don't think this is enough to pass a System Design interview. There were a few videos I thought went into decent depth from this channel, but this one is not it. For those who are studying System Design, I recommend checking out Pratiksha Bakrola. Her videos go in-depth in the system designs and breaking things up at a microservice level.
I would expect the interviewer to be active and ask technical questions here and there. The interviewer isn't really playing a role in the interview - which is something really important if the purpose here is to practice interviews. Feels strange.
It's really amazing place to learn system design.. Thank you Exponent.!!!
I think this was great, especially the calculation aspects.
isnt usermetadata 20 gb and not 20 tb?
I think he was not looking at the window which shows the "Interviewer". Thats the whole reason he was able to concentrate and focus on the system design.
What's the system/app used for story boarding in this video?
whimsical
When a trade-off is in the way it often means we have options. I would usually try all options and let reality reveal its truth. Otherwise, as we all probably experienced in our careers as an engineer at some stage, a hole will be created based on our "correct" assumptions. Our customers, teammates, business, and new joiners will suffer from something we could have prevented when we have the opportunity to experiment.
@19.45 when explaining the choice of Postgres it’s not clear , should discuss more about availability just easy to query shouldn’t be a closure to a very important section about choosing a database
I guess those calculations are mere numbers if you don’t factor in the resiliency required to support that big of a demand on your platform
Netflix actually encrypts videos with a time-dependent session key which expires after 24h or 48h or something I have experienced it many times in the past where you have a paused video and you try to unpause the video and it fails because the session key has expired. I have heard that they actually statically encrypt the keys and encrypt new versions every day or something like that which can be a big win for performance.
Hey D Gillies, thank you so much for sharing your knowledge with us and the community. Appreciate it! 🤝
Perfect example to show what you cannot do at a system design interview, totally incomplete and no sense design
User Metadata 2 000 000 * 100 KB =
1,525.88 Gigabits, not 20 TB! :)
I don't think, this is a good episode. We didn't discuss about the steps required in upload and streaming actions. We didn't discuss about chunkify the videos and API designs.
Should have added the point of s3 intelligence tiering..Because not all videos gonna be accessed frequently.So less frequent videos should be put on to the storage class which is cost effective$. Because as a solution architect or software architect I think it is the primary thing to work on:)
I have one doubt, how do we handle Streaming issues. Like user internet quality drops to a certain level. how in runtime we can solve that issue?
This is great video, it immediately motivates me to learn more. Thanks again for this great content.
By the way which tool Andros is for designing?
whimsical
@@freezefrancis Thanks
10 GB/hour standard quality, 20 Gb/hour high, conclusion in video average = 30 Gb/hour? Do I miss something in calculations?
I would not watch this video if you're planning to use it as a reference for a system design interview. It falls short in many ways
Since this user data gathering is being done for future video recommendations to the user, don't we need to join the video meta data with user activity? We have sharded user activity DB on the basis of userID. How are we going to extract data like, say, this user likes to watch sci-fi movies, from this DB? Shouldn't we be emitting events of user activity in a stream that gets enriched with such video meta data instead of simply inserting user activity in DB like this?
Is nosql make more sense for this service? I can’t imagine any reltions in the movie name and movie description and etc..
I wonder whether is it really necessary to do the data storage calculation during system design interview? Any thoughts?
it's not a bad way to start, it can use up some time and might be good to do thruput calcs as well. it's good to come up with baseline and max data expectations and you can design to those numbers.
guess it depends, for video systems, storage capacity might be a bottleneck. For e-commerce, db qps may be the bottleneck. And bottlenecks need to be quantified.
Calculating numbers (number of users, active users, number of writes, number of reads - what about in 5x, 10x peaks?) are important when it comes to scalability and can actually be helpful to determine the design. Is your system powerful enough to scale with increased users and large storage requirements? Do you need to sharding or to partition the database? Do you need more than a single cache server? Do you need a load balancer? Do you need queues? Remember that scalability matters in system design, otherwise you can design the most basic diagram and be right. Calculated numbers can help you determine how/what kind of database to use (RDBMS, No-SQL, document), caching strategies, etc. A RDBMS can reasonably have 10,000 reads per second. What happens if you require 20,000 reads per second? What do you do? What do you do if the caching server exceeds its storage limit of 500GB? Numbers can help you determine design choices.
From the cache part, whether it’s good to cache all metadata other than only hot ones? It’s only 10k movies.
user metadata definitely more than 100bytes
Extremely helpful!
Very informative. Can someone tell me which software is this where he is taking notes ?
He is using Whimsical
Blob storage is absolutely NOT good for this use case.
His explanation of choosing S3 over rdbms for video storage is not correct. S3 is actually highly scalable and fast (with video chunking and sophisticated indexing). It makes no sense to choose S3 just because there is a top level amount of video content. Traditional database is essentially not designed for storing binary large objects and can be very slow and expensive to scale for video content.
I honestly didn't understand the answer to the question "why use blob storage"
Your question is "design netflix"
🤔 🤨
Is S3 an example of Blob storage though? When I google about it, there are articles which state the differences between S3 (a distributed object storage service) and a Blob store.
Hey suvajitchakrabarty! You can use S3 as a traditional blob storage, but S3 can also be used for other purposes (like the ones you have probably read in the articles)
Thanks for the clarification!
Never did sharding on postgres, it looks terrible according to documentation.
It requires the creation of a partition for the table and next connecting the remote table with postgres-fdw.
But what to do if we lose a connected server, how to automate linking shards?
Thank you so much! It was very informative and fun!
Overall very helpful, but this felt too rehearsed. The interviewer did not challenge anything the candidate said and went along with every single assumption he made. He basically got to design a system he wanted and not what the interviewer wanted. And throughout the process, she never questioned his choices and asked a "what about this" type of follow up question. I think for authenticity and for people watching these videos to actually pass these interviews, there should be more back and forth between the candidate and interviewer.
Thanks for the feedback, frefai!
Andres needs to whisper a bit more quietly....
I can almost hear him if i turn my volume all the way up.
Its quite vague..they don't discuss access pattern or any other specific attribute why we prefer A over B amd just say that this makes sense. Well..it doesn't make sense to me
great video but I don't particularly understand the video metadata cache. What would that schema look like/what would actually be cached here? You mentioned, for a given user, you would cache their most frequently watched video... so would the cache be on the userId level?
what's the name of the whiteboard app ?
whimsical
Great Job!!!
feels weird, why would somebody watching video update the static content here ? can we do that in netflix or hulu or whatever. we can like and dislike but caching is useless there
10gb for an hour of standard def video? Off by order of magnitude.
Excellent job!!! One thought - it would have been much more cleaner to look it these different components were put under the respective serving layer (OSI). And also, options to consider, pluses, minuses etc. Thoughts???
What tool he is using to make notes and draw?
Hi riteshsinha587, the whiteboard tool used here is "Whimsical"!
I have one doubt, how do we handle Streaming issues. Like user internet quality drops to a certain level. how in runtime we can solve that issue? Could anybody recommend some content which talks about this issue?
Pretty weak performance from both sides, they both sounded like they didn't know what they were talking about.
What whiteboard tool does he use?
Hey stocky5354, it's called "Whimsical"!
Barely any feedback, no challenging during the decisions. Could be much better from a learning perspective.
Store the videos in a DFS with SFTP to allow for CDNs for higher availability/lower latency? I'm hearing some say this would compromise security though....
Hey MrRetroboyish! Storing videos in a Distributed File System (DFS) and utilizing a Content Delivery Network (CDN) can significantly improve availability and reduce latency. However, you are right in stating that this could introduce extra security vulnerabilities.
By implementing stringent access controls, encryption for data at rest and in transit, and opting for reputable CDN providers with strong security features (to name a few), we can mitigate such vulnerabilities.
Hope this helps!
What application does he use for notes and block schemes?
it was great! But I would have preferred NoSQL document based database (MongoDB), for User data/metadata, using Postgres would be a poor choice, and if we want some analytics to be done then Cassandra.
What is the name of website that being used ?
What whiteboard SaaS app is that being used in the video?
whimsical
What is this design software used by him?
The videos are very insightful. Can you please make some for personswith commerce background?
Can anyone please tell me the drawing tool used in this video?
Hey! The drawing tool being used here is “Whimsical”. They have a free and paid version so do check them out if you are interested!
what's the whiteboarding tool being used here?
Hi nj026! The whiteboard being used here is “Whimsical”. They have a free and paid version so do check them out if you are interested!
Energy level was really bad, not so engaging.
What is the whiteboard app ? looks great, thanks
it's whimsical
What is the drawing and typing tool used here for this interview?
Whimsical
1hour video should only be sveral gb, not 30gb. 😅
Sort of unrelated to the video, but does anyone know what app they are using to jot down notes and create basic blocks to denote services? During a remote interview, are we allowed to use any app? I've had to previously use things like a Google Drive Word file, and it was very painful.. lol
The app we use for everything is called Whimsical. It's free for personal use whimsical.com/
@@tryexponent Great ty!
Which white boarding tool is this?
Hi newjdm! The whiteboard being used here is “Whimsical”. They have a free and paid version so do check them out if you are interested!
Great video 👍🏻
Can we expect some data analyst interview in future?
Does System Design questions pertain to only TPMs or to PMs as well (those applying for PM roles in Tech companies)?
By PM if you mean Product Manager. Then for tha role product related interviews happen not system design. Exponent already has PM role specific mock interviews, do check them
You Guys stole the content from ALGOEXPERT - SHAME ON YOU. Even the examples like 10K hours content, math, clarification question all that was c
What web app are they using in the video?
whimsical
Am I expected to answer system design questions with these kind of calculations?
Well yes.... its a system design job role so you are expected to atleast be able to give a *generic* figure as to what the system requiremnts are. Sure you cant be accurate but good/experienced engineers should be able to home in on the magnitude of data that you are working with. If you are worried about this aspect, brush up on the main data tyoe sizes (chars, int, longs etx) and please feel free to being a caluclator if you need 😉
Yes - system design interviews care about scalability and trade-offs. Why choose X over Y? If we didn't care about reasonable estimates, then you could just say you design a system with a database that has 1000TB storage that allows 1000000 writes per second. This is just silly. You have to keep the system design reasonably realistic and enough to be able to scale if writes/reads and storage increased by 10X. Personally, calculating numbers can make it easier to determine how you'll design the system. if you calculate you'll have 5000 reads per second based on activity, then a RDBMS is a good choice and if you only need 5GB of storage, you probably don't care about sharding or partitioning the database. With these interviews, it looks poor on you if you try to ignore scalability - such as if you say "Let's say we have 100 active users at any given time", that's a terrible number as it requires just a basic design. You should use large reasonable numbers such as millions of users.
Guy's thinking process is not bad, but his skill of drawing the diagrams is very poor... From where the original call comes, where goes, what does is not shown on the diagram. Chache for some reason calls APIs etc...
what is he using to diagram?
whimsical
No kidding, sharding is difficult in no-sql comparing to rdbms??
I think what he meant was that sharding in rdbms is harder compared to straight-up using nosql
what website do you use for system design?
They use Whimsical
All these mock interviews on this channel are so useless for anyone preparing for a system design interview
In most cases it feels like the interviewees are L3s who read a couple of articles on system design and throw a bunch of terms without knowing what they mean
In most cases what you can learn from these videos is what NOT to do lol
The assumption that a Netflix user only watches 1000 videos had me laughing so hard. I’m def at like 3k
😁
You're called an outlier. Systems design assumptions are more about average user activity. I'm probably at less than 1000.
Can you improve the quality of your videos? It's 2023. The video quality is unacceptable. There are tools that let users in a live stream record their portion of the video, and upload it online. The sound quality is also sub par. I understand if you're a small time UA-camr doing this for the benefit of users, that's fine, but you're trying to make money off of this thing. Do better.