As a hiring manager, here is one tweak I'd add to this interview. When asked if you'd like to improve your design, always say yes. Have a few simple concepts in your backpocket. E.g redis clustering, queueing, analytics using nosql etc.
How about I give an honest and full answer and you don't judge me when I have nothing to add because I already gave everything? God I hate you corporate scumbags and your stupid games.
Seems kinda obvious to say yes but can expand what specifically are you looking for? Your suggestion implies to me you’re looking for knowledge of optimising and monitoring production applications.
thats a very vague question in system design, you need to specify the area where it needs to optmize. learn to ask question first. so called hiring manager
This is how it should be. System design interviews are supposed to be interactive and interviewee should feel welcoming and that he is discussing with his future team mate. But in majority of the cases atleast from where I come, interviewers feel like they are the supreme architects who designed all the great apps on the internet and interviewee is a kid.
Using other providers' product is a way to learn what others provide and remain updated. Using in-house product means you close yourself from the world.
This is a great example of a system design interview. I have been interviewing for mid level developer roles and those questions tend to focus more on the API design rather than the distributed system design. I would like to see more videos that focus on designing the API rather than the distributed system. Thank you for all these videos. They are very helpful
Surely the Googler can afford a better mic; frankly it amazes me how people just ignore terrible audio quality when a substantial improvement is easy and would cost ~$30!
13:30 I don't feel as though there was enough thought put into the SQL vs NoSQL discussion. In my opinion based on the non functional requirements presented to us for a highly available system and 1 million active users per day, I think the data scalability should be a huge consideration in that decisions but wasn't even mentioned. Horizontally scaling a SQL database is possible with sharding etc. but can be complicated to do. NoSQL can pretty trivially and cost effectively scale horizontally especially in cloud environments. And the data we are storing related to the videos is just small chunks, we aren't really leveraging a lot of structured data schemas or anything like that. I don't know, I just think I challenge that decision a little bit.
How to pull the videos of users you follow ,I guess that needs a join.Otherwise we have to denormallize and keep all the user and follower information in single table which I assume is bit complex and might have lot of data redundancy
The sound quality is a bit disappointing: all these noises and crackles. But the interview itself was pretty interesting and useful. Thanks for sharing 👍
This perons is so pro. Technical background is solid, talking is so good. I wouldn't be surprised he can become director soon! All in all talking and expressing your thoughts is the most important.
Great job mediating the interview, I really like how you asked Adam to dive into the discussion about relational vs non-relations DBs, we would miss good chunk of information without it.
There is an entire industry built around passing the SE interview :) Looks similar to the 'how to become rich in the stock market' courses, the authors are making way more money apparently teaching you how to go and pass the interview then going themselves and pass it :)
Agree. It also makes me think that here we're learning how to pass a system design interview, whereas one should really be studying "how to design systems". It might sound like a subtle difference but it is quite a sad realisation if you think about it
Something to add: 1. how do you handle creation of unique video ids? 2. how do you authenticate users? 3. how do you handle the thumbnails and different solutions of the videos? 4. how do you handle the video duplications? for example, if a user uploads a video twice? how about different users upload same video? 5. how do you handle hot videos or users? 6. I would add some message queues to decouple the components
A bit disappointed, in my opinion below aspects should have been focused on - 1. How will your architecture support a million users. 2. What makes your architecture scalable 3. What mechanism will you use to search content across millions of video metadata 4. How will your data storage be available across geographies 5. How will you enable a stream of videos of users that you follow These are the core questions, what he designed can be a good starting point, but unfortunately the challenging part wasn't tackled
In my experience in India, usually interviewers aren't that patient. I remembered asking do you want me to explain as I go. The guy said "no no, you implement take your time we will discuss it later" 😀
I wish I can be calm, smooth, and slowly add the complexity to my design like this man. This man is so smooth , just like the milkshake in the background. Great mock interview.
Nice video. One thing I think ‘likes’ should not be one column of video table. Its access pattern is different from other columns, like video description, blob link. ‘likes’ could be changed very frequently. I feel it should be a separate service and may be like a even driven thing (e.g. a msg queue followed by nosql db)
also, isn't all likes needs to be stored with userid. Because we are certainly allowed to unlike a video. So, if they don't store like count. and which user likes the video there are no option for unlike.
Yea, it didn't make sense to me at all. You have databases, how can you access a DB across CDNs? lol, it only makes sense for static stuff, as you said. In the case of Tiktok, that would be grabbing videos blobs/files.
imagine how many years it needed to understand and practice all those techs. i can catch up to DB design, but beyond that; CDN, LB, caching ... im lost XD (never done that before) so yeah, this is very aspiring interview.
They all work together to improve performance and hence UX, so you'll probably come across all of them soon enough. And you don't need to dive deep to understand what they do and how they're beneficial.
Choose a SQL database has a big problem with big data and doesn't scale cheaply, it could cost a lot of money. A better solution is to use a NoSQL database that is designed for big data, scales and needs low investment in comparison with SQL database products. It's a good subject for an open discussion and to learn more. Anyway the System Design Mock Interview was great!
This is a good and bad interview. Experiences : 2-4 Years --> Good. As he was able to build the structure around the problem. Experience : 4 Years + --> Bad . Did not go into any details. Just generic statements. PS: I wanted the interviewer to ask more questions and make him struggle.
I once asked to design a video platform like Netflix. They were going so into detail at everything I said. I believe I did better than this video with similar components like cache, CDN, pre population for the feed etc. Talked about NoSQL, scaling, security, etc... but guess what, I failed :D experience was 5 years
For the DB discussion around minute 21:00, would it make sense to have a scheduled ML job that reads from the RDB or the replicated read only version and creates the user curated list, and then drop the results into a NoSQL db (like Apache Cassandra, with primary key of user Id plus clustering on say uri of video). This would scale horizontally very easily and it’s purpose built for quick reads and supports the non functional requirement of high availability.
I feel the interview could have been a bit more engaging, Few areas would have covered: 1. SQL vs NoSQL, given that its High Avalaibility, Read heavy, huge data volume and the only relation we have is user-video, why would you use SQL 2. Elaborate more on Video upload API. How would the upload happen? Multipart form data? 3. Talk more about the pre cache service but you mentioned you would have done that with more time
Nice effort but You totally missed some important functionality like how videos will be streamed to user? Also how you will meet low latency high throughput etc.. Considering estimates you didn't used them to calculate your size of database and how you will optimise DB OPS (using indexes etc)
His high level design is his highlight, clearly done some practice on designing UA-cam. He broke up the DB and precache smooth AF Also like how he backfill the db model at 24:10
i feel youve developed a great system in 2005 architecture. i expected choreography with events, kafka and streaming . very disappointed in this design that would not scale well, requires huge joins and is very inneficient in eventual consistentancy
Thanks for sharing video! I was impressed by a concept he mentioned while organizing the requirements ---- how budget affects the reliability of a service? This wasn't something I had heard in previous system design interviews or discussions. Although I'm not sure how this would play out in an actual interview, I personally find it very practical. It made me immediately consider how much cloud storage or other resources I might need to calculate. On the other hand, I definetly read through the comparisons and debate on sql vs nosql above. It helped me a lot.
The ASMR in the background was a really nice touch... although the tingling were overwhelming from time to time. Jokes aside, very helpful video though. Thanks for this.
this gives me hope cuz he says things in the most obvious way "It can run on a schedule... or like on demand, or something like that" Google here I come
like the cdn for the video may want to also go into the audio/video encoding of video before storage or if live streaming how to encode and deliver content before a full store, they talked about scale but may also want to discuss adapting to a change in network bandwidth
be very careful before writing down non functional requirements. high availability is almost always enough but, if you wanna say 99.999% which is dream come true scenario, then don't really address the requirement it's a negative imo.
Interviewer asked very realistic questions. There could be more descriptive explanations though. Adam do have clear understanding of each component but could have been much more informative if he could have shared from his experience more in depth explanations of each component.
Great video, nicely explained everything. However about database sharding; I think sharding should not be done based on region, right? Because for a particular region with lots of users database sharding could be done based on user_id. Moreover, we also need to keep in mind about popular users. If too many popular users are in one DB, this will create too much load on a single database. So there should be some strategies to evenly distribute the ids of those users among the available servers.
Yes, I agree that database sharding should not be based on region. Coming on your suggestion, when we do database sharding based on user_id, this might create huge traffic for a celebrity user. So, what if we do sharding based on video_id(post_id) ? In this way, yes one shard will have multiple instances but we can get away with traffic on a particular shard.
For the question about scaling during the question of Sharding, there was a mention of a sharding service. so are we talking about algorithmic sharding or dynamic sharding? Seems like an external module to do the sharding. I believe tik Tok would do algorithmic sharding based on region. I would shard on user_id (horizontal) as recommendation feed was a part of the design. This would make querying much faster than have based on region. Also the API gateway was missed and may be we need to talk about rate limiting as well as such systems open to public could be possible DDos vulnerable. Also security/monitoring/logging not much focus on those aspects..
1. Hot, Warm, Cold videos. The hotter, the more copies. 2. "Viral Regional video" is probably more realistic than "Global Viral". Hence bring data closer to the consumers. 3. Data is not relational, as videos are not tied together - NoSQL? 4. Data does not change much, but there are lots of reads - Immutability? 5. 4k Videos are heavy - CPU/GPU optimized service compression? 6. Composable UI - so that features like comments, likes etc work independently and in case of partial failure, system continues to offer a best-effort set of features. 7. Likes feature - mention about Top K problem and hashmaps 8. Client side caching - once video is already watched allocate a reasonable buffer for storing them. 9. Protocol focused on delivering video/audio content ? At least these are immediate things I could think of...
If a person is following another person and eac reside in separate shards, how you're gonna tackle that ? I think if you give this solution in an actual google interview, you will be rejected. It lacks a lot of details. The pre-cache service will only populate initial feed objects, what about later ones ? There's no mention of that.
This problem should have more focus on big data. All database design is based on RDBMS relationship. Having something like Cassandra and memcache / Redis as solution would have been better. Suggested approach of running the algo before user logs for faster response, is tricky. Considering you have 1M users and you don't know who will login when pre curating list for all 1M user is kind of useless. That should be build from memcache directly with out hitting DB. and once user is active, we can refresh the cache for user's followers.
What about counting read:write ratio? counting RPS for reads and writes? (to understand what kind of system in should be) Initial DB storage and storage scaling calculations? How much data you will store in a month? Throughputs? Why cache and not NoSQL in case of GET user feed? What to do with stale cache data? What if you lose all cache data?
I feel like an interviewee can get away with simply stating that the system will be highly available and nodes going down will not cause undue service outages. The reason for that is if I state 5 9's of availability and my interviewer asks why not 6 9's then I have to justify that and strictly speaking how much does your design change if at all? Maybe for a solutions architect or a staff role i'd be a little more OK with dropping solid numbers like that but i'd then grill the person on what changes and how it would be different at different levels of 9s.
Watched this video in three settings. By the third time, my 2 year old daughter would hear the interviewer's voice and immediately start imitating the milkshake sounds. "Dad, why does what you are watching go swshh swshhh swshhh"
I will use additional technologies - maybe even I see mistakes, where is autoscaling environment for microservices? Where is nosql database? No all trafic should go through CDN Where is authentification service? Monitoring? Real time analysics? etc..
I agreed on the choice of RDB for read/write to start, but was a bit confused by his assuming the possibility to shard on an RDB rather than switching over to a NoSQL since NoSQL are known to be sharding friendly?
Depends on the skill set of the team. Some NoSQL databases can be harder to shard than some relationship databases now (such as postgres with built in partitioning)
One question - after we have consumed the precomputed feed how do you recompute, is it based on timestamp / relevance etc and how will it get served ? Whether this new feed will be stored in cache and how ordering will be maintained ? Whether we will have to persist this ?
Hello, I'm an iOS Engineer and I'm very curious about the mobile app partial but you skipped it in the video. Could you please make videos about system design interview for mobile area ?
Don't leave your career to chance. Sign up for Exponent's system design interview course today: bit.ly/3PhhbMY Liked this video and want to see more? Let us know by clicking "Subscribe"
This didn't feel like an interview, but rather just some quick-and-dirty drafting. You should have questions and considerations prepared for potential design proposals to help give a bit of depth and better pick the brain of the "interviewee". For instance: "Wouldn't running jobs to pre-load per-user caches of queries poorly scale with user count?", and listen to how this might or not be a problem, and if so, how would we proceed in order to reduce potential bottlenecks. Just my 2 cents :-)
@@omerocak4710 I guess that’s possible, I just think that for educational purpose and for the sake of preparation going a little bit deeper into the details would have been great
I would use a nosql db like for likes liked follows and comments that can fit CAP theorem. RDBS would be high latent for such things even with sharding.
I wonder if anyone ever starts with the domain model of the problem at hand? I felt we jumped into tables and stuff pretty quickly. PS - I am preparing for such interviews and looking for recommendations.
depends on your previous experience/background. Some start with APIs, some start with data models first (tables and relations is just one example of data modeling). Typical framework to start is - Functional, Non-Functional, Constraints. Then discuss major actors, their use cases and typical user stories. Unfortunately, the exact answer to many software architecture questions is always - it depends on many factors (your background, type of interview, time constraints, role you're interviewing for, ...)
I totally couldn't understand why using UserId as the PK for the actual "Video table". First, it out of intuitive, userId should not be the PK for the video considering that user to video is 1 : N. Another reason is, if we did sharding by UserId, the system will be out of balance....
The overall structure is good, however, the answer itself to me is not really strong, He did not really answer the question, especially the bottle neck, database choice, sharding part.
Beginning of the interview "I don't know much about tiktok, I have never used it, can you please talk a little about what we are building" 5 mins later, "I know that tiktok uses a special algorithm to create a user feed", "I know that tiktok videos are 30 seconds" Using RDB for this system is a disaster, it will never scale the way nosql will. If videos are stored in blob storage, why do we need the read only database? esp given that there is a caching layer!
How reasonable is it for someone with no system design experience (but platform software design experience) to do reasonably well in a question like this? Is it necessary to have experience to interview well for roles that will require system design, or is it sufficient to have at least studied it?
Studying will help you get the structure of the system design interview down. But ultimately experience is the best way to develop intuition for how to approach the functional (ie. product) and non-functional requirements for any given problem. That’s why system design interviews are how companies identify senior/staff SWE from junior SWE. Don’t let that stop you though from studying, practicing and going for interviews.
I found this helpful, however a piece of feedback: Moving back and forth between the people and whiteboard constantly was distracting. Most folks watching this are interested in the design, so I suggest keeping the whiteboard up the entire time and keeping the interviewer and candidate small windows
Interviewer - Design TikTok
Candidate - I never used TikTok
Interviewer - Great. Go ahead.
Candidate: 23:43
As a hiring manager, here is one tweak I'd add to this interview. When asked if you'd like to improve your design, always say yes. Have a few simple concepts in your backpocket. E.g redis clustering, queueing, analytics using nosql etc.
I mean, he basically said "yes, and here's what I would add if I had time."
How about I give an honest and full answer and you don't judge me when I have nothing to add because I already gave everything? God I hate you corporate scumbags and your stupid games.
Seems kinda obvious to say yes but can expand what specifically are you looking for? Your suggestion implies to me you’re looking for knowledge of optimising and monitoring production applications.
"just toss in redis"
thats a very vague question in system design, you need to specify the area where it needs to optmize. learn to ask question first. so called hiring manager
This is how it should be. System design interviews are supposed to be interactive and interviewee should feel welcoming and that he is discussing with his future team mate.
But in majority of the cases atleast from where I come, interviewers feel like they are the supreme architects who designed all the great apps on the internet and interviewee is a kid.
I love how all the google employees reference S3 and EC2 rather than GCP itself 😂
This is amazing and funny at the same time
He isn't yet working for Google. He said he was about to move there from oracle.
@@alHailHalePoint still stands, Oracle has their Cloud services.
Yeap, similar to how all of Microsoft employees use Linux and Mac computers
Using other providers' product is a way to learn what others provide and remain updated. Using in-house product means you close yourself from the world.
This is a great example of a system design interview. I have been interviewing for mid level developer roles and those questions tend to focus more on the API design rather than the distributed system design. I would like to see more videos that focus on designing the API rather than the distributed system. Thank you for all these videos. They are very helpful
That milkshake someone is drinking in the background sounds delicious!
Z
His mic is constantly brushing his clothes causing that annoying noise :(
@@devakash2 r/woosh
Surely the Googler can afford a better mic; frankly it amazes me how people just ignore terrible audio quality when a substantial improvement is easy and would cost ~$30!
@@LoganGabriel He wasn't onboarded to Google by that time. He just left Oracle.
Probably Oracle didn't pay him well 🤷♂️
9:40 very concise way to check if you're on track and move to next stage
11:22 I like how you skip the uninteresting stuff that aren't a problem
13:30 I don't feel as though there was enough thought put into the SQL vs NoSQL discussion. In my opinion based on the non functional requirements presented to us for a highly available system and 1 million active users per day, I think the data scalability should be a huge consideration in that decisions but wasn't even mentioned. Horizontally scaling a SQL database is possible with sharding etc. but can be complicated to do. NoSQL can pretty trivially and cost effectively scale horizontally especially in cloud environments. And the data we are storing related to the videos is just small chunks, we aren't really leveraging a lot of structured data schemas or anything like that. I don't know, I just think I challenge that decision a little bit.
Totally agree with you. I had the exact same thought. I didn’t see any relational operation requirements that would require SQL database.
Jokes aside, it can be good to start with a simpler solution and then later on adjust it as you work through bottlenecks.
How to pull the videos of users you follow ,I guess that needs a join.Otherwise we have to denormallize and keep all the user and follower information in single table which I assume is bit complex and might have lot of data redundancy
He has outsourced the pull videos from ppl one follows to a prefetch service lol
@@123vat69 that makes sense.
I'm really impressed by the questions he asked as part of the Functional Requirement. Those were serious questions ❤️
True
The sound quality is a bit disappointing: all these noises and crackles.
But the interview itself was pretty interesting and useful. Thanks for sharing 👍
How to create view feed is perhaps the most important and complex question, for some reason not touched, the only mentioning that it will be prebuilt
Yep too high level. If this is the interview bar I'm doing really great. Totally a SDE9
This perons is so pro. Technical background is solid, talking is so good. I wouldn't be surprised he can become director soon! All in all talking and expressing your thoughts is the most important.
Great job mediating the interview, I really like how you asked Adam to dive into the discussion about relational vs non-relations DBs, we would miss good chunk of information without it.
The guy on the left is perfect Clark Kent. Take the specs off and we have Superman!
I feel the answers were a bit weak overall.a few times actually I felt the answer wasn’t given at all to the question, rather avoided
that's what i do in my interviews.
I really liked how he carried the conversation....interactive
There is an entire industry built around passing the SE interview :)
Looks similar to the 'how to become rich in the stock market' courses, the authors are making way more money apparently teaching you how to go and pass the interview then going themselves and pass it :)
facts
But they probably can actually pass them if they want to lol
Agree. It also makes me think that here we're learning how to pass a system design interview, whereas one should really be studying "how to design systems".
It might sound like a subtle difference but it is quite a sad realisation if you think about it
some have a passion for teaching. Imagine all your coding mentors focused on writing code instead of writing books and going for talks.
Yea too many fucking ppl entering this industry. Its getting nauseating.
Something to add:
1. how do you handle creation of unique video ids?
2. how do you authenticate users?
3. how do you handle the thumbnails and different solutions of the videos?
4. how do you handle the video duplications? for example, if a user uploads a video twice? how about different users upload same video?
5. how do you handle hot videos or users?
6. I would add some message queues to decouple the components
A bit disappointed, in my opinion below aspects should have been focused on -
1. How will your architecture support a million users.
2. What makes your architecture scalable
3. What mechanism will you use to search content across millions of video metadata
4. How will your data storage be available across geographies
5. How will you enable a stream of videos of users that you follow
These are the core questions, what he designed can be a good starting point, but unfortunately the challenging part wasn't tackled
Nearly impossible to cover this in the time available for a systems design interview
pre-cache needs more explanation. when will it invalidate for billions of users. W/o users on the system it will still keep running. Is it optimal?
@@count3ears Bruh they spent only 32 mintues. System design interviews are generaly 1 hr. It would look bad if you end the interview in just 30 mins.
In my experience in India, usually interviewers aren't that patient. I remembered asking do you want me to explain as I go. The guy said "no no, you implement take your time we will discuss it later" 😀
That should tell you something about the company. :(
@@marcosabreu4684 That tells a lot about your mom
I wish I can be calm, smooth, and slowly add the complexity to my design like this man. This man is so smooth , just like the milkshake in the background. Great mock interview.
lmfao the milkshake.
LOL
Nice video. One thing I think ‘likes’ should not be one column of video table. Its access pattern is different from other columns, like video description, blob link. ‘likes’ could be changed very frequently. I feel it should be a separate service and may be like a even driven thing (e.g. a msg queue followed by nosql db)
likes should be in a graphdb which bis faster. they should also appear in nosql document
also, isn't all likes needs to be stored with userid. Because we are certainly allowed to unlike a video. So, if they don't store like count. and which user likes the video there are no option for unlike.
@@thenextguy4570 you are right, what is this guy talking about
need to store info on which user liked it so they can unlike, and so you can create custom feeds based on videos that a User likes
CDN should sit in front of static content only. Not sure you need to put that it in front of all APIs.
Yea, it didn't make sense to me at all. You have databases, how can you access a DB across CDNs? lol, it only makes sense for static stuff, as you said. In the case of Tiktok, that would be grabbing videos blobs/files.
@@HardwareAddiction If he used a document-based database he could store the responses as JSON on the CDN
imagine how many years it needed to understand and practice all those techs.
i can catch up to DB design, but beyond that; CDN, LB, caching ... im lost XD (never done that before)
so yeah, this is very aspiring interview.
They all work together to improve performance and hence UX, so you'll probably come across all of them soon enough. And you don't need to dive deep to understand what they do and how they're beneficial.
Choose a SQL database has a big problem with big data and doesn't scale cheaply, it could cost a lot of money. A better solution is to use a NoSQL database that is designed for big data, scales and needs low investment in comparison with SQL database products. It's a good subject for an open discussion and to learn more. Anyway the System Design Mock Interview was great!
I like how he drove the direction of the interview
This is a good and bad interview.
Experiences : 2-4 Years --> Good. As he was able to build the structure around the problem.
Experience : 4 Years + --> Bad . Did not go into any details. Just generic statements.
PS: I wanted the interviewer to ask more questions and make him struggle.
collection of experiences
I once asked to design a video platform like Netflix. They were going so into detail at everything I said. I believe I did better than this video with similar components like cache, CDN, pre population for the feed etc. Talked about NoSQL, scaling, security, etc... but guess what, I failed :D
experience was 5 years
The interviewer himself is not very good to be honest. I've seen some of his other videos and he appears to be a rather mediocre engineer.
@@sedat4842 Very interesting. Did you fail becoz of the system design interview? Or were there other rounds that led to the consequence?
For the DB discussion around minute 21:00, would it make sense to have a scheduled ML job that reads from the RDB or the replicated read only version and creates the user curated list, and then drop the results into a NoSQL db (like Apache Cassandra, with primary key of user Id plus clustering on say uri of video). This would scale horizontally very easily and it’s purpose built for quick reads and supports the non functional requirement of high availability.
I feel the interview could have been a bit more engaging, Few areas would have covered:
1. SQL vs NoSQL, given that its High Avalaibility, Read heavy, huge data volume and the only relation we have is user-video, why would you use SQL
2. Elaborate more on Video upload API. How would the upload happen? Multipart form data?
3. Talk more about the pre cache service but you mentioned you would have done that with more time
Nice effort but You totally missed some important functionality like how videos will be streamed to user? Also how you will meet low latency high throughput etc.. Considering estimates you didn't used them to calculate your size of database and how you will optimise DB OPS (using indexes etc)
HLS streaming for Videos?
Ever heard of pied piper? :P
29:45 shows how huge scaling exponentially increases components complexity. Nice video
His high level design is his highlight, clearly done some practice on designing UA-cam. He broke up the DB and precache smooth AF
Also like how he backfill the db model at 24:10
i feel youve developed a great system in 2005 architecture. i expected choreography with events, kafka and streaming . very disappointed in this design that would not scale well, requires huge joins and is very inneficient in eventual consistentancy
I wish everyone gets such a decent interviewer
Interviewee needed to talk a bit more about fault tolerance. You might introduce a queue to handle processing the videos after they are uploaded.
Great instruction! - a side question: what's the tool you use to draw your flow chart? Thx!
He's using Whimsical!
@@tryexponent Thanks!
@@tryexponent Was gonna ask the same thing, thanks
Thanks for sharing video! I was impressed by a concept he mentioned while organizing the requirements ---- how budget affects the reliability of a service? This wasn't something I had heard in previous system design interviews or discussions. Although I'm not sure how this would play out in an actual interview, I personally find it very practical. It made me immediately consider how much cloud storage or other resources I might need to calculate. On the other hand, I definetly read through the comparisons and debate on sql vs nosql above. It helped me a lot.
The ASMR in the background was a really nice touch... although the tingling were overwhelming from time to time. Jokes aside, very helpful video though. Thanks for this.
This is what true artistry looks like, just amazing
Congratulations 👏😊
Brilliant interview
Started and stayed humble throughout
this gives me hope cuz he says things in the most obvious way "It can run on a schedule... or like on demand, or something like that" Google here I come
Likes requires a many to many table users videos. Follows can be a many to many users users.
like the cdn for the video may want to also go into the audio/video encoding of video before storage or if live streaming how to encode and deliver content before a full store, they talked about scale but may also want to discuss adapting to a change in network bandwidth
I really benefited from this video! thanks guys for your time and effort
be very careful before writing down non functional requirements. high availability is almost always enough but, if you wanna say 99.999% which is dream come true scenario, then don't really address the requirement it's a negative imo.
S3 being 99.99% availability, that could play against him to not achieve the 99.999%
Interviewer asked very realistic questions. There could be more descriptive explanations though. Adam do have clear understanding of each component but could have been much more informative if he could have shared from his experience more in depth explanations of each component.
More than Exponent you have been promoting Whimsical! :D
Finally, someone spoke about it.
Great video, nicely explained everything. However about database sharding; I think sharding should not be done based on region, right? Because for a particular region with lots of users database sharding could be done based on user_id. Moreover, we also need to keep in mind about popular users. If too many popular users are in one DB, this will create too much load on a single database. So there should be some strategies to evenly distribute the ids of those users among the available servers.
Yes, I agree that database sharding should not be based on region. Coming on your suggestion, when we do database sharding based on user_id, this might create huge traffic for a celebrity user. So, what if we do sharding based on video_id(post_id) ? In this way, yes one shard will have multiple instances but we can get away with traffic on a particular shard.
For the question about scaling during the question of Sharding, there was a mention of a sharding service. so are we talking about algorithmic sharding or dynamic sharding? Seems like an external module to do the sharding. I believe tik Tok would do algorithmic sharding based on region. I would shard on user_id (horizontal) as recommendation feed was a part of the design. This would make querying much faster than have based on region. Also the API gateway was missed and may be we need to talk about rate limiting as well as such systems open to public could be possible DDos vulnerable. Also security/monitoring/logging not much focus on those aspects..
1. Hot, Warm, Cold videos. The hotter, the more copies.
2. "Viral Regional video" is probably more realistic than "Global Viral". Hence bring data closer to the consumers.
3. Data is not relational, as videos are not tied together - NoSQL?
4. Data does not change much, but there are lots of reads - Immutability?
5. 4k Videos are heavy - CPU/GPU optimized service compression?
6. Composable UI - so that features like comments, likes etc work independently and in case of partial failure, system continues to offer a best-effort set of features.
7. Likes feature - mention about Top K problem and hashmaps
8. Client side caching - once video is already watched allocate a reasonable buffer for storing them.
9. Protocol focused on delivering video/audio content ?
At least these are immediate things I could think of...
I would expect interviewer to be more actively challenging and ask for deep diving into areas of interest.
I actually came very close to this design while I was designing TikTok on my own. I'm very proud of myself:) Thanks for the video!
i would have saved chat about redis cache for the "further optimisation" sections... gotta leave something in your pocket!
Great video but the microphone disturbance makes it unbearable to watch it!
I just looooove watching mock interview videos
yea wait until you gotta do em. they aint fun
@@rafaeashraf605 I'm still very new to it XD
If a person is following another person and eac reside in separate shards, how you're gonna tackle that ? I think if you give this solution in an actual google interview, you will be rejected. It lacks a lot of details. The pre-cache service will only populate initial feed objects, what about later ones ? There's no mention of that.
Love you contents, I've learnt alot in just few minutes, my head wants to explode 🤣🤣.
This problem should have more focus on big data. All database design is based on RDBMS relationship. Having something like Cassandra and memcache / Redis as solution would have been better.
Suggested approach of running the algo before user logs for faster response, is tricky. Considering you have 1M users and you don't know who will login when pre curating list for all 1M user is kind of useless. That should be build from memcache directly with out hitting DB. and once user is active, we can refresh the cache for user's followers.
I think you pretty much can keep 5 next videos for every user. Not goes along well with trendy videos, so yeah, a debatable choice.
Great video! Would an interviewer expect a TPM to be this detailed? I'd love to hear some feedback.
And would this interview be a fair expectation for post MBA TPM interviews.
Or would they be less technical
More of a software developer interview.
Yes for Senior TPM or software engineering TPM
What about counting read:write ratio? counting RPS for reads and writes? (to understand what kind of system in should be) Initial DB storage and storage scaling calculations? How much data you will store in a month? Throughputs? Why cache and not NoSQL in case of GET user feed? What to do with stale cache data? What if you lose all cache data?
Respect when he mentioned he never used tiktok
this is great. it'd be great with more video like these
I feel like an interviewee can get away with simply stating that the system will be highly available and nodes going down will not cause undue service outages. The reason for that is if I state 5 9's of availability and my interviewer asks why not 6 9's then I have to justify that and strictly speaking how much does your design change if at all? Maybe for a solutions architect or a staff role i'd be a little more OK with dropping solid numbers like that but i'd then grill the person on what changes and how it would be different at different levels of 9s.
“It’s always interesting trying to design something that you’ve never used”…that’s pretty much the purpose of System Design! 🙄
Really Nice Interview, Thanks Adam
Watched this video in three settings. By the third time, my 2 year old daughter would hear the interviewer's voice and immediately start imitating the milkshake sounds. "Dad, why does what you are watching go swshh swshhh swshhh"
The scratchy background noise due to headphones is SO annoying. otherwise, pretty good, and helpful. thanks.
I will use additional technologies - maybe even I see mistakes, where is autoscaling environment for microservices? Where is nosql database? No all trafic should go through CDN Where is authentification service? Monitoring? Real time analysics? etc..
I agreed on the choice of RDB for read/write to start, but was a bit confused by his assuming the possibility to shard on an RDB rather than switching over to a NoSQL since NoSQL are known to be sharding friendly?
Depends on the skill set of the team. Some NoSQL databases can be harder to shard than some relationship databases now (such as postgres with built in partitioning)
One question - after we have consumed the precomputed feed how do you recompute, is it based on timestamp / relevance etc and how will it get served ?
Whether this new feed will be stored in cache and how ordering will be maintained ? Whether we will have to persist this ?
Always start with ERD 🙂
Is this a reasonable expectation for a TPM interview or did he just do exceptionally well?
Good video. The audio quality needs improvement. Crackling noises are heard throughout the video, making it difficult to focus on the content.
It would be nice if he explained how he would populate the read only DB
That's a feature you can get from AWS for "free" so that's probably why
@Exponent What editor your interviewee's use to draw the system diagrams?
Whimsical
Hello, I'm an iOS Engineer and I'm very curious about the mobile app partial but you skipped it in the video.
Could you please make videos about system design interview for mobile area ?
Great system design but there is a lot of background noise.
Don't leave your career to chance. Sign up for Exponent's system design interview course today: bit.ly/3PhhbMY
Liked this video and want to see more? Let us know by clicking "Subscribe"
This didn't feel like an interview, but rather just some quick-and-dirty drafting. You should have questions and considerations prepared for potential design proposals to help give a bit of depth and better pick the brain of the "interviewee". For instance: "Wouldn't running jobs to pre-load per-user caches of queries poorly scale with user count?", and listen to how this might or not be a problem, and if so, how would we proceed in order to reduce potential bottlenecks.
Just my 2 cents :-)
it is a typical TPM interview and believe me, he did great for a TPM
@@omerocak4710 I guess that’s possible, I just think that for educational purpose and for the sake of preparation going a little bit deeper into the details would have been great
This guy is built different
Great video! This video is well structured and it helped me a lot! Thank you for uploading!
Thank you for great example of system design question.
Thanks for these videos. They are really helpful. 😄
Congrats, thanks for sharing the interview
Our pleasure!
What was the online designing tool you used for the design? that was pretty cool
He's using Whimsical
I would use a nosql db like for likes liked follows and comments that can fit CAP theorem. RDBS would be high latent for such things even with sharding.
How do the non-functional requirements have any impact on the design here? I don't see why people always do that.
can you do a System Design Interview as an Engineering Manager for Club House :) ?
I wonder if anyone ever starts with the domain model of the problem at hand? I felt we jumped into tables and stuff pretty quickly. PS - I am preparing for such interviews and looking for recommendations.
depends on your previous experience/background. Some start with APIs, some start with data models first (tables and relations is just one example of data modeling).
Typical framework to start is - Functional, Non-Functional, Constraints. Then discuss major actors, their use cases and typical user stories. Unfortunately, the exact answer to many software architecture questions is always - it depends on many factors (your background, type of interview, time constraints, role you're interviewing for, ...)
I totally couldn't understand why using UserId as the PK for the actual "Video table". First, it out of intuitive, userId should not be the PK for the video considering that user to video is 1 : N. Another reason is, if we did sharding by UserId, the system will be out of balance....
Yeah I think that was not meant to be called the PK obviously that wouldn’t make any sense here. But you do need to store the user id on that table.
Sorry, quick question. What are you guys using to draw this diagram?
thank you very mcuh
What tool did they use for drawing the "UML"?
@@kaanicer Thank yyou
Again which tool is this? Can you provide the link if you know?
@@RohanSharma-eh4kf it's called Whimsical.
@@yesterdayisold thanks
The overall structure is good, however, the answer itself to me is not really strong, He did not really answer the question, especially the bottle neck, database choice, sharding part.
what's the name of the program used for white boarding?
Hey hanyhabib3859, it's called "Whimsical"!
Beginning of the interview "I don't know much about tiktok, I have never used it, can you please talk a little about what we are building"
5 mins later, "I know that tiktok uses a special algorithm to create a user feed", "I know that tiktok videos are 30 seconds"
Using RDB for this system is a disaster, it will never scale the way nosql will.
If videos are stored in blob storage, why do we need the read only database? esp given that there is a caching layer!
Thanks for sharing! I like the interactions of the interviewer and interviewee, and a lot good questions.
How reasonable is it for someone with no system design experience (but platform software design experience) to do reasonably well in a question like this? Is it necessary to have experience to interview well for roles that will require system design, or is it sufficient to have at least studied it?
Curious as well
Studying will help you get the structure of the system design interview down. But ultimately experience is the best way to develop intuition for how to approach the functional (ie. product) and non-functional requirements for any given problem. That’s why system design interviews are how companies identify senior/staff SWE from junior SWE. Don’t let that stop you though from studying, practicing and going for interviews.
Thanks for the video, but the constant mic rustling and cutting away from the workspace to show your face really made it hard to follow...
I found this helpful, however a piece of feedback: Moving back and forth between the people and whiteboard constantly was distracting. Most folks watching this are interested in the design, so I suggest keeping the whiteboard up the entire time and keeping the interviewer and candidate small windows
Hey frefai, thanks for your feedback! We'll consider it for our future videos 🙏