FAANG System Design Interview: Design A Location Based Service (Yelp, Google Places)

Поділитися
Вставка
  • Опубліковано 27 гру 2024

КОМЕНТАРІ • 400

  • @ByteByteGo
    @ByteByteGo  2 роки тому +315

    Corrections: In the geohash length to grid size mapping table at 15:30 and 20:01, the correct values for 7, 8, 9 and 10 are:
    7 152.9m × 152.4m
    8 38.2m × 19m
    9 4.8m × 4.8m
    10 1.2m × 59.5cm

    • @RahulSingh-ln7bg
      @RahulSingh-ln7bg 2 роки тому +8

      Hi Alex, I think you need to correct the definition, at 14:00 onwards you are talking about quadtree, not Geohash, Geohash divides the grid initially into 32 smaller grids since the base here is 32. You may like to correct it.

    • @aaditkamat4995
      @aaditkamat4995 2 роки тому +2

      No wonder, that is why I was thinking why the values are considered too small when in fact they are in the ballpark of 4, 5, 6.

    • @kalmyk
      @kalmyk 2 роки тому +2

      Yeah, that was confusing I had to double check

    • @Penly11
      @Penly11 9 місяців тому +1

      In the video Sahn says that a database in the terabyte range is on the borderline where sharding might make sense. He also says that our read qps of 5,000 is quite high. I was wondering how he came to these conclusions and if there are specific numbers he looks for to determine if a number is high enough to warrant a design change?

    • @lameshithead
      @lameshithead 8 місяців тому

      even though its quite some time since i dealed with geoinformatics, i think you orgot/mixed up some points here. first of all: how do we retrieve the data and how is it stored? when it comes to osm you can get the data in a lot of diferent ways, but you always end up in using quadtree/knearest/bruteforce, but that leads us to the next question: are the searchalgos clientside? and if not, why dont we replace those searchalgos alltogether by storing data in a different way? third: what kind of transformation are we using here? most likely datums i guess, but using the one with the letters in the rows, would make the answer go in a complete different way, than equatorial and easting, or datums without eastings

  • @zachgoll
    @zachgoll Рік тому +186

    Hey UA-cam algorithm, if you’re reading this, I just want to say, this is the type of video you should be recommending to software people. K thanks

  • @prashanttoxeed
    @prashanttoxeed 2 роки тому +322

    Best design video ever. I am so happy that Alex decided to make videos.

  • @pramodpotdar5416
    @pramodpotdar5416 2 роки тому +80

    this video is a FLOW. Could not stop watching... Beautiful animation and narration. Perfect

  • @miale3593
    @miale3593 2 роки тому +149

    So much effort spent in how to make this video so informative, well structured, precisely explained and amazingly illustrated. Thank you for sharing this with us!

  • @TheTacoReview
    @TheTacoReview 10 місяців тому +3

    Holy crap... finally a guy that actually knows what he's talking about. I've been doing this for 20 + years and this is how it's done. Kids you don't need to make it overly complex, just build it so it can scale not at scale.

  • @M3t4lstorm
    @M3t4lstorm 2 роки тому +15

    Notes:
    - The SQL query at 20:27 would not work for the same reasons you mentioned previously in the video (prime meridian + equator).
    - You assume that the long/lat is the centre of the business/place, if users are able to add their own businesses this will probably not be the case
    - Businesses/places can span multiple grids (think of shopping malls)
    - Businesses/places can be bigger than a grid (think of airports)
    - At the very end you say it would "use the long/lat to rank the businesses and return to the client" (sorting), you also need to check if the business is still within the radius the user specified (filter); just because the business is in the same grid/neighbouring grids doesn't mean its within the radius they asked for.
    The first thought might be to make the geohash a list (of all the grids that it covers), but how do you calculate that? You would need a polygon (a long/lats for each vertex) that covers the (rough?) area of the business/place, then you get onto the trouble of validating that :)

  • @vskovzgird
    @vskovzgird 2 роки тому +41

    Hey, Alex. I learn x100 times more from this video than from my past year in IT. Totally awesome content!

  • @kakanishrikant
    @kakanishrikant 2 роки тому +27

    This is by far the BEST VIDEO on questions like "Design Yelp". Pure Gold !! Thanks, Alex, and ByteByteGo team for your outstanding work.

  • @mallipooalwa9085
    @mallipooalwa9085 2 роки тому +8

    Wow, I'm an enterprise software engineer and from my work experience and knowledge, this is one of the best channel that digs into the ways of working and architecturing an enterprise software. Kudos 👏

  • @geeklogbook
    @geeklogbook 2 роки тому +5

    This video should be taught at universities and bootcamps. Not only for the interesting topic but also to learn how to think about this kind of problem and system design in general. Thank you!

  • @pizzatime9565
    @pizzatime9565 Рік тому +12

    We need more full system design videos like this one from you!

  • @pixusru
    @pixusru 8 місяців тому +1

    I got this question on my Meta on-site interview. Absolutely bombed it.

  • @bkswain101
    @bkswain101 2 роки тому +10

    I have been following a number of channels related to system design in last few months. While many of them are brilliant, Sahn has a very unique and effective way of communicating complex technical topics. Thanks Sahn and team for this content. Hope to see more of these.

  • @emreboga
    @emreboga 2 роки тому +6

    Dude, your content is gold! I've been coding almost 20 years and prepared for interviews countless times. This has been one of the best content so far!

    • @ByteByteGo
      @ByteByteGo  2 роки тому +11

      Thank you for the encouragement. This is the first chapter-length video we made, and it was a lot of work. Your feedback is much apprecated.

    • @ChrisCox-wv7oo
      @ChrisCox-wv7oo 2 роки тому

      @@ByteByteGo wonderful job.

  • @robl39
    @robl39 Рік тому +2

    This is hands down the best design video I’ve ever seen

  • @arpanghoshal2579
    @arpanghoshal2579 2 роки тому +5

    By far the best system design channel ever. Crisp presentation and fluid animations to easily showcase complex topics in a simple manner.

  • @alfazahmed
    @alfazahmed Рік тому

    This has to be the highest quality system design explainer video on UA-cam. * take a bow *

  • @深夜-l9f
    @深夜-l9f Рік тому

    he explains like a teacher you would find in a school that everyone loves

  • @MaxRunia
    @MaxRunia 2 роки тому +5

    I bought Vol 2 but I find video format slightly easier to digest. I really appreciate you making these videos.

  • @TheMeanmachine1989
    @TheMeanmachine1989 2 роки тому +3

    Thank you so much for starting to make these videos. Your step by step approach is so clear and crisp. Best System Design material ever!!!

  • @andreishakh4863
    @andreishakh4863 2 роки тому +1

    I had exactly this problem on my system design interview @ faang, failed it miserably)) Excellent quality of material here, will help a lot in future!

  • @nishantgarg2815
    @nishantgarg2815 2 роки тому +17

    15:37 for 7-10, units would be in metre not km

  • @huynguyenquang2656
    @huynguyenquang2656 Рік тому

    i hope u make 10 videos everyday so i can learn forever about system design. Thanks so much

  • @arekf3470
    @arekf3470 2 роки тому +1

    Please continue making videos. The speaker did an amazing job, clear with a nice tone. The visual presentation is also nicely executed. 10/10, subscribed.

  • @ksuhdilla
    @ksuhdilla 2 роки тому +1

    This is such an underrated channel - very good visuals, concise speech pattern, and extremely well thought out approach to each topic

  • @effy1219
    @effy1219 2 роки тому

    I've watched like most of the SD videos from YT, i could tell you guys this one is the best. Thanks Alex

  • @Fahadahammed
    @Fahadahammed Рік тому

    Your approach is fascinating. It kept me watching the whole video and I find this very rare. Thank you.

  • @king0s
    @king0s Рік тому

    Beautiful poetry just opens up my eye. Overwhelmingly grateful. 🙏 and ❤from Chennai 🇮🇳

  • @gauravgarg2344
    @gauravgarg2344 2 роки тому +2

    As humble as he is, his videos are super awesome too. Go alex..!!

  • @dominusgloriae
    @dominusgloriae 2 роки тому

    I'm really lucky to see this video released. It's been about a year since I work in the area of location based service. There is not much of related information that is up to date.
    Thanks a lot for this video! Keep on!

    • @ChrisCox-wv7oo
      @ChrisCox-wv7oo 2 роки тому

      Are you saying the video information is out of date, or up to date?

    • @dominusgloriae
      @dominusgloriae 2 роки тому +1

      @@ChrisCox-wv7oo it's up to date

  • @aaditkamat4995
    @aaditkamat4995 2 роки тому +1

    I am currently going through the System Design Course offered by Design Gurus on Educative which covers similar concepts but in a text based format. I have heard a lot about ByteByteGo's courses and books as well and I'm glad that I looked up this video. Thanks Alex and the team!

  • @sanathkumaru6358
    @sanathkumaru6358 2 роки тому +2

    Please continue to make such wonderful videos. It’s pure gold, glad I discovered the channel.Thank you

  • @red9090
    @red9090 2 роки тому +3

    This was super fun to watch and got me interested again in algorithms and systems in general.
    I decided to go ahead and buy the two books straight ahead. So having a youtube channel definitely helps getting the word out :))

  • @mattbutalla6498
    @mattbutalla6498 Рік тому +1

    I worked on a weather alert system that used S2, you more or less described its design and all the rationale for our decisions (we indexed by S2 cell id). Excellent video.

    • @mattbutalla6498
      @mattbutalla6498 Рік тому

      And for the record, because of this I bought your vol 2 of sys design

  • @raymondmichael4987
    @raymondmichael4987 2 роки тому

    I really like this video, the presenter is very friendly and calm, please extends this, I would like to learn more.
    You got +1 sub from Tanzania 🇹🇿
    Greetings from Tanzania 🇹🇿

  • @nathanthainguyen
    @nathanthainguyen 2 роки тому

    This channel gonna blowup. The graphic designs are the game changer

  • @deepakbhoria4172
    @deepakbhoria4172 Рік тому

    I have tried different videos but this one definitely stands out for understanding location based system design.

  • @kamal-xd7id
    @kamal-xd7id Рік тому +4

    If we can have some sort of video course also like the book you published, I would buy the course right away. The content and the explanation you deliver is really simple to understand and that's the beauty of a good Teacher. "Explain me like I am 5th grade" This really goes for you!

  • @yonahkarp7273
    @yonahkarp7273 11 місяців тому

    This is gold! Toss in a Geosptial DB on your interview and it’s sure to impress them

  • @user-yr1uq1qe6y
    @user-yr1uq1qe6y 2 роки тому +81

    In early 2000s we designed such a system for a popular real estate site very similar. We thought we had nailed the algorithm, but started getting complaints from agents and consumers.
    It turns out in that domain “distance” is almost always driving distance which is an entirely different problem and was much more complex in the days before quick road route planning 😀

    • @kumarsamaksha7207
      @kumarsamaksha7207 2 роки тому +1

      Damn where do you work now?

    • @user-yr1uq1qe6y
      @user-yr1uq1qe6y 2 роки тому +5

      @@kumarsamaksha7207 Nothing real estate related since 2008!

    • @kumarsamaksha7207
      @kumarsamaksha7207 2 роки тому +1

      @@user-yr1uq1qe6y Cool

    • @ivanleon6164
      @ivanleon6164 2 роки тому +2

      i think then you can increase the radius by a % and do a new query of driving distance on that reducer set and rearrange.

    • @riahmatic
      @riahmatic 2 роки тому +3

      literally the traveling salesman problem :p

  • @tommylongpham3889
    @tommylongpham3889 2 роки тому +1

    I just started reading volume 2 of System Design Interview and I'm really enjoying the content so far. Even more so when I realized you created videos to further solidify the readings! Keep them coming!

  • @TheManTheMythThe
    @TheManTheMythThe 11 місяців тому

    Would definitely love to get more full systems design interview videos like this from you. Great video!

  • @ЏонМастерман
    @ЏонМастерман Рік тому

    You sir are a natural for teaching, please dont stop, you're doing world a favor! Cheers !

  • @gigakoresh
    @gigakoresh 2 роки тому +3

    Really good content, as always. You can also add a Kafka or some other streaming service to the business service, so that writes send events to the streaming system. You can then connect that streaming system to the write database as a sink, which will allow you to distribute the write loads evenly throughout the day and handle peak loads without consuming extra resources. Streaming systems like Kafka are so heavily optimized for such use-cases that even a relatively smol cluster with 3 4Gb+2CPU+100Gb should easily be able to handle these loads and have a lot of headroom if you use z-compression on the topic. As an added benefit you can perform change management on the database transparently to the user because streaming system will buffer all write operations automatically while the sink is offline. Streaming systems generally pair with a heavy read/low write system quite well.

    • @aniketshukla9568
      @aniketshukla9568 2 роки тому

      Do we really need to add streaming services? I mean, as we know that write traffic is really very low and also we can compromise with the consistency(will be eventually consistent), so wouldn't Kafka or any streaming service be inefficient to use?

    • @gigakoresh
      @gigakoresh 2 роки тому

      @@aniketshukla9568 Not with that kind of scale, no.

    • @hawkmne
      @hawkmne 3 місяці тому

      As he mentioned in the video, the number of write operations is very low hence no need for Kafka.

  • @anup21d
    @anup21d Рік тому

    A clear and in depth explanation of the proximity service design. Specifically I liked the detail ways to index the geospatial databse.

  • @stoneshou
    @stoneshou 2 роки тому +9

    Hi Sam, you mention use (geohash, business id) as a compound key so we can remove business from the table efficiently. I don't think that's the right thing to do. Instead of a compound key, we should just add another index for the business id. Removing a business thus would only need to check that index. With a compound key we'd need to calculate the geohash for that business, walk the geohash index first, then look for the business id.

  • @mailsiraj
    @mailsiraj Рік тому +1

    wow - what a wonderful video. So well thought out - clearly lays down the ideas using very easy to understand visuals.Thank you so much Alex.

  • @drucev
    @drucev 2 роки тому +3

    excellent video ... I have one basic question, at 19:45, why do you use a compound key combining geohash and business id? if the business id is a unique key, then you are just concatenating some extra junk onto something which is already unique. or did I miss something, why does it make removal more efficient? good stuff!

  • @sergeykichuk2586
    @sergeykichuk2586 Рік тому +2

    In this particular example Load balancer presented not well, because the idea 8:23 of load balancer is to distribute the same request to instances of the same service, here better would to say that there is API gateway and load balancer for each particular service instances!!

  • @joeb6705
    @joeb6705 Рік тому

    This content is unbelievable. Will be checking out your books and newsletters!

  • @thephore
    @thephore 2 роки тому +2

    This is one of the best technology videos I've ever watched! You explained everything very well and with great, informative graphics. Only small suggestion - maybe use a mono spaced font for the queries etc. to improve readability. Thanks for this!

  • @ovidiurosu6632
    @ovidiurosu6632 2 роки тому

    This is the best system design video I've seen so far. Great job mister

  • @josephlabs
    @josephlabs 2 роки тому

    Literally was designing a location based service and wanted advice, its criminal that you don't have more subs and views.

  • @kapilricky
    @kapilricky Рік тому

    This is neat and effective. I only wish i saw this 3 years back when i was asked same in interview🙈😇

  • @amachefe0
    @amachefe0 2 роки тому

    One of the best technical video i have ever watched

  • @ThugLifeModafocah
    @ThugLifeModafocah 2 роки тому

    That geohash part was superb... Damn, so many things I didn't know.

  • @tomeralmog424
    @tomeralmog424 Рік тому +2

    Great video!
    Question please:
    Usually these services will have filters like business type, pricing etc.
    Which table would you add it to?

  • @GughaGSrinivasan
    @GughaGSrinivasan 2 роки тому +3

    I have your System Design Interview Volume I book... Never knew you were the author :) I learned CAP theorem, eventual consistency, and many others from it... I wasn't aware that there is volume 2.. will be adding it to my cart... Great explanation once again!

  • @nayandas1774
    @nayandas1774 5 місяців тому

    Thanks for this video! Cleared most of my problems in my upcoming project!! 😭😭

  • @mahanteshambali
    @mahanteshambali 2 роки тому +2

    As usual amazing video with great details. Thank you Alex. Wish to see more videos of length 20+ mins.

  • @SarKurd
    @SarKurd Рік тому

    This video is so good, i need to watch it a few more times, there is a lot of valuable information here

  • @doxologist
    @doxologist 2 роки тому

    Amazing video. Only thing I'd add is timestamps on the video. But the content itself is priceless

  • @tqh28
    @tqh28 2 роки тому

    you opened my mind about the system design. unbelievable

  • @petelin3181
    @petelin3181 2 роки тому +4

    This is an awesome video. I liked how you were able to break down complex concepts so that even beginners could understood them at a high level. Was able to learn a lot under half an hour about the complete picture of system design

  • @cortfitzpatrick5700
    @cortfitzpatrick5700 5 місяців тому

    Really great explanation. The thoughtful visuals helped me to understand much better.

  • @amanrubey
    @amanrubey 2 роки тому

    Absolutely love this video! 15 year old me craved this information but got it after 8 years

  • @moranh461
    @moranh461 2 роки тому +3

    Loved it, well made and highly educational. Looking forward for more of those.

  • @pobodjjd
    @pobodjjd 10 місяців тому

    Another great video with clear explanation and informative graphs for software engineers

  • @bhavyemalhotra1584
    @bhavyemalhotra1584 3 місяці тому

    better than anything that I have found for this topic! Thanks!

  • @essamgouda1609
    @essamgouda1609 2 роки тому

    This is amazing, what a gem of a youtube channel !

  • @thiagolopessilva
    @thiagolopessilva 2 роки тому

    You are the best person to explain that content. Thanks to share it with us!

  • @imranhussain8700
    @imranhussain8700 2 роки тому +2

    2 minutes in the video and liked it already 👍. What an amazing way of explaining stuff, simple yet effective slides. Thanks man appreciate your efforts.

  • @thuantang
    @thuantang 2 роки тому +1

    Absolutely love how succinct and on point the explanation is.

  • @RichardCurrie
    @RichardCurrie 2 роки тому +1

    Currently the search API will need to query a few hundred businesses after the geoindex read. So business table will have 10x+ query load as the geoindex table

  • @user-jt5nd3yq4u
    @user-jt5nd3yq4u 11 місяців тому

    This is by far the greatest video, thanks a lot for putting this together ❤

  • @RahulSingh-ln7bg
    @RahulSingh-ln7bg 2 роки тому +1

    Hi Alex, I think you need to correct the definition, at 14:00 onwards you are talking about quadtree, not Geohash, Geohash divides the grid initially into 32 smaller grids since the base here is 32. You may like to correct it.

  • @eilselwerdna
    @eilselwerdna 2 роки тому +1

    Amazing video thanks. One thing I did not quite get, are we storing the geohash of each location at the lowest (smallest) precision and letting the client work it out?

    • @ByteByteGo
      @ByteByteGo  2 роки тому +3

      Right. We chose 3 precisions in our design, IIRC? The idea is to store at the smallest grid size of the three precisions, and the server could ask the database for the bigger grid sizes by constructing a query using the LIKE operator.

  • @rumeysadogan7444
    @rumeysadogan7444 2 роки тому +1

    Another eye-opening video. Thank you very much for these high quality, insightful videos. Great work!!👏

  • @abhishek00able
    @abhishek00able 2 роки тому

    So glad I discovered your channel. These are unparalleled videos on system design. Amazing work.

  • @NarayananPrabath
    @NarayananPrabath 2 роки тому

    Alex. I am your great admirer. I would humbly request you to make more System design videos.

  • @sarathfromsaudi
    @sarathfromsaudi 2 роки тому +1

    Excellent content. I liked your structured way of thinking

  • @matiasmoresi5040
    @matiasmoresi5040 2 роки тому +1

    Mindblowing content and way of presenting it! Thank you so much! Keep it up!

  • @chrishabgood8900
    @chrishabgood8900 Рік тому +1

    business info does not change very often so it would be a really good case for caching.

  • @andreffrosa
    @andreffrosa 2 роки тому +1

    7:00 Why is it better to separate the two services than having just one web server doing both? The "you can scale them independently" doesn't suffice since you can scale the "merged" server as well or is there some hindrance on performance that I'm not seeing?

    • @GughaGSrinivasan
      @GughaGSrinivasan 2 роки тому +1

      Location based Services could be used x10 times more than Business service... If you bundle it together like a monolithic model, in most of the cases Business service would be underutilized and wasting system resources... And for that reason, we can just segregate services based on usages/consumption... like in our example we separate Location Based Service instance and increase replica with 20 or more.... While Business service we can set the replicas with 4...

    • @andreffrosa
      @andreffrosa 2 роки тому

      @@GughaGSrinivasan Thank you for your reply but it'l doesn't really answer the question. How exactly would it be underutilized and wasting resources if it is just a function that is invoked based on the type of request?

    • @GughaGSrinivasan
      @GughaGSrinivasan 2 роки тому +1

      @@andreffrosa well you are not wrong with that solution... lets consider 1000 user wants to consume service A and 20 users for service B... service A response time is 1 sec... service B response time is 2 sec...
      If we bundle both these services together and scale this app, lets say 10 instances... lots of http thread requests for service A would block service B to effectively serve... and vice versa...
      If we segregate these services, we can scale 8 for Service A and 2 for service B... now both the service are functioning at best compared to previous solution...

    • @andreffrosa
      @andreffrosa 2 роки тому

      ​@@GughaGSrinivasan "If we bundle both these services together and scale this app, lets say 10 instances... lots of http thread requests for service A would block service B to effectively serve... and vice versa..." how exactly would it block if requests are handled concurrently? You would only slowdown when all the servers are really congested but in that case you should really add another server anyway regardless of the option picked. Having 10 or 8+2 would still be 10 servers in total and each requests would take 1s and 2s anyway. What am I missing?

    • @andreffrosa
      @andreffrosa 2 роки тому

      @@GughaGSrinivasan Assuming each request is from independent users that establish a different connection for each request.
      ​Case I:
      10 servers handling all requests
      80% 1s
      20% 2s
      max of 10K connections simultaneously per server
      (1) That would give an average of 0.8*10K connections with 1s and 0.2*10K connections with 2s, simultaneously per server.
      (2) Following that, we get an average of 0.9*10K connections handled per second per server ((0.8/1 + 0.2/2)*10K)
      (3) From there, the total number of requests processed per second by the whole system would be 10*0.9*10K = 90K on average.
      Case II:
      8 servers handling requests A (1s)
      2 servers handling requests B (2s)
      max of 10K connections simultaneously per server for both types of servers
      (1) That would give an average of 10K connections handled per second per server of type A and 5K connections handled per second per server of type B.
      (2) That entails a total of 8*10K requests processed per second of type A and 2*5K requests processed per second of type B (on average).
      (3) Overall, that would be 8*10K + 2*5K = 9*10K = 90K requests processed per second by the whole system on average.
      Did I make a mistake?

  • @KGTube
    @KGTube 2 роки тому

    The best System Design video so far! Thanks and Keep it going!

  • @aba44614
    @aba44614 2 роки тому

    I am truly thankful for these videos, please keep making more !

  • @ParisPolyzos
    @ParisPolyzos 2 роки тому +4

    The design elements are at another level! Great work! Keep it up!

  • @M1ntAll
    @M1ntAll 2 роки тому

    Bought your book on system design. Great experience, thanks!

  • @mohammadkareem1187
    @mohammadkareem1187 2 роки тому

    Fantastic explanation. Great job. Waiting to see more system design videos from you. Good luck

  • @KENTOSI
    @KENTOSI Рік тому

    Great video where everything was clearly explained. Thank you!

  • @bhagyashingale5663
    @bhagyashingale5663 2 роки тому +2

    I would index the city column. This would prevent table scans. You can then further filter using geo data.

  • @stevecanny1583
    @stevecanny1583 2 роки тому

    Thanks so much for this Sahn! I really appreciate the greater depth of this one :)

  • @tareekhdaan9582
    @tareekhdaan9582 2 роки тому

    A very detailed and well-described video about proximity service

  • @EngineeringPrepGrid
    @EngineeringPrepGrid 2 роки тому +1

    Gr8 design video. Thanks Alex for such high quality, clear concise explanation.

  • @srinivasareddy162
    @srinivasareddy162 2 роки тому

    This one video introduced me to so many concepts. Thank you Alex.

  • @khaledhamoul1403
    @khaledhamoul1403 11 місяців тому

    Amazing! I always thought proximity system are much more complicated than that 😅

  • @vothanhtai6357
    @vothanhtai6357 2 роки тому

    Awesome! The best system design video. Keep doing your great work because it helps people a lot.

  • @bhatibhawani
    @bhatibhawani 2 роки тому

    Best design videos I have ever. Keep the good work.

  • @Leviathan91X
    @Leviathan91X Рік тому

    This is a god tier video. Please keep them coming.