Redis Deep Dive w/ a Ex-Meta Senior Manager

Поділитися
Вставка
  • Опубліковано 22 гру 2024

КОМЕНТАРІ • 117

  • @eastsideGK
    @eastsideGK 5 місяців тому +52

    I've been waiting for videos on deep dives. More deep dives please!! These rock

  • @1234fewgfwe
    @1234fewgfwe 3 місяці тому +20

    1 more thing to add;
    sorted set, internally is implemented with skip lists and hashtables in Redis. Thats why adding value to sorted set is O(logN) on average and searching is O(logN + M). If you bring this up, congratulations you passed your interview

    • @abhijit-sarkar
      @abhijit-sarkar 20 днів тому +3

      On the contrary, if I brought up this statement in an interview after having seen it on UA-cam, I just blew the interview. Any interviewer when faced with a candidate that is trying to impress by throwing out irrelevant buzzwords, will ask "oh yeah, tell me more about how to implement skip lists".

    • @bhawanasahu3000
      @bhawanasahu3000 9 днів тому +2

      @@abhijit-sarkar couldn't agree more. Mentioning such points on your own always brings a follow up question, now I have to also read how skip lists works.

  • @Global_nomad_diaries
    @Global_nomad_diaries 5 місяців тому +12

    Can't thank you enough. You made system design so much easier to understand.

  • @troyfischer6786
    @troyfischer6786 2 місяці тому +1

    Echoing the existing sentiment in that this content is incredibly informative and well put together. Really appreciate your work

  • @remipan4604
    @remipan4604 2 місяці тому

    Hi i just want to say your work is truly fantastic and you are an excellent educator! I started watching your videos to prepare for system design interviews and your explanations and examples are top notch.
    I'm motivated to continue watching not just to prep but also to expand my knowledge and improve my craft. Eagerly awaiting the great work you'll do next!

  • @rahulshrivastava3040
    @rahulshrivastava3040 3 місяці тому

    You are best channel for system design interview. Such a indepth technical understanding, I really appreciate you making these videos.

  • @artemkobeliev6042
    @artemkobeliev6042 5 місяців тому +4

    10:00 didn't quite catch how adding random suffix would solve hot node problem

    • @hello_interview
      @hello_interview  5 місяців тому

      Splits the hot shard into multiple, so you can distribute that load across nodes.

    • @nez14526
      @nez14526 5 місяців тому

      @@hello_interview sorry, what's the approach? I think somewhere else in the video talks about it, but I also somehow did not understand that.
      Is it roughly the following?
      Generate X random suffixes. Maintain a map on the (Redis') client side of .
      Whenever we want to do some sort of "write", we get a random suffix from our list and append it to our key in the Redis command.
      Whenever we want to do some sort of "read", we get _all_ random suffixes (X of them) from our set, make X reads from Redis, and aggregrate on the client side.
      ----
      Or maybe it's sometimes the opposite way. For example, in a cache, I'm guessing we'd want to write our value to each and every one of the key+randomsuffix.
      Then for a read, we can (randomly) pick one of the suffixes.
      ----
      Maybe the missing link (for me and others similarly confused) is how/where the random suffixes are managed.
      There's also the question of how we (automatically?) identify a hotkey, or develop an algorithm that generates (or increases) the number of suffixes depending on some sort of access rate.

    • @jain007neeraj
      @jain007neeraj 3 місяці тому +3

      @@hello_interview but how will you query the key then, as at read time you don't know what random suffix got added

    • @Xiao-p9m
      @Xiao-p9m 3 місяці тому

      @@hello_interview What if the new key conflicts with any keys added later?

    • @vedavyasa9542
      @vedavyasa9542 3 місяці тому

      @@Xiao-p9m How will it conflit, we append the certain value every single time so there wont be a conflict

  • @MQ.2000
    @MQ.2000 5 місяців тому +2

    At 22:55 how does hashing tweet IDs help if we're looking for the most liked tweets overall? Or were you referring to getting top liked tweets for specific keywords when splitting that key across multiple nodes?

  • @anipendakur
    @anipendakur 4 місяці тому

    Really cool deep dive, updated with time as well. Thanks a lot for making this. Looking forward to more!

  • @SagarSood
    @SagarSood 5 місяців тому +3

    Loving the consistency of uploads! :)
    would love to see a deep dive on Kafka as well

    • @hello_interview
      @hello_interview  5 місяців тому +1

      Kafka is in the works! Written resource will be posted to the website this week or early next!

  • @nbx-bi1sk
    @nbx-bi1sk 5 місяців тому +7

    Very informative and helpful video as always. I would appreciate if the visualization of the internals could be a bit more detailed and in sync with what you describe, rather than just a couple of boxes. Regardless, this is a refreshing take to step away from the usual system design videos to dive into technologies that are useful in both interviews and real life. Hope to see more of this in the future.

    • @hello_interview
      @hello_interview  5 місяців тому

      Good feedback, thank you! Will try to make the visuals more engaging in the future.

  • @davidoh0905
    @davidoh0905 5 місяців тому +8

    The problem of hot key is clear but solution of hashtag is very difficult to understand. do we have to handle that all in the client? like figure out what are the hot keys?? and what will adding hashtag do really?

  • @riyaarora7517
    @riyaarora7517 5 місяців тому

    Just love the way you add practical aspects! Thankyou for the thoughtful, concise and articulate deep dive! Was definitely a high ROI on time :)

  • @henryso2717
    @henryso2717 4 місяці тому +1

    Great content. This is a great resource for learning before i apply Redis to my own projects. One thing I was wondering was if your content would be able to be added into Spotify so I could listen along on the go?

  • @haribhatt34
    @haribhatt34 5 місяців тому +3

    Absolutely loved it !!!
    Can we have one deep dive on SQL vs NoSQL, a topic always asked in system design interview.

    • @hello_interview
      @hello_interview  5 місяців тому +1

      We actually have an opinion on that very topic, check this out: www.hellointerview.com/learn/system-design/in-a-hurry/key-technologies#core-database

  • @jordanhasnolife5163
    @jordanhasnolife5163 5 місяців тому +67

    Couldn't focus, too handsome

    • @hello_interview
      @hello_interview  5 місяців тому +5

      The GOAT has arrived and has something important to say!

    • @jordanhasnolife5163
      @jordanhasnolife5163 5 місяців тому

      @@hello_interview Gigachad onlyfans allocator (of) toes, that's very kind of you

    • @coledenesik
      @coledenesik 3 місяці тому +4

      Hey Jordan looks like you are getting your life back.

    • @jordanhasnolife5163
      @jordanhasnolife5163 3 місяці тому +4

      @@coledenesik unclear at best

  • @lesterdelacruz5088
    @lesterdelacruz5088 4 місяці тому +1

    What is the difference between Redis Pub/Sub and Kafka? From the other video, what I understand is Kafka is queue and event driven as well.

    • @hello_interview
      @hello_interview  4 місяці тому

      You can use Kafka as pub/sub! Differences in durability (Redis in memory, Kafka persists to disk), which then has consequences on other things like ease of disconnection, etc. But they can both be used.

    • @hello_interview
      @hello_interview  4 місяці тому

      Redis pub/sub is really dumb, at-most-once delivery of messages. You can think of it as a hashmap from a topic (string) to a socket connection to a subscribing server with all the associated problems (messages might get lost?) and benefits (incredibly lightweight, low-latency etc.)
      Kafka is more full-featured. Kafka has message persistence, you can make guarantees on delivery, etc.

    • @lesterdelacruz5088
      @lesterdelacruz5088 4 місяці тому

      Thank you.

  • @dashofdope
    @dashofdope 21 день тому

    so in terms of consumer groups subscribing to topics in a pub/sub system...
    can you summarize when to use Redis vs Kafka?

  • @Keyboardscholar
    @Keyboardscholar 3 місяці тому

    I really enjoy your content on the website, the videos as well as posts on reddit. Can you make another video on behavior/leadership principles interview next? I would like to learn how to tell a convincing story about my leadership potential.

  • @Nnngao4231
    @Nnngao4231 2 місяці тому

    this video is awesome! exactly what I expected!

  • @rahulbm
    @rahulbm 5 місяців тому +2

    Great content. Really liking it. Can you please do a deep dive on TiDB, Rocks DB and Cassandra too

  • @SrinivasMuthukrishnan
    @SrinivasMuthukrishnan Місяць тому

    Fantastic video, thank you!😊

  • @Marcus-yc3ib
    @Marcus-yc3ib 2 місяці тому

    Hi, you talked about append a random number to solve the hotkey issue. But when you want to retrieve the data, you won't know the key because you append a random number.

  • @kiranpai8
    @kiranpai8 3 місяці тому

    Enjoyed both your videos on Kafka and Redis. Can you please make a video on CQRS?

  • @nishanthooda
    @nishanthooda 5 місяців тому +1

    Great watch! A similar deep dive video for Kafka would be amazing 🙏

    • @hello_interview
      @hello_interview  5 місяців тому +2

      Kafka is in the works! Written resource will be posted to the website this week or early next!

    • @nishanthooda
      @nishanthooda 5 місяців тому

      @@hello_interview amazing!

  • @aanurraj
    @aanurraj 5 місяців тому +5

    Can we have a deep dive on Apache Flink please ?

  • @armanmalik
    @armanmalik 5 місяців тому +4

    Why would I want to use Redis stream over something like Kinesis or Kafka streams

    • @hello_interview
      @hello_interview  5 місяців тому +1

      In practice? Substantially faster and cheaper if you can accept some of the (many) compromises. In an interview setting it can be helpful to have a few multifaceted tools vs having to know all about Kafka (video on that soon).
      If your business was entirely built around append-only logs I probably wouldn't take Redis as an acceptable substitute, though it works in a pinch.

  • @adityakirankorlepara4500
    @adityakirankorlepara4500 5 місяців тому +1

    Excellent video. Please also make video on mock interviews

    • @hello_interview
      @hello_interview  5 місяців тому

      Mock interviews are what we do! What is it you want to see?

  • @SlimJones62
    @SlimJones62 5 місяців тому +2

    Wait .... I thought I clicked on a Dua Lipa music video. 😀. Just kidding. Stefan I love this - clear, concise, articulate and I learned something. Keep 'em coming.

  • @rogermarin1712
    @rogermarin1712 5 місяців тому +1

    Can i book sys design coaching sessions on hello interview or is it only for mock interviews?I'm a senior engineer and I've just started preparing do you recommend to jump straight into mocks?

    • @hello_interview
      @hello_interview  5 місяців тому

      Send us an email at support@hellointerview.com and we might be able to make something work. Have some products planned here for people in your shoes.

    • @rogermarin1712
      @rogermarin1712 5 місяців тому

      @@hello_interview done

  • @ItsMeIshir
    @ItsMeIshir 5 місяців тому

    It's a great deep dive. I'd like to give a suggestion, Can you guys add chapters to the video, so that, if I want to rewatch again about certain portion of the video then that would be easy thing to do.

    • @hello_interview
      @hello_interview  5 місяців тому +1

      You should have chapters on this one! If you look at the description "Key Moments" will take you to the relevant sections. LMK if we're not doing something else we should be doing!

    • @ItsMeIshir
      @ItsMeIshir 5 місяців тому +1

      @@hello_interview I see that, you missed to add "00:00" time stamp in your "Key moments", that's causing YT to not auto chaptering your video. I have checked couple of other videos in the channel, videos which are having 00:00 in their key moments, respective chapters are added.

  • @russbaker6861
    @russbaker6861 Місяць тому

    Would a sorted set implementation be appropriate for a priority queue?

    • @hello_interview
      @hello_interview  Місяць тому +1

      For many, yes. Depends a bit on the scalability requirements.

    • @russbaker6861
      @russbaker6861 Місяць тому

      @@hello_interview Awesome, thanks so much for the quick response. I really appreciate what you guys are doing, this is a great resource and y'all rock!

  • @kunalsharmain
    @kunalsharmain 4 місяці тому

    Loved it! Want to see more on how communication between service on global scale like using kafka with async or consistent connection around this please, if you get the idea, Sorry i am bad at explaining

  • @ganeshkudva817
    @ganeshkudva817 5 місяців тому +2

    Can you please do deep dives on Kafka and Spark too ? I love your videos

    • @hello_interview
      @hello_interview  5 місяців тому

      Written deep dive on Kafka coming in a couple days!!

  • @hazemabdelalim5432
    @hazemabdelalim5432 5 місяців тому

    But how will you handle the at most one delivery of redis in the case messaging?

    • @hello_interview
      @hello_interview  5 місяців тому

      Check out our Whatsapp guide for one example of this: www.hellointerview.com/learn/system-design/answer-keys/whatsapp

  • @nikolagrgic3773
    @nikolagrgic3773 Місяць тому

    Thank you very much

  • @hello_interview
    @hello_interview  5 місяців тому +11

    Re-upload, sorry! Stefan is a UA-cam noob.

  • @刘天旻
    @刘天旻 3 місяці тому

    Thanks so much for the video. I would be really interested in deep dive on elastisearch/opensearch

  • @PrasannaRanganathan6078
    @PrasannaRanganathan6078 5 місяців тому

    Nice! Can you also do Deep Dive on Graph Databases(eg, neo4j) as well. please!

    • @hello_interview
      @hello_interview  5 місяців тому +1

      Will add it to the list! Typically, graph dbs are not as commonly used at scale.

  • @maxvettel7337
    @maxvettel7337 5 місяців тому

    It would be nice to see a Deep Dive video about location database. Pretty hard to understand for me

    • @hello_interview
      @hello_interview  5 місяців тому

      Which part is most confusing? Geohashing? Quad trees? Where to use them?

    • @maxvettel7337
      @maxvettel7337 5 місяців тому

      @@hello_interview Geohashing algorithm is too complex. I just cant take in all this diagrams. I also dont understand how proximity systems can search something inside the circle using square blocks of geohashes

    • @hello_interview
      @hello_interview  5 місяців тому

      @@maxvettel7337 Gotcha! I'll earmark this for follow-up. The nice thing is you don't necessarily need to know about the geohashing internals to use something like Redis' proximity search in practice!

  • @alpacasCodebytes
    @alpacasCodebytes 5 місяців тому

    Amazing Content.. Can't thank you enough!! Any Plans for DSA Mock Interviews 👀

    • @hello_interview
      @hello_interview  5 місяців тому

      What are you looking to see?

    • @alpacasCodebytes
      @alpacasCodebytes 5 місяців тому

      @@hello_interview DSA Mock Interviews with FAANG Candidates..

  • @VIPINKUMAR-dr7vu
    @VIPINKUMAR-dr7vu 3 місяці тому

    Please make a video on Elastic Search

  • @implemented2
    @implemented2 4 місяці тому

    One can imagine the implementation of the geo-spacial index using 2d indexing data structures - quad tree, R-tree and others.

  • @databasemadness
    @databasemadness Місяць тому

    Solid value.

  • @womanwithtoomanyhobbies
    @womanwithtoomanyhobbies 2 місяці тому

    Finding it difficult to understand stream and consumer group

  • @flyingpiggy741
    @flyingpiggy741 4 місяці тому

    Could Even cover Redis again?

    • @hello_interview
      @hello_interview  4 місяці тому

      Nah, this explanation is killer. Couldn’t do it better if I tried!
      -Evan

    • @flyingpiggy741
      @flyingpiggy741 4 місяці тому

      @@hello_interviewPlease

    • @flyingpiggy741
      @flyingpiggy741 4 місяці тому

      @@hello_interview at least we can highlight the knowledge point . Have some summary on the diagram

    • @flyingpiggy741
      @flyingpiggy741 4 місяці тому

      14:40 the nice thing about this stream is if we have got something we need to make sure all of the items of the stream are processed, the redis give us a bunch of premitives to work with. What does this mean?

    • @Richard-yw9if
      @Richard-yw9if 2 місяці тому

      I agree with you . This is the first out of ten video I felt I learned nothing after watching in this channel

  • @Richard-yw9if
    @Richard-yw9if 2 місяці тому

    Why not talking about cache invalidation when using as a cache?

  • @ediancomachio2783
    @ediancomachio2783 5 місяців тому

    I have never clicked so fast!

  • @guidoscalise
    @guidoscalise 5 місяців тому

    Kafka next, please!

  • @joemiller1057
    @joemiller1057 26 днів тому

    For hot key you really need to have caching in the service that's hitting Redis, unless you have a ton of hot keys a small LRU cache should work. Although this trades off some consistency unless you have a real short TTL :)
    - Dynamically sharding keys adds complexity to the application layer
    - Dynamically scaling read replica nodes seems wasteful (you dont need to duplicate the whole node)
    All solutions seem non optimal.

  • @anuragtiwari3032
    @anuragtiwari3032 5 місяців тому

    I see , I like.

  • @adithyar3160
    @adithyar3160 3 місяці тому +1

    this video is too high level..need more simpler deep dive on redis if possible

    • @hello_interview
      @hello_interview  3 місяці тому

      Which part did you miss? Maybe we can point you in the right direction.

  • @flyingpiggy741
    @flyingpiggy741 4 місяці тому

    I missed the co founder now

  • @randyorton06
    @randyorton06 5 місяців тому

    do a deepdive on ethereum latest version, l2 rollups

  • @XxXx-sc3xu
    @XxXx-sc3xu 3 місяці тому

    Format reminds me of acloudguru

  • @ayushjindal4981
    @ayushjindal4981 2 місяці тому

    I had to struggle with the accent. Can I pls request to speak a bit slower and maybe a bit more clear accent, if possible. Thanks!

  • @dafivers4127
    @dafivers4127 2 місяці тому

    brah why do you use so much big words. lmao slow down with the vocabs damn.