Thank you so much everyone for watching this video! If you want to talk more about tech, you are welcome to join my Discord channel (5k+ members)! discord.gg/bY6N9z26et How do I get Discord to notice me?! 🤔
Hey Lewis, great video, Today WhatsApp crashed in INDIA for some time, can you dig deep into this issue and post a video explaining this, it would be interesting to know how engineers at these tech companies solve these issues in such a short time span
Hi Lewis! I'm a civil engineering graduate(2018 passed out) but i love coding. Now I'm studying python on my own aiming to study towards data science. Is it possible that I can get a job in the data science area. Or what should I do to simply enter the IT field? Can you give me any suggestions please.
thanks alot for the great content matey.. would love to see video like this .. like how facebook or big social media store their data and how it works so on
Hey Lewis thanks alot for all your videos they've really helped me in my tech journey. Could you please do a mobile app developer roadmap. I'd really appreciate it thank you
Thanks SO much everyone for the kind words ❤️… and the constructive criticism! My camera kept going out of focus… my mistake 😅. Also I will turn down the camera shake
Great video man! I'm a developer myself and this video made me realize I should be reading more articles like the one you based this video on. Great work!
You lost me around 4:57 but well done. By far this is one of your greatest videos. I know it takes a lot to put these together. Again, great job. Keep up the hard work.
@@CodingWithLewis Bro, what fundamental knowledges do you recommend before learn to code?(my aim is to get into big tech companies and become a good software engineer)
I love the way your content is made , flows well , gets the points of the topic clearly and concisely . Just had to give you the credit you deserve really quickly and also mention that i haven't been seeing notifications for your videos? strange ill just check the discord more often .
Great video. Would recommend diving into a Spotify-based topic. Their engineering blog is quite rich with similar examples and could be a good platform to explain basic concepts from.
@8:25 You sure about that? AFAIK reading is much more computationally expensive than writing to a DB, especially in NoSQL databases as the indexes has to be updated when data changes in one place.
You called out the YT algorithm for bringing me here, and uh, yep! Great video! I have been a programmer for 10 years but managed to only skirt around the sides of the database world. This helped clarify that a bit more!
You tell the story with so much joy while I as an engineer imagined a rollercoaster of tears, frustration and susfpense in the face of each one of the engineers working to deliver the migration. As you correcly said. DB Migrations are the worst of our nightmares
This video is gold. Though I wanted you to go more in-depth into this , but I liked it. I want more of this and please go more in-depth. I would love it.
Engineer here who worked on these systems. A lot has changed since 2016. We don't even use Cassandra anymore. We should probably write a new blog post. 😂
this is seriously interesting, as a developer and discord user. I've asked the question 'how the fuck do they do this all?' many many times, and you gave a well explained answer! Amazing work!
this was really really nice bro as an aspiring backend developer seeing stuff like this and knowing how it works on the backend its amazing lol i was glued to the the screen from the start thanks for an amazing video this makes me push more towards my goal of becoming a backend developer thats just amazing💯💯💯💯💯keep up the good work
Great video! I just found your channel, and I'll for sure be going down the rabbit hole. If you're willing to accept some feedback, I would consider significantly reducing (or IMO getting rid of) the amount of simulated camera shake in your profile shots. I personally found the amount used in this video just tolerable, but it was becoming hard to watch at some points. A little goes a long way!
This was an amazing video, I'm a SWE myself, and everyday I've used this damn god send of an app, I've always wondered how it works so well with millions of users, servers, channels and billions of messages. Do ICloud next!
yeah, no. You don't go back to something that works and has been deeply integrated in your infrastructure like this. They either use Cassadra or fix Cassandra but that's what they got for the end of the times
@@javierflores09 That's blatantly false. You're never stuck with something till the end of times. In the tech field, there's always room for improvement.
@@ZombieAssassin7777 developers make improvement, companies don't necessarily go along with them. In production you are not changing your underlying infrastructure unless you have a damned good reason to, just look at banks still using Cobol; if it was a startup which isn't yet as established I'd understand but something like Discord can't really handle the weight of these changes anymore, so yes, you are in fact stuck with it
@@javierflores09 Still wrong, I work at a bank. Yes we still use Cobol and Mainframe, but are working towards moving off of them and also in the process of changing our database configuration.
Wow. This video is awesome. Please make a video on hotstar stack, It was really amazing to see hotstar handling more then 1crore streamers at once during cricket match.
@@Lark703 Well, many ways like the new Android version which is horrible, and checking if your bot's description in the app directory contains "youtube" to unverify it.
@@OatmealTheCrazy In a technical sense, it is just the IOS version slapped onto Android, which means that it won't be optimised for Android making it slow and it has lots of stupid problems like I sometimes have to wait up to a couple of minutes for it to connect. None of these things were issues pre-new-android-version
bruh, my phone's specs are like close to top of the line, yet there's this one single gif that makes Discord run at 10 fps. Not even a problem with the gif itself, Firefox and the Photos app handle it very smoothly. Nice "native performance"
Very informative video showing the ins and outs of a mega data management platforms!!!! I do still wonder what the hell did discord do for their september release on android that made it so much buggier haha Good video overall!
8:24 _"Reading from a database is more computationally expensive than writing to it"_ Huh? That just doesn't sound correct. Writing requires so much overhead like locking and writing to a disk, whereas reading can be as simple as fetching from disk or even from cache (which needs to be invalidated and updated while writing so another overhead). That's a pretty big miss.
I mean we only have about 300k users in our website & little app combined, and we’re having all sorts of issues with our database, and we’re struggling to keep the thing up and running 😂, let alone having 150 million using your service, it must be a challenging day to day task for the engineers at Discord.
Reading is less expensive than writing in a DB :) This is due to the fact e.g that reading can be concurrent with no problem. You said it the other way around
Thank you so much everyone for watching this video! If you want to talk more about tech, you are welcome to join my Discord channel (5k+ members)!
discord.gg/bY6N9z26et
How do I get Discord to notice me?! 🤔
Hey Lewis, great video, Today WhatsApp crashed in INDIA for some time, can you dig deep into this issue and post a video explaining this, it would be interesting to know how engineers at these tech companies solve these issues in such a short time span
Hi Lewis!
I'm a civil engineering graduate(2018 passed out) but i love coding. Now I'm studying python on my own aiming to study towards data science. Is it possible that I can get a job in the data science area. Or what should I do to simply enter the IT field? Can you give me any suggestions please.
thanks alot for the great content matey.. would love to see video like this .. like how facebook or big social media store their data and how it works so on
Hey Lewis thanks alot for all your videos they've really helped me in my tech journey. Could you please do a mobile app developer roadmap. I'd really appreciate it thank you
Thanks SO much everyone for the kind words ❤️… and the constructive criticism!
My camera kept going out of focus… my mistake 😅. Also I will turn down the camera shake
those 'effects' look cool actually
This is what UA-cam was made for. Congratulations, good sir. Bless your work.
Wow, thanks!
ok
I like turtles 🐢
0:21
Subscribed.
Forget about Discord, think about UA-cam 🤯
Literally wouldn’t be able to comprehend 😳😳
down
@@character640p bruh
@@CodingWithLewis a video on it???
Or probably a short
@GGLVXD bro you're familiar... are u from ISTREEM's server?
You should do more of these technical videos. Love the insights . Now I need to choose a weekend project just as an excuse for me to use Cassandra DB😂
Would love to see more of these videos, been studying system design and this is way more interesting than the videos I’ve been watching
Great video man!
I'm a developer myself and this video made me realize I should be reading more articles like the one you based this video on. Great work!
You lost me around 4:57 but well done. By far this is one of your greatest videos. I know it takes a lot to put these together. Again, great job. Keep up the hard work.
"Wow guys look how cool, Discord is logging all of my private messages and everything i say!!"
Oh hey, it's one of those times UA-cam recommends a channel I've never heard of before but turns out to be really good educational content. *Subbed*
Really great video! Loved how technical but easy to understand it is. The camera shake was a bit too much for me tho.
I really like your longer content!
Im glad :) they take sooo much longer to make but love doing it
@@CodingWithLewis But then again it's much more profitable :D
@@CodingWithLewis Bro, what fundamental knowledges do you recommend before learn to code?(my aim is to get into big tech companies and become a good software engineer)
@@jannatgaoshiqqalb3598just get into coding, you'll learn things along the way.
I love the way your content is made , flows well , gets the points of the topic clearly and concisely . Just had to give you the credit you deserve really quickly and also mention that i haven't been seeing notifications for your videos? strange ill just check the discord more often .
Amazing video 😍. Being a beginner in software development, It shows me how much is there to explore in this field.
Loved to watch this video. I remember using Discord as a reference for an A-Level project I did and I ended up reading their development blog
Great video. Would recommend diving into a Spotify-based topic. Their engineering blog is quite rich with similar examples and could be a good platform to explain basic concepts from.
Nice video! Discord is insane. This video reminded me when like 6 months ago discord was down for 1-2 hours.
Love these videos on "tech stories from companies".
Insane video! I found you through your Shorts but I love the longer videos!
Videos keeps getting better
Now this the content UA-cam should push for. Pure gold ♥️
@8:25 You sure about that? AFAIK reading is much more computationally expensive than writing to a DB, especially in NoSQL databases as the indexes has to be updated when data changes in one place.
You called out the YT algorithm for bringing me here, and uh, yep! Great video! I have been a programmer for 10 years but managed to only skirt around the sides of the database world. This helped clarify that a bit more!
Idk why i am thinking so much about this video, what a amazing work you've done here.
👆send a direct message for support and guidance . . .
This is the style I like! Keep up the good work!
always wondered about this! Thanks for making a video about this 👍🏼
No problem!!
You tell the story with so much joy while I as an engineer imagined a rollercoaster of tears, frustration and susfpense in the face of each one of the engineers working to deliver the migration.
As you correcly said. DB Migrations are the worst of our nightmares
👆send a direct message for support and guidance . . .
This video is gold. Though I wanted you to go more in-depth into this , but I liked it. I want more of this and please go more in-depth. I would love it.
Woah, great video mate!
Def nailed this vid Lewis. Follower from the start, you sure grew quick ! Talk about scalability huh!
Keep more videos like this coming. Loved it
Love this type of video! Nice work :)
Amazingly explained!Great graphics and attention to detail.. Make more such vids like this
Great Video! 👍
Greetings from 🇩🇪 :D
Engineer here who worked on these systems. A lot has changed since 2016. We don't even use Cassandra anymore. We should probably write a new blog post. 😂
this is seriously interesting, as a developer and discord user. I've asked the question 'how the fuck do they do this all?' many many times, and you gave a well explained answer! Amazing work!
Soooo Interesting :)
This was awesome, I’m subscribing rn
this was really really nice bro as an aspiring backend developer seeing stuff like this and knowing how it works on the backend its amazing lol i was glued to the the screen from the start thanks for an amazing video this makes me push more towards my goal of becoming a backend developer thats just amazing💯💯💯💯💯keep up the good work
"and what if I fill a bucket with tombstones, wouldn't it lead to resource exhaustion?"
5 seconds later
"Knew it "
UA-cam, you are the only half decent social network left, these bots are so easy to see, please...
Really cool video!
Loved it Lewis, thank you so much!
👆send a direct message for support and guidance . . .
Can't believe an app that I use to frequently is so incredibly complex
So this is why tech companies ask interviewees to invert binary trees
Great video! I just found your channel, and I'll for sure be going down the rabbit hole.
If you're willing to accept some feedback, I would consider significantly reducing (or IMO getting rid of) the amount of simulated camera shake in your profile shots. I personally found the amount used in this video just tolerable, but it was becoming hard to watch at some points. A little goes a long way!
Very interesting video, and the jokes are just a cherry on top!
Amazing explanation mate❤️
This was amazing. Thank you 🙏
You're so welcome!
This was an amazing video, I'm a SWE myself, and everyday I've used this damn god send of an app, I've always wondered how it works so well with millions of users, servers, channels and billions of messages. Do ICloud next!
Beginner friendly and easy to grasp lewis❤
Beautiful. Too much information in too little time. Pure gold
Brilliant stuff, ty for sharing!
Hi Bro, I learnt how Big data to be managed ❤️🔥 keep doing and rocking
Love from 🇮🇳
Love from 🇨🇦
Subbed, great Content.
I already knew this since the very first day they switched to Cassandra but still watched the video, lol.
I am a mechanic engineer and I still enjoy your informative videos
Thank you
NoSQL doesn't stand for "No SQL", but for "Not Only SQL".
Amazing video! 👍
With MongoDB going through many major changes over the years, I wonder if Discord has tried to go back and test the performance of it today.
yeah, no. You don't go back to something that works and has been deeply integrated in your infrastructure like this. They either use Cassadra or fix Cassandra but that's what they got for the end of the times
mongodb is child toy company, if we talk about planet scaling we talk with vitess or cassandra
@@javierflores09 That's blatantly false. You're never stuck with something till the end of times. In the tech field, there's always room for improvement.
@@ZombieAssassin7777 developers make improvement, companies don't necessarily go along with them. In production you are not changing your underlying infrastructure unless you have a damned good reason to, just look at banks still using Cobol; if it was a startup which isn't yet as established I'd understand but something like Discord can't really handle the weight of these changes anymore, so yes, you are in fact stuck with it
@@javierflores09 Still wrong, I work at a bank. Yes we still use Cobol and Mainframe, but are working towards moving off of them and also in the process of changing our database configuration.
Interesting video. I had this question also for a really long time but never took the time to look into it. Now I have the answer
Have to point out that it's "sea-lla" instead of "sky-lla". Chatted with folks from ScyllaDB, and Discord is pretty much on Scylla now.
Very informative insights, and I subscribed 🎉
It's amazing!! how u people are helping the whole world . Thanks for ur existence ☺️
Great video everything well explained
Great video dude.
your cam is constantly moving? It's like you taped while having constant earthquakes and later tried to warp-stabilized it.
I love the way you broke it down but the artificial camera shake was a little disorienting since i was eating while watching
Love ur content tho!
I really liked it! Thanks. Could you make another video but with the Telegram.
mongodb then and mongodb now is total different, and i love how mongodb progressed over the year. until... surreal appear on the surfaces
They use a sqlite3 database on an extra laptop
👆send a direct message for support and guidance . . .
This video is amazing. Loads of information.
Good Way Of Explaining Sir....
This content what I want
Thanks Lewis ☺️
Blazing fast 1 year old messages?
I always have buffering ... its not fast at all for me :(
Wow. This video is awesome. Please make a video on hotstar stack, It was really amazing to see hotstar handling more then 1crore streamers at once during cricket match.
Back in the day, when Discord engineers focused on improving the platform instead of ruining the platform...
I’m out of the loop. How are engineers ruining the platform these days?
@@Lark703 Well, many ways like the new Android version which is horrible, and checking if your bot's description in the app directory contains "youtube" to unverify it.
@@mushroomcraft I'm unsure what's wrong with the new Android version other than maybe seeing too many image previews at once when trying to upload
@@OatmealTheCrazy In a technical sense, it is just the IOS version slapped onto Android, which means that it won't be optimised for Android making it slow and it has lots of stupid problems like I sometimes have to wait up to a couple of minutes for it to connect. None of these things were issues pre-new-android-version
bruh, my phone's specs are like close to top of the line, yet there's this one single gif that makes Discord run at 10 fps. Not even a problem with the gif itself, Firefox and the Photos app handle it very smoothly. Nice "native performance"
7:38 seems like you can test the collision condition before the db reaches scale.
great video!
Very informative, thank you!
Surprisingly, but paperless receipts use a very similar solution, because they operates with even bigger data.
Hi Bro, keep going
Love from Uzbekistan 🇺🇿 (whose grandfather is founder of Algorithm)
why would you mention an article you are summarizing and then not link it in the description...
Thanks for the video!!!
Great video!
Very informative video showing the ins and outs of a mega data management platforms!!!!
I do still wonder what the hell did discord do for their september release on android that made it so much buggier haha
Good video overall!
Afaik they switched to react native from native apps which caused a lot of issues
nice video bro
8:24 _"Reading from a database is more computationally expensive than writing to it"_
Huh? That just doesn't sound correct. Writing requires so much overhead like locking and writing to a disk, whereas reading can be as simple as fetching from disk or even from cache (which needs to be invalidated and updated while writing so another overhead).
That's a pretty big miss.
Why is the “k” hiding behind Lewis’s head in 0:16?
Boners
This is gold!
Thank you for exposing how Cassandra works!!
No problem!!
I mean we only have about 300k users in our website & little app combined, and we’re having all sorts of issues with our database, and we’re struggling to keep the thing up and running 😂, let alone having 150 million using your service, it must be a challenging day to day task for the engineers at Discord.
Would like to see more of how they built it
You should do twitch next, how they moved from REST api to GraphQL in 2017
Reading is less expensive than writing in a DB :) This is due to the fact e.g that reading can be concurrent with no problem. You said it the other way around
2:32 no pun intended 😂
Now do Slack, let's see how they approach similar problems
How is Cassandra non-relational but it stores data with a primary key and a foreign key (timestamp)?
In my mind now: lets build chat app with cassandra
they're doing these amazing stuff yet im using their technology to send memes in general
There is some difference between Cassandra and mongoDB i think something related with the use of MapReduce