Why is Redis so FAST
Вставка
- Опубліковано 3 жов 2024
- Get our 158-page System Design PDF for free by subscribing to our weekly newsletter: bit.ly/bytebyt...
Animation tools: Adobe Illustrator and After Effects.
Checkout our bestselling System Design Interview books:
Volume 1: amzn.to/3Ou7gkd
Volume 2: amzn.to/3HqGozy
The digital version of System Design Interview books: bit.ly/3mlDSk9
ABOUT US:
Covering topics and trends in large-scale system design, from the authors of the best-selling System Design Interview series.
I just love that watching this topic the he says "the last one is a bit unintuitive" while different types of videos would say "the last one is crazy!"
A true intellect
he definietly read the influencer handbook audience retention chapter
let cache = {} //in memory cache 😅
Yep. You just beat redis in one line.
This is great implementation as long as you are running single program on a single machine:)
lol done this. So much easier
You haven't had heap issues due to garbage collection
@@justiceessiel6123 Honestly, I've only done this with static data, like a list of things that won't change much so it's been fine. It's been working in production for 2+years and now it's working in a clustered environment too. I wouldn't do this for large, dynamic data.
You're getting very good at these rapid explanations! Thanks! 😎✌️
Redis is „trying” to implement threads but because it wasn’t designed for that from get go so it’s very hard. It has been years. Dragonfly is redis compatible reimplementation with multithreading and is much more faster.
if memory serves, there was a memtier test between the two. Redis won.
you lack of knowledge of an optimal Redis deployment, for example your assumption is that we are using one instance of Redis process per machine, this is far from true (there is roughly 1 redis process per core in an ideal deployment)
good luck
@@aviadshalom66in your production do u guys get memory for free ?
Dragonfly has an inherently smaller memory usage for the same dataset. Dragonfly is faster at both loading the dataset and producing the snapshot. Dragonfly's memory usage does not grow during the snapshotting process, whereas Redis almost doubles its RAM requirements.
@@aviadshalom66you can’t share memory between OS processes only between threads within same process. Meaning each process would need to keep own of the cache data causing inefficiencies and limit usability of system memory. A distributed cache cluster is of course possible but that’s different and can still cause inefficiencies if multiple nodes run on the same physical hardware.
@@aviadshalom66 Junior level take
@@kavinhonever heard of mmap?
Redis is basically the JavaScript of databases.
It is the Linux Live of OSes.
redis stability sucks with large volume data. we have ton of redis usage and we know how unstable when running in kubernetes pods with hundreds of parallel transactions happens. similar operation on postgresql is great.
You are not supposed to store a lot of data. You're supposed to store data that you only need to access really fast.
Skiplists are under appreciated
maybe but aa-trees definitely are
Basically how JS implements Non-Blocking IO in its REPL, a lot of languages are talking notes from the single threaded style of execution.
Brilliant explanation. Thanks!
thank you for the explanation
Redis is slow. Dragonflydb is fast.
Bro, you are GEM!
also, threads are slow due to the necessary memory allocation associated with a thread. this is a reason why you keep thread pools.
if you think Redis is fast then Aerospike is going to blow your mind
So basically JavaScript’s async event loop technique.
Yeah like u know everything 😂
great nugget on redis. Thanks! how did you create these graphics btw?
Man i love what you are doing. I
have a question and a request.
1: where can I get your free system design ebook please?
2: please do a video on threads, processes, heaps, and stacks whenever you can.
I think there's an Amazon link to the book in the description
@@sourandbitter3062 description? Where?
@@sourandbitter3062also it's not free on amazon
@@rahul169sharmaOn the video in the bottom right there is a button with vertical 3 dots. Click it, and you will see a menu, the first option is "Description".
@@rahul169sharma
blog.bytebytego.com/p/free-system-design-pdf-158-pages
What an amazing video!
make sure your server has enough memory or your OS will store it in the file system by using temporary files :)
The other way around is also the case: if you keep loading the same file and you have enough memory the OS will load the file in memory to avoid reading it from file system.
So Redis works to make it simpler for a developer/devops to not think about it
The best content ❤
all programming languages store variables in RAM
dude should pin comment with full video link on top
The passion mixed with sexuality in his emotions in this man drives me study software engineering more and more!
Cue Salesforce using it and effing it up.
Very good explanation!
Imagine trying to sell being single threaded as something good. At best, it can be not so bad.
great thanks.
Well you left out the biggest issue with Redis. The lack of scalability.
You can’t scale what you can only run on one thread. But only a Software Developer could call the biggest flaw a feature while every System Integrator is calling it the lack of Hardware understanding!
Plenty of companies use redis at scale.
Plenty of companies use NodeJs at scale.
Both have single thread architecture.
@@khalilmohammadmirza4070 and how dose that work for them? I can tell you, not so good!
@@khalilmohammadmirza4070 Plenty of Companies use Microsoft because someone started doing it, that doesn’t mean it was a good idea or that it works particularly good. It only means it works not that it’s good.
@jacobscheit4128 dotnet is the fastest common web framework. Consistently in top 10 in techempower ranking. Using Microsoft has a lot of pros like easier access to Microsoft and Windows ecosystem, very fast server, cross platform development and deployment.
Also, how did Microsoft come into this discussion of single thread architecture.
Comparison memcached and redis in terms of speed (not vatility)
"why is redis so fast?" due to the exploitation of open source labor
I just want 64GiB of L1 cache.. lockable and hardware encrypted.
Ofcourse it’s fast, it’s just using some in memory efficient data structure, it’s like comparing memory access vs disk access (persistent storage). You don’t compare an apple to orange
Consider me a loyal subscriber. Your content is top-notch!
Not open source anymore, use KeyDB.
Really?
@@nitkarshchourasia2406 yes, they switched its licensing from an open source BSD license to both a Source Available License and a Server Side Public License (SSPL).
@@nitkarshchourasia2406yep. They changed their license. Now they are only "source available"
keydb has no helm chart
Nice video
Excellent! Btw, how do you create the animated diagrams? Amazing
You and I have wildly different ideas of what "fast" means.
What do you mean?
@@Akrob55555 one afternoon I rewrote a mini-Redis that was an order of magnitude faster. And it was in TypeScript.
@@nonarKitten And then you woke up.
@@Akrob55555 not at all, I still have the source. I called it 'jedis' and we used it to backend the connection state for a remote client system. Took a little more than a day to debug and flesh out, but it worked really well because we only used a subset anyway.
@@Akrob55555 we later ported that code into Go, because it's better at handling threading, that's when performance really took off.
Wait, what's this redis thing you are talking about? don't you mean Valkey?
Lmao. Yeah but just make a dict/object and spare yourself the dependency and mess
To bad that company stole redis from the maintainer
How do you create the animation?
Yes.
Same concept as NodeJs
Redis is actually not lightning fast, it is one of the slowest k/v stores out there currently, cause it’s single threaded 😂
im guessing you didnt watch the video
How about running multiple redis servers, to mimic multiple threads?
There's a concept called clustering which can get you some of the benefits you're thinking about, but it's most useful for high availability or keeping your data available if one or more nodes go offline.
You can get a read speed boost by distributing reads to all the nodes but writes still have to be handled by a primary node for consistency and then synchronized to the others.
node balancing ?
KeyDB be like: Poser!
i prefer rocksdb KeyDB
There are a few other redis drop in replacements which are slightly faster in different scenarios.
Either way drop in replacements make everybody’s lives easier if you ever have performance issues.
Can you recommend one for windows os?
So wait - when it receives the requests, it sends that request do the Event Loop? What in that diagram reads from the Event Loop and processes the requests?
How such animation are made ?
Yes
Sorry where do I grab the free book?😅
Redis is the slowest Redis implementation 😂 Garnet and Dragonfly are faster
Do you have a merch store with the systems design graphics in posters/hoodies?
How do they make this animation?
Awesome ❤
Small suggestion: ALWAYS USE SUBTITLES. Period.
How can I get the ebook
Alex, my Dev Monk
Does it ever overloads a computer’s RAM? Sorry if the questions is dumb,I am not a techy person.
But wondering if someone searches loads of complex queries a day and then searchers for them again but slightly different, how will ri process it? Storing a brand new search or adapting the old one?
Sorry again! lol but trying to understand this thing here
Beautiful
Is there any lock in the multiple inputs?
wich software did you use for the animation in this vid?
x2
@@slvdr510 can you give me the link to download it
not Redis
@@slvdr510 can you reply?
@@kaushalagrawal6258 what is the name of the software used to make this animation
How do I get the book?
How is it much faster than a 3k solid state partioned or indexed disk?
Cause RAM is still much much faster then SSD. Otherwise we wouldn't have a need for RAM memory in our computers ;)
What if Redis stored data in the GPU memory ? Would that be even faster ?
No, the latency will be much higher
Link to system design big archive book?
So it doesn't use all cores but all ram of my server?
Link is broken (
How do I become a master at data stuff
Learn
@@aiueo8962 I can learn but what’s the best way to practice so I can upgrade my job?
practice
Redis is slow af
Better solution?
Yep. A dict/object is faster and 0 dependency
Can this be used in game development to optimize complex game object interactions between mutable objects?
No, it is better for multi instance server software. Maybe useful for game server.
Why he is talking like michio kaku ? Is that cause he is Asian?
cool! your diagrams now get animated! could you please at least give us a clue?
I did not understand a single word
it is as if they engineered the most hard to listen to person & made him a teacher...
😂😂😂 CouchbaseDB or ScyllaDB, not Redis 😂😂😂
Can you please, explain it further?
System design
What an amazing video!