The benchmarks should really also include memory allocations. array map/filter create new a new copy of the array every time so even if they look decent on isolated benchmarks they put load on the GC which may introduce lag spikes if you spam them too much.
We should just transpile code and let it choose the best for loop :P maybe allow an annotation for expected loop size or even better let the runtime figure it out and optimize over time due to it running its own measurements ;D These are things we shouldn't be worried about when it could potentially be automated. We should instead be focused on conveying expectations.
@@rothbardfreedom Sure good hardware lets you write inefficient code and have it still run fast, but that's not an excuse to be ignorant. The entire point of benchmarks is to gather performance information. The more relevant information you can gather the better you can make decisions (even if that decision is to waste memory because it doesn't matter in your circumstance)
The most important thing to measure is the performance of your application, not your modules or functions. Measure those once you’ve detected a performance issue with your app, and need to drill down as to why and fix the issue. A slow sum or sort is usually pointless to optimize if its impact on your app is negligible.
I don’t just use a DB… I use an excel spreadsheet, and when it runs of space(which is hardcoded at like 1mil rows), it creates a new spreadsheet with a new ID which is specified in the filename and nowhere else, and use that, all of those files together, instead of a DB. And all of them are .xlsx files btw.
0:24 "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."
Also, if you don't architect for speed, you won't get it. You won't be able to come along later and "optimise the hot spots". Because that's not how applications work. There's this idea that you'll have one or two hot loops l, and you'll just have to focus on those. If you've profiled any real applications you'll see two things: 1. One random getter is causing all the problems. Fix it, and you arrive at point 2. 2. Everything is equally slow, but not really slow. Like everything takes just a few milliseconds and there is no hot spot.
Quicksort is not a stable sort. With a stable sort routine equal items will have the same relative order compared to the original. Quicksort does not have this feature. Most of the time you might not care, but that IS a difference worth noting.
Thank you, and with sort getting faster on strings, it's probably trying to minimize comparisons, something TimSort does in Java and Python, because comparing is more expensive. Plus TimSort is stable.
We've been able to squeeze out more by declaring i before and not using a lookup for the length. I know it sounds terrible, but it's ever so slightly faster. let i = 0; const len = arr.length for (; i < len; i++) Use at your own peril.
I've done similar. As long as you maintain ownership over the array (to ensure the length isn't suddenly shorter than it was when you'd began), this is fine.
I can see .length as having a slight overhead and so making a difference over a big number of loops. But the declaration of i doesn't seem logical to me as that part of the for loop is only run once. Guess I have to try Deno bench now
@@S7hadow i believe the declaration is because scoping rules for loops are different from the rest of the function in javascript. the interpreter also needs to differentiate between which type of for-loop it is, which the semi-colon disambiguates immediately as opposed to reading the entire declaration before knowing which type of loop the declaration is for.
Running the benchmark on my PC, the traditional loop can run 14,680 iters, but the reverse loop only ran 14,570 iters. It's practically the same speed, so I don't think it really matters. I'm guessing that it's a set value that only gets updated when something is added or removed, rather than recalculating every time it's called. Weirdly enough tho, the for..of loop runs 14,840 iters, so it's somehow faster despite being slower in this video. I also added for..in for good measure, and it ran 247 iters, so it's clearly the loser lol
I would hope for normal array types in Javascript that there's no difference between saving the length ahead of time and checking it every time. Feels like an ideal place to optimize on the runtime's part. But if your length is computed by an actual function, then you should definitely save it first since the runtime can't know if your function has intended side-effects.
@almicc yeah, browser runtimes used to be pretty unoptimized. I haven't run any benchmarks in Firefox in a while, so idk about how well it performs, but V8 has optimized this.
8:02 Yes, yes, noooooo! Using something like Deno bench doesn't tell you where bottlenecks in your program are! You need to profile your actual code in-situ. Then if you prove there's a bottleneck somewhere and it is affecting the performance of your program in a meaningful way then you can use bench to try and find faster alternatives, but then you need to test it again in-situ to make sure it actually performs better. You might have misunderstood the shape of the data, or the shape of the traffic to your code and find that your speed up didn't actually change much. You can't always test these things accurately synthetically, and you won't know it until you try it with the real thing :)
The set example is not really accurate. Although I agree that it definitely provides performance benefits over Array.includes() in case of large data but while benchmarking we must also include the one time overhead taken to create the set as we are originally dealing with array data type.
@@aleksander5298 big set is only faster if you actually keep the set up to date, and use it for multiple times, if you remake the set every single time then it's slower.
I disagree. The array is already in memory, so the Set should be too. The objective is to "find an element", not create the original data structure. One-time setup is insignificant on a task that runs millions of times. But just for fun, recreated the Set on each benchmark run and it still performed 20x faster.
@beyondfireship yeah that's what I said, set will be faster, no doubt in that. But the performance gain is not a million times faster if we consider the time to create the set too, and that's what I felt was inaccurate.
As pertains to the first example, repeat after me children : I shall NOT microbenchmark things where the function call overhead overshadows the work being done by an order of magnitude or more. Seriously, this is such a common thing it should be a meme for all intents and purposes. Now, show us a run where the work being done doesn't amount to "basically nothing at all", ie, something a tiny bit realistic...
I think in crazy situations like using JavaScript to essentially sort data as an API, which is 99% of all web services, the function overhead does matter enough to the point that devs need to reduce the use of overhead. I also like to consider the fact that in apps with large enough user bases, the difference between sorting being in the scale of ms doesn't matter to individuals, but if you consider the large scale power consumption, that difference could literally reduce energy consumption to a significant degree. Consider that downloading metadata on NPM is unironically a million dollar problem that costs a lot of energy and disk space when the correct solution is to just copy paste the function you wanted.
Just as an example, I did implement an improvement to an AWS system where we simply descale servers when there's nothing to do, and removed a bunch of legacy promise/lib code and used native promises and traditional for loops. We went from several servers running 24/7 to a couple running for two hours a day. I'm serious when I say the only changes were traditional for loops, native promises, and removing a bunch of lodash/ other lib code. JavaScript is quite literally the only language where function overhead is truly the worst performance impact.
@@almicc Sorry if this sounds harsh, but you can do all that, but cant READ? Function CALL overhead, not function overhead. The cost of calling a function, not the cost of the function runtime. With simplistic code like in the video, the cost of setting up the call and invoking it can be statistically significant, as opposed to vs a function that actually does some realistic workload. Same issue with setting up threads for parallel work, using LINQ, etc. The more work the function does, the less the cost of calling it matters, and obviously vice versa. And the code used in the example will 100% sure run straight from the CPU cache, grossly exaggerating the results. As for the JS particulars, well... JS has a knack for making things worse, but this isn't something JS specific, it's a "general benchmarking" issue, no matter what language you use.
5:20 no fair, your benchmark doesnt account for the compute power to create the Set in the first place. You're not starting from the same place so it's not an accurate comparison
Ideally you'd be using a Set from the get-go instead of an array, then you are starting from the same place. But if it's an all-at-once allocation instead of incremental then sure. This is why benchmarks are not ideal, test your real code to find actual bottlenecks and then you have the real situation.
This is called "micro benchmarking". Problem is - they tell you how fast your module runs in isolation, but not as part of the whole. Example: you have an algorithm that just fits into your CPU's L1 cache, it runs pretty fast, but if you include it into a larger program, suddenly it's very slow because it's not in cache - something else is. I've had many cases where I would benchmark something to perfection, only to find out that in production performance is opposite of what benchmark says. I think microbenchmarking is useful, and it's fun, and it helps you build some confidence about your assumptions. But it's very rarely reliable in the context of a larger application.
The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt. It must be familiar, roughly C-like. Programmers working at Google are early in their careers and are most familiar with procedural languages, particularly from the C family. The need to get programmers productive quickly in a new language means that the language cannot be too radical. Actual Rob Pike quote
@@netssrmrz React isn’t a language but just a view library though, with no strongly typed features (which is a synonym to trying to avoid idiotic mistakes), of course if you not being forced to use ts
You can actually make the for loop even faster by "caching" the length like this: for (let j = 0, len = testArray.length; j < len; j++) { // loop body } But, it would only make sense for very large arrays. Props for the video and course.
That's why TDD is crucial: If you can make produce your code driven from a suite of tests, you can 1 - measure the code and 2 - safely change it. If you don't drive the code from the tests, (2) will always be a shoot in the dark.
Excellent video but Success depends on the actions or steps you take to achieve it. Building wealth involves developing good habits regularly putting money away in intervals for solid investments. Financial management is a crucial topic that most tend to shy away from, and ends up haunting them in the near future.., I pray that anyone who reads this will be successful in life!!
This definitely is the kind of content I'd love to see more, even pay for it! I'm not very interested in Deno though, but if you launch a broad course in which you pass on important concepts like this one, please do advertise it and have my money 💸💸💸
You can also accomplish this using a while loop which should be the fastest way. let arr = [1,2,3,4,5]; let sum = 0; while( arr[0] != undefined ){ sum += arr.shift(); }
@@piff57paff i suppose it does depend on whether the data is always created in bulk or incremental. This is why you don't find bottlenecks via these kind of benchmarks, you time your real code so you know the situation
@@piff57paffyou usually create a collection once but use it many times, so insertion cost is generally not as important as the cost to use the collection.
Premature optimization is no good, but don't write code with poor performance if you know it'll process a lot of data. It's not premature at that point.
@amine7 Could do what a colleague of mine did. Do a big import, loop through the entire file and compare for every potential matching item in the database. That way I could improve the performance by 1000x by just using a hash map.
It's a misleading oversimplification to say the type of loop doesn't matter if the array is small. It might not matter if it runs infrequently. But if it's a part of a high traffic path, then you can have hundreds of small loop executions a second and that would mean the small differences can compound into an impactful performance hit.
You know why I prefer the For Loop? It's explicit, meaning any programmer from the last five decades, coming from almost any language, can understand WTF is going on.
Log things you care about with timestamps, always include millis, ensure clocks are synced if different steps run on different machines, log latency in nanos for things you really want to drill down on, and use good log-analysis tools to understand system-wide latencies. Benchmarking individual functions or sections of code is only useful after you have identified that code to be a major bottleneck in the overall system.
Using a "Set" for checking if an element is contained in an array is probably mostly usefull if you do this over and over. But in those cases I mostly want to get a result from that check so a Map will usually be the structure of choice. Sets are still good to remove duplicates though.
Notably missing here is doing a performance recording on a real world app. That will allow you to see where the bottleneck is. You won't get that information by running some arbitrary small component of your app an unrealistic amount of times in a benchmark. It's beyond me how this is not standard advice when it comes to JS performance. You literally just press F12, find "Performance" tab, hit record, do any sequence of interactions, and just observe where it's spending time. This will point you right to the function calls that are your bottleneck.
I have one easy tip for performance. All array methods suck arse, and are hard to read, just write a for loop. This is the best answer every time all time. There is no question. If you do more than one thing then you'll chain methods without realising that you're looping more than once. Even if you don't chain, you still need to run a callback on methods. The for in/of loops are generators. Therefore the fastest is ALWAYS a regular for loop. I only write for loops because I don't know for sure that the array or object will be small enough, and also for loops are extremely versatile (doing what you usually do with 3 loop methods, in one loop), and readable.
With experience, you can often tell places or things that CAN be problematic if done wrong, so you up-front in design follow rule 5 + 3 (that's the order to think of them) to speculate on which data will my code be working on, what range of sizes will it have, where is it (RAM, disk, over the network) and how can i make sure i get the stuff that needs to be local there in a good performant way by the time i need it. Then you do educated guesses whether each is this a big or small set and how they interact. If you know your problem space that's generally easy, and anything under 1K-10K elements are small unless you do stupid stuff like nested foreach lookups (like demonstrated here). Unless you're making a physics engine, or do stuff over a network, your data processing is what will take time. Foreach looping over a network is a crime against your CPU and user. And in accordance to 1, 2, and 4: Verify your speculative assumptions with some debug timestamps and benchmarks, and unless you're working on 100K+ items or cross products, don't bother thinking about instructions and cache hierarchy (if you do, then look at those where you data volume and time spend actually happens and iteratively tune those). Most my code has been slow because of UI libraries doing stuff that really shouldn't be taking that long, ORMs doing ridiculously stupid things with predicates or accesses over the network, or unoptimized data structures where volumes ended being significantly larger than expected (gotta love tuning those for easily 50-1000x perf boosts in a tuning session).
In the bubble sort implementation, I noticed that you were swapping variables using the array destructuring syntax: [a, b] = [b, a]. For small arrays, this is fine, but if the arrays are large, then creating a new array allocation just for swapping might also cause GC which will slow down the entire bubble sort impl. For benchmarking, it would be better to swap the variables the tradition way by creating a temp variable.
@@RustIsWinningyeah but you can't deny the fact that they're the reason why deno is atleast trying to do better because pre deno 2 there wasn't much reason to use it. And I find bun to be still pretty good for servers and web sockets
@@thatsalot3577 Wrong. Yall just living under a rock and getting benchmarkbrainwashed by twitter influencer andys who are still stuck with the worse ecosystem lmao
@@RustIsWinning there is no wrong or right... compatition is always improves the products.. period. I don't and never says bun is better than deno... deno will be always better at compatibility due to v8 engine
Set.has has some optimizations (most likely completely eliminated) tied to it, if you do nothing with the result it it's like 80 times faster than if you do something. In your example, just having an accumulator n and doing n += +testSet.has(value) (and doing the equivalent in the other test), will significantly reduce the difference between the two. This is especially noticeable if you add a bench test where you do literally nothing (an empty body/semicolon), it has the exact same perf as the set one.
If you can still change the video in your course: Not including the creation in the benchmark is a big flaw because creating a set is slower than an array. Obviously doesn't make such a difference, but it's a mistake and worth talking about
The set example is misleading. Remember, the act of creating a set is always more expensive than linearly searching through an array for a value. It’s only worth it if you are going to do multiple look ups on the same set.
You don't create a new set every time you want to search it though. You create the set once and use it throughout your code. Benchmarks generally exclude setup code (including creating test data)
Irony is people want performance from high level language like JS for which multiple runtime exists, each trying to impmement same stuff very differently from each other. If you really need it, than you won't be using JS or Go or even rust and rely C for close to metal performance. It also has own abstractions so it really drills down to a Software Engineers ability to understand machine at it's core level. Unfortunately alot of devs are not computer engineers or taught formally about computer architecture so squeezing out performance would be done by someone who excels in both relms and most of the world won't even need that kind of optimizations either. Thus compering benchmarks is pretty amature, at the end of the day solution and it's worthiness totally depends in usecase specific implementation. These type of comparison is inevitable because of lack of education on topics thst really matters after some point. And no if you're writing your app in JS for loading millions of records, that's not going to give you performance that a bare metal code will provide. Choose your tools wisely.
In a perfect world you get to choose your tools, but we do not live in a perfect world and sometimes you're just given the tools and have to work with them. While it's worth pointing out that JS should not be your language of choice for performance oriented tasks, it doesn't mean you can't try and make the best of what you have
I thought Set initialization should be included in the benchmark, since the problem stated at 4:43 is "finding a value in an array". Although I feel like for large arrays converting to a set would still be faster.
It is worth saying: Knuth was talking about fiddling with bits in assembly. He was NOT talking about writing your programs in a basically performant way. He was CERTAINLY not saying "don't think about performance at all until later". Rob Pike's rules assume that you are writing your program in a sane way already, rather than in dog-slow nightmare spaghetti. There is a reason that Go is designed the way it is - it's not because all organizations of your program are equally valid and the structure doesn't matter until the end when you measure it. If you write your code in an inherently slow way, then once you measure it, it is too late. When you go to measure it, you will discover that there is no particularly expensive part you can optimize because the entire thing is sludge. Also, quicksort is not a fancy algorithm. It is known and extremely easy to copy. If you are not confident about your ability to google quicksort and write it correctly, you should find another career.
Hard agree about ur last point, this is basic data structures and algorithms...same for the set vs array comparison. They are two different data structures for different use cases. You shouldn't be converting an array to a set, you should have decided between set vs array before coding
Not even. Micro benchmarks aren't super indicative of actual real world run times unless it's something hilariously trivial. Measure the entire application performance then make decisions if it's too slow. From my own experience, the fewer heap allocations the better. Like I'll take slower algorithms that perform significantly fewer heap allocations. Managing the GC pileups from languages becomes significant and results in massive performance penalties when you approach saturation limits. I find that matters more in real world production workloads in the long run.
5:06 Is this not O(n^2), as it's using includes in a loop, which in turn loops every value of an array? Or is it O(n) just because the arrays are different or something?
It's O(n*m). (10000 * 1000) The issue with the set lookup example is that it didnt include the extra overhead for indexing the set - which is O(n) because it has to index each element. On the other hand, the other example in the scenario is actually O(n*m) because you are iterating twice, once over the array with the elements to search for and once for each element in the base list. So the errors in both examples kind of cancel out. Still goes to show that O(n) notation is confusing af and should only be use if absolutely necessary.
Thanks? I’ve been programming for 40 years, and these days I generally don’t worry about performance in the browser or Node, other than code and data size. The sort and search stuff pretty much all happens in the database. So I have to make all the execution time measurements against THAT, and avoid crippling it with stupid crap like ORM. So, yeah, same process, but not the place I run it. Batch C code on files in the 90s. Again, same process.
These examples don't consider the fact that JavaScript is a JITed language. These microbenchmarks are never accurate to how little/much this function would actually get optimized if it was in a real application
Started a new job a few months ago and have fallen in love with maps and sets in combination with the data array. Fuck the users RAM, they want SPEED :D
Fight Club is a good example of flawed programming-fighting a system to solve a personal problem (split personality) that exists only because of the fight itself ;)
The set lookup comparison to array contains isn't fully fair though. There should also be a benchmark version comparing the set creation + N lookups to just running the foreach array search. In most cases though, unless that whole context is in a loop or inner loop (in which case, dafuq?) the cost of making the hash set when N is small won't be noticeable as a part of the user experience, so guarding against a horrible bad perf case if there is scale comes at a low absolute cost when it isn't.
The traditional for loop btw that you wrote isn't that optimised. If you cache the length of the array as a variable it doesn't need to pre-calculate it. There are even crazier optimisations with while loops you can do.
Can you please do a programming terminology explanation video because I miss most of the information because of it and I find a lot of confusing and completely different definitions online
I've never had anything negative to say about any of these videos before, but I thought the explanation of why set lookups are faster than array.includes was kind of poor. Though he mentioned some setup overhead, I can see a lot of newer programmers after watching this video just replacing `array.includes` with `new Set(array).has` and thinking they're going to see a big improvement, and wondering why they didn't. The overhead that he mentioned is going require iterating the array once, so for the set optimization to work, you need be creating the set once before doing multiple checks on it.
When you're creating a new set of 10 000 elements, you're using a lot more memory in comparison to using "includes". It's still a much better solution, but I feel like it should be mentioned at least.
Because it's part of setting up the test data, which is usually excluded from benchmarks. You wouldn't create a new set every time you want to search for something, so it doesn't make sense to include it in the benchmark.
5:40 the analogy between database indexes and O(1) Set lookups doesn't make sense to me. Most database indexes are stored as B-trees or B+trees, and the lookup complexity there is O(log n). So either the time complexity of a Set lookup is also O(log n), or the Set is actually stored as a hashmap - which is what I would've guessed - giving you an average case of ~ O(1).
But if you're really looking to optimise a sorting algorithm, and have solid static typing, and don't want to learn a completely different language... Use a systems language like assemblyscript.
The benchmarks should really also include memory allocations. array map/filter create new a new copy of the array every time so even if they look decent on isolated benchmarks they put load on the GC which may introduce lag spikes if you spam them too much.
And memory reads. I would rather do ten compares than one RAM read.
We should just transpile code and let it choose the best for loop :P maybe allow an annotation for expected loop size or even better let the runtime figure it out and optimize over time due to it running its own measurements ;D
These are things we shouldn't be worried about when it could potentially be automated. We should instead be focused on conveying expectations.
As a counter-point, memory is something you can buy and it's cheap. Time, on the other hand, can't be bought.
@@rothbardfreedom Sure good hardware lets you write inefficient code and have it still run fast, but that's not an excuse to be ignorant. The entire point of benchmarks is to gather performance information. The more relevant information you can gather the better you can make decisions (even if that decision is to waste memory because it doesn't matter in your circumstance)
@@rothbardfreedom he/she is talking about time though
7:52 "you're good enough even when you're not at your best" love this statement, very deep
The most important thing to measure is the performance of your application, not your modules or functions. Measure those once you’ve detected a performance issue with your app, and need to drill down as to why and fix the issue. A slow sum or sort is usually pointless to optimize if its impact on your app is negligible.
Louder!!!
Shout out to those developers, like me, who write gibberish and it works!
I don’t just use a DB… I use an excel spreadsheet, and when it runs of space(which is hardcoded at like 1mil rows), it creates a new spreadsheet with a new ID which is specified in the filename and nowhere else, and use that, all of those files together, instead of a DB. And all of them are .xlsx files btw.
@@ARandomUserOfThisWorld awesome. I use fopen
@@ARandomUserOfThisWorld I just use a json file
💪👑
Us
0:24 "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."
Also, if you don't architect for speed, you won't get it. You won't be able to come along later and "optimise the hot spots". Because that's not how applications work.
There's this idea that you'll have one or two hot loops l, and you'll just have to focus on those. If you've profiled any real applications you'll see two things:
1. One random getter is causing all the problems. Fix it, and you arrive at point 2.
2. Everything is equally slow, but not really slow. Like everything takes just a few milliseconds and there is no hot spot.
Quicksort is not a stable sort. With a stable sort routine equal items will have the same relative order compared to the original. Quicksort does not have this feature.
Most of the time you might not care, but that IS a difference worth noting.
Thank you, and with sort getting faster on strings, it's probably trying to minimize comparisons, something TimSort does in Java and Python, because comparing is more expensive. Plus TimSort is stable.
@@dentjoener python doesn't use timsort anymore, they changed to powersort in 3.11. still stable though
We've been able to squeeze out more by declaring i before and not using a lookup for the length. I know it sounds terrible, but it's ever so slightly faster.
let i = 0;
const len = arr.length
for (; i < len; i++)
Use at your own peril.
I've done similar. As long as you maintain ownership over the array (to ensure the length isn't suddenly shorter than it was when you'd began), this is fine.
@@jerichaux9219 That sounds like an absolutely horrible bug to discover
I can see .length as having a slight overhead and so making a difference over a big number of loops. But the declaration of i doesn't seem logical to me as that part of the for loop is only run once. Guess I have to try Deno bench now
@@S7hadow i believe the declaration is because scoping rules for loops are different from the rest of the function in javascript. the interpreter also needs to differentiate between which type of for-loop it is, which the semi-colon disambiguates immediately as opposed to reading the entire declaration before knowing which type of loop the declaration is for.
also making that a while-loop should be faster than a for loop
The introduction was the most interesting part of this video.
Technically, there is a fifth way to loop: `for (let i=arr.length-1; i>=0; i--)` or `for (let i=0,l=arr.length; i
Running the benchmark on my PC, the traditional loop can run 14,680 iters, but the reverse loop only ran 14,570 iters. It's practically the same speed, so I don't think it really matters. I'm guessing that it's a set value that only gets updated when something is added or removed, rather than recalculating every time it's called.
Weirdly enough tho, the for..of loop runs 14,840 iters, so it's somehow faster despite being slower in this video.
I also added for..in for good measure, and it ran 247 iters, so it's clearly the loser lol
@jmvr yeah, the reverse js loop is an arcane optimization from a more civilized era.
I would hope for normal array types in Javascript that there's no difference between saving the length ahead of time and checking it every time. Feels like an ideal place to optimize on the runtime's part. But if your length is computed by an actual function, then you should definitely save it first since the runtime can't know if your function has intended side-effects.
@almicc yeah, browser runtimes used to be pretty unoptimized. I haven't run any benchmarks in Firefox in a while, so idk about how well it performs, but V8 has optimized this.
8:02 Yes, yes, noooooo! Using something like Deno bench doesn't tell you where bottlenecks in your program are! You need to profile your actual code in-situ. Then if you prove there's a bottleneck somewhere and it is affecting the performance of your program in a meaningful way then you can use bench to try and find faster alternatives, but then you need to test it again in-situ to make sure it actually performs better. You might have misunderstood the shape of the data, or the shape of the traffic to your code and find that your speed up didn't actually change much. You can't always test these things accurately synthetically, and you won't know it until you try it with the real thing :)
The set example is not really accurate. Although I agree that it definitely provides performance benefits over Array.includes() in case of large data but while benchmarking we must also include the one time overhead taken to create the set as we are originally dealing with array data type.
that's why small array is faster than small set, but big set is faster than big array, you edgy boy
@@aleksander5298 big set is only faster if you actually keep the set up to date, and use it for multiple times, if you remake the set every single time then it's slower.
I disagree. The array is already in memory, so the Set should be too. The objective is to "find an element", not create the original data structure. One-time setup is insignificant on a task that runs millions of times. But just for fun, recreated the Set on each benchmark run and it still performed 20x faster.
@@Slackow why would you remake set lmao are you dumb? just use set instead of array, ez, you have skill issue or what?
@beyondfireship yeah that's what I said, set will be faster, no doubt in that. But the performance gain is not a million times faster if we consider the time to create the set too, and that's what I felt was inaccurate.
As pertains to the first example, repeat after me children : I shall NOT microbenchmark things where the function call overhead overshadows the work being done by an order of magnitude or more. Seriously, this is such a common thing it should be a meme for all intents and purposes. Now, show us a run where the work being done doesn't amount to "basically nothing at all", ie, something a tiny bit realistic...
I think in crazy situations like using JavaScript to essentially sort data as an API, which is 99% of all web services, the function overhead does matter enough to the point that devs need to reduce the use of overhead. I also like to consider the fact that in apps with large enough user bases, the difference between sorting being in the scale of ms doesn't matter to individuals, but if you consider the large scale power consumption, that difference could literally reduce energy consumption to a significant degree. Consider that downloading metadata on NPM is unironically a million dollar problem that costs a lot of energy and disk space when the correct solution is to just copy paste the function you wanted.
Just as an example, I did implement an improvement to an AWS system where we simply descale servers when there's nothing to do, and removed a bunch of legacy promise/lib code and used native promises and traditional for loops. We went from several servers running 24/7 to a couple running for two hours a day. I'm serious when I say the only changes were traditional for loops, native promises, and removing a bunch of lodash/ other lib code. JavaScript is quite literally the only language where function overhead is truly the worst performance impact.
@@almicc Sorry if this sounds harsh, but you can do all that, but cant READ? Function CALL overhead, not function overhead. The cost of calling a function, not the cost of the function runtime. With simplistic code like in the video, the cost of setting up the call and invoking it can be statistically significant, as opposed to vs a function that actually does some realistic workload. Same issue with setting up threads for parallel work, using LINQ, etc.
The more work the function does, the less the cost of calling it matters, and obviously vice versa. And the code used in the example will 100% sure run straight from the CPU cache, grossly exaggerating the results.
As for the JS particulars, well... JS has a knack for making things worse, but this isn't something JS specific, it's a "general benchmarking" issue, no matter what language you use.
can't unsee NODE transitioning to DONE 😭
you mean DENO
No he means done but it'll be oden before it because done😊
NODE -> DENO -> ODEN -> DONE 🎉
sort(NODE) = DENO
Learning during an 8 minute ad for a Deno course, love it!
5:20 no fair, your benchmark doesnt account for the compute power to create the Set in the first place. You're not starting from the same place so it's not an accurate comparison
Ideally you'd be using a Set from the get-go instead of an array, then you are starting from the same place. But if it's an all-at-once allocation instead of incremental then sure. This is why benchmarks are not ideal, test your real code to find actual bottlenecks and then you have the real situation.
This is called "micro benchmarking". Problem is - they tell you how fast your module runs in isolation, but not as part of the whole.
Example: you have an algorithm that just fits into your CPU's L1 cache, it runs pretty fast, but if you include it into a larger program, suddenly it's very slow because it's not in cache - something else is.
I've had many cases where I would benchmark something to perfection, only to find out that in production performance is opposite of what benchmark says.
I think microbenchmarking is useful, and it's fun, and it helps you build some confidence about your assumptions. But it's very rarely reliable in the context of a larger application.
Great video. The for loop can be sped up even more if you create the for(let i=0, len=arr.length; i < len; ++i)
"Languages that try to disallow idiocy become themselves idiotic."
-- Rob Pike
sounds a lot like rust to me
great quote. sounds like React to me.
@@okie9025Here they come
The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt.
It must be familiar, roughly C-like. Programmers working at Google are early in their careers and are most familiar with procedural languages, particularly from the C family. The need to get programmers productive quickly in a new language means that the language cannot be too radical.
Actual Rob Pike quote
@@netssrmrz React isn’t a language but just a view library though, with no strongly typed features (which is a synonym to trying to avoid idiotic mistakes), of course if you not being forced to use ts
You can actually make the for loop even faster by "caching" the length like this:
for (let j = 0, len = testArray.length; j < len; j++) {
// loop body
}
But, it would only make sense for very large arrays. Props for the video and course.
That's why TDD is crucial: If you can make produce your code driven from a suite of tests, you can 1 - measure the code and 2 - safely change it.
If you don't drive the code from the tests, (2) will always be a shoot in the dark.
Excellent video but Success depends on the actions or steps you take to achieve it. Building wealth involves developing good habits regularly putting money away in intervals for solid investments. Financial management is a crucial topic that most tend to shy away from, and ends up haunting them in the near future.., I pray that anyone who reads this will be successful in life!!
You're correct!! I make a lot of money without relying on the government,
Investing in stocks and digital currencies is beneficial at this moment.
Job will pay your bills, business make you rich but investment build and wealth long term, the future is coming.
Life is easier when the cash keeps popping in, thanks to jeffery kathryn services. Glad she's getting the recognition she deserves
@@dritazane6837Wow! Kind of in shock you mentioned expert, Jeffrey Kathryn What a coincidence!!
Thank you Lord Jesus for bringing expert Kathryn into my life and my family, $14,120.47 weekly profit Our lord Jesus have lifted up my Life!!!
This definitely is the kind of content I'd love to see more, even pay for it! I'm not very interested in Deno though, but if you launch a broad course in which you pass on important concepts like this one, please do advertise it and have my money 💸💸💸
You can also accomplish this using a while loop which should be the fastest way.
let arr = [1,2,3,4,5];
let sum = 0;
while( arr[0] != undefined ){
sum += arr.shift();
}
The initialization of Set takes some time though, should be included in the benchmark too
Only if the Set is temporary, if you use a Set instead of an Array altogether then it's the same thing.
Even then, there is more overhead during insertion. So you still need to look at it holistically.
@@piff57paff i suppose it does depend on whether the data is always created in bulk or incremental. This is why you don't find bottlenecks via these kind of benchmarks, you time your real code so you know the situation
@@piff57paffyou usually create a collection once but use it many times, so insertion cost is generally not as important as the cost to use the collection.
Using the SET trick to sort is much like using hash map ✅
This Deno commerical brought to you by Deno.
A Deno a day keeps the exception away
Premature optimization is no good, but don't write code with poor performance if you know it'll process a lot of data. It's not premature at that point.
How else would you later make your service 5x faster and create a blog post and a youtube video bragging about it?
@amine7 Could do what a colleague of mine did. Do a big import, loop through the entire file and compare for every potential matching item in the database. That way I could improve the performance by 1000x by just using a hash map.
Just add a wait(1)
It's a misleading oversimplification to say the type of loop doesn't matter if the array is small. It might not matter if it runs infrequently. But if it's a part of a high traffic path, then you can have hundreds of small loop executions a second and that would mean the small differences can compound into an impactful performance hit.
Anton mentioned 🎉
Yes he was. I'm recently creating quite some PRs for his tests 😅
06:49 unfair test! `[69,420].toSorted()` result is `[420,69]` but other sort functions output is `[69,420]`
Love seeing more Deno videos!
For loops are great, it’s always funny to me that people say it’s outdated when it’s just amazing and fast.
You know why I prefer the For Loop? It's explicit, meaning any programmer from the last five decades, coming from almost any language, can understand WTF is going on.
Rob Pike mention ftw
what I was most curious about is the correspondence between the sounds and the iterations of the sorting algorithms....
very nice and weird
Log things you care about with timestamps, always include millis, ensure clocks are synced if different steps run on different machines, log latency in nanos for things you really want to drill down on, and use good log-analysis tools to understand system-wide latencies. Benchmarking individual functions or sections of code is only useful after you have identified that code to be a major bottleneck in the overall system.
Using a "Set" for checking if an element is contained in an array is probably mostly usefull if you do this over and over. But in those cases I mostly want to get a result from that check so a Map will usually be the structure of choice.
Sets are still good to remove duplicates though.
Yearly Beyond Fireship Upload les go 🔥🔥
Make the traditional for loop even faster by assigning a const to arr.length before the loop 🚀
great video. would love to see a version for frontend.
Bro is sold to the DENO cause, we wanna see Bun content as well 😛
Notably missing here is doing a performance recording on a real world app.
That will allow you to see where the bottleneck is. You won't get that information by running some arbitrary small component of your app an unrealistic amount of times in a benchmark.
It's beyond me how this is not standard advice when it comes to JS performance. You literally just press F12, find "Performance" tab, hit record, do any sequence of interactions, and just observe where it's spending time. This will point you right to the function calls that are your bottleneck.
In the sumUsingForLoop you should make arr.length a separate variable to make it faster
The Centauri write in Javascript. The Narn write in Python. The Shadows use C++. The Vorlons use Java.
...and God write in Rust. 😎
@@npc-drew God wrote it all in LISP
I have one easy tip for performance. All array methods suck arse, and are hard to read, just write a for loop. This is the best answer every time all time.
There is no question.
If you do more than one thing then you'll chain methods without realising that you're looping more than once.
Even if you don't chain, you still need to run a callback on methods.
The for in/of loops are generators.
Therefore the fastest is ALWAYS a regular for loop.
I only write for loops because I don't know for sure that the array or object will be small enough, and also for loops are extremely versatile (doing what you usually do with 3 loop methods, in one loop), and readable.
Eh, I'm quite fond of filter and sort methods. They're a PITA to translate to a pure for loop, and the resulting code is hard to read.
You forgot to loop with a while loop.
Probably the exact same as a for I loop
Yep:
var L = arr.length;
while(L--) console.log( arr[L] );
It compiles to essentially the same bytecode as a for loop.
@@Daniel15au Compile? My javascript usually runs in a runtime environment.
@@jakobsternberg1807 V8 engine uses JIT compilation for hot code (code that runs frequently).
One of your best videos! Thanks a lot!
With experience, you can often tell places or things that CAN be problematic if done wrong, so you up-front in design follow rule 5 + 3 (that's the order to think of them) to speculate on which data will my code be working on, what range of sizes will it have, where is it (RAM, disk, over the network) and how can i make sure i get the stuff that needs to be local there in a good performant way by the time i need it. Then you do educated guesses whether each is this a big or small set and how they interact. If you know your problem space that's generally easy, and anything under 1K-10K elements are small unless you do stupid stuff like nested foreach lookups (like demonstrated here).
Unless you're making a physics engine, or do stuff over a network, your data processing is what will take time. Foreach looping over a network is a crime against your CPU and user.
And in accordance to 1, 2, and 4: Verify your speculative assumptions with some debug timestamps and benchmarks, and unless you're working on 100K+ items or cross products, don't bother thinking about instructions and cache hierarchy (if you do, then look at those where you data volume and time spend actually happens and iteratively tune those).
Most my code has been slow because of UI libraries doing stuff that really shouldn't be taking that long, ORMs doing ridiculously stupid things with predicates or accesses over the network, or unoptimized data structures where volumes ended being significantly larger than expected (gotta love tuning those for easily 50-1000x perf boosts in a tuning session).
Are you ok man? You sound a little down compared to your usual. Either way awesome video!
Having built in easy to use bench marking is really what makes fast code. Make benchmarking and debugging common place .
forOf is my goto now a days for looping. One it is one of the faster and it is also easiest to think about in pseudocode.
In the bubble sort implementation, I noticed that you were swapping variables using the array destructuring syntax: [a, b] = [b, a]. For small arrays, this is fine, but if the arrays are large, then creating a new array allocation just for swapping might also cause GC which will slow down the entire bubble sort impl. For benchmarking, it would be better to swap the variables the tradition way by creating a temp variable.
let a = new Array(42);
if (Object.seal) { a.fill(undefined); Object.seal(a); }
Use fixed length arrays if you want to be fast.
Deno is becoming awesome
All thanks to bun..
Our cute little deno couldn't haddle the competition from Js's round buns XD
@@vaisakh_kmHahaha nice copium from the runtime with 500+ segfaults 😂
@@RustIsWinningyeah but you can't deny the fact that they're the reason why deno is atleast trying to do better because pre deno 2 there wasn't much reason to use it. And I find bun to be still pretty good for servers and web sockets
@@thatsalot3577 Wrong. Yall just living under a rock and getting benchmarkbrainwashed by twitter influencer andys who are still stuck with the worse ecosystem lmao
@@RustIsWinning there is no wrong or right... compatition is always improves the products.. period. I don't and never says bun is better than deno... deno will be always better at compatibility due to v8 engine
Set.has has some optimizations (most likely completely eliminated) tied to it, if you do nothing with the result it it's like 80 times faster than if you do something.
In your example, just having an accumulator n and doing
n += +testSet.has(value)
(and doing the equivalent in the other test), will significantly reduce the difference between the two.
This is especially noticeable if you add a bench test where you do literally nothing (an empty body/semicolon), it has the exact same perf as the set one.
If you can still change the video in your course: Not including the creation in the benchmark is a big flaw because creating a set is slower than an array. Obviously doesn't make such a difference, but it's a mistake and worth talking about
I think y'all are thinking that you'd still have an Array, but why? Just use the Set the whole time, no need for the original Array.
The set example is misleading. Remember, the act of creating a set is always more expensive than linearly searching through an array for a value. It’s only worth it if you are going to do multiple look ups on the same set.
You don't create a new set every time you want to search it though. You create the set once and use it throughout your code. Benchmarks generally exclude setup code (including creating test data)
you soab .. i found this channel rn..
dk how much I missed on..
Irony is people want performance from high level language like JS for which multiple runtime exists, each trying to impmement same stuff very differently from each other. If you really need it, than you won't be using JS or Go or even rust and rely C for close to metal performance. It also has own abstractions so it really drills down to a Software Engineers ability to understand machine at it's core level.
Unfortunately alot of devs are not computer engineers or taught formally about computer architecture so squeezing out performance would be done by someone who excels in both relms and most of the world won't even need that kind of optimizations either. Thus compering benchmarks is pretty amature, at the end of the day solution and it's worthiness totally depends in usecase specific implementation. These type of comparison is inevitable because of lack of education on topics thst really matters after some point. And no if you're writing your app in JS for loading millions of records, that's not going to give you performance that a bare metal code will provide. Choose your tools wisely.
In a perfect world you get to choose your tools, but we do not live in a perfect world and sometimes you're just given the tools and have to work with them. While it's worth pointing out that JS should not be your language of choice for performance oriented tasks, it doesn't mean you can't try and make the best of what you have
5:35 , thought it would be fair to include the initialisation of Set from the array within the benchmark, no?
No because you'd start with a set from the beginning, if it's the best data type for your data.
I thought Set initialization should be included in the benchmark, since the problem stated at 4:43 is "finding a value in an array". Although I feel like for large arrays converting to a set would still be faster.
@@nano_sweet I think that's likely a mistake in the name of the benchmark, and it should be "finding a value in a collection".
Deno course!!!
JavaScript developers when iterating over an array is slower than a single equality comparison: 🤯
brb rewriting all my for-loops in our codebase to traditional for-loops
It is worth saying: Knuth was talking about fiddling with bits in assembly. He was NOT talking about writing your programs in a basically performant way. He was CERTAINLY not saying "don't think about performance at all until later".
Rob Pike's rules assume that you are writing your program in a sane way already, rather than in dog-slow nightmare spaghetti. There is a reason that Go is designed the way it is - it's not because all organizations of your program are equally valid and the structure doesn't matter until the end when you measure it.
If you write your code in an inherently slow way, then once you measure it, it is too late. When you go to measure it, you will discover that there is no particularly expensive part you can optimize because the entire thing is sludge.
Also, quicksort is not a fancy algorithm. It is known and extremely easy to copy. If you are not confident about your ability to google quicksort and write it correctly, you should find another career.
Hard agree about ur last point, this is basic data structures and algorithms...same for the set vs array comparison. They are two different data structures for different use cases. You shouldn't be converting an array to a set, you should have decided between set vs array before coding
In databases, the index lookup performance is not O(1), but O(log n) assuming the index uses a B-Tree
Right, but he is not talking about database. it's for Set which uses hashing and is O(1) in almost every time.
Sets are almost always O(1), and the worst case isn't even that bad.
@@pixiedevhe compared the O(1) lookup of a Set to a database index, which as @whyneet pointed out, is not O(1).
Not even. Micro benchmarks aren't super indicative of actual real world run times unless it's something hilariously trivial. Measure the entire application performance then make decisions if it's too slow.
From my own experience, the fewer heap allocations the better. Like I'll take slower algorithms that perform significantly fewer heap allocations. Managing the GC pileups from languages becomes significant and results in massive performance penalties when you approach saturation limits. I find that matters more in real world production workloads in the long run.
at 5:10 for the arr-includes test: Set has a `intersection(B)` method. this should be even faster.
More videos about Deno please!!
5:06 Is this not O(n^2), as it's using includes in a loop, which in turn loops every value of an array? Or is it O(n) just because the arrays are different or something?
It's O(n*m). (10000 * 1000)
The issue with the set lookup example is that it didnt include the extra overhead for indexing the set - which is O(n) because it has to index each element. On the other hand, the other example in the scenario is actually O(n*m) because you are iterating twice, once over the array with the elements to search for and once for each element in the base list.
So the errors in both examples kind of cancel out. Still goes to show that O(n) notation is confusing af and should only be use if absolutely necessary.
Thanks?
I’ve been programming for 40 years, and these days I generally don’t worry about performance in the browser or Node, other than code and data size. The sort and search stuff pretty much all happens in the database. So I have to make all the execution time measurements against THAT, and avoid crippling it with stupid crap like ORM.
So, yeah, same process, but not the place I run it.
Batch C code on files in the 90s. Again, same process.
Like for the DENO course
like because this account is a bot.
You should run benchmarks on different machines. Maybe 100-150 to get some stats
Bless Deno for making this a built-in feature. Such an underestimated aspect for developers!
These examples don't consider the fact that JavaScript is a JITed language. These microbenchmarks are never accurate to how little/much this function would actually get optimized if it was in a real application
All this time i have be using traditional for loops because I was too lazy to learn a cleaner way. Looks like my laziness has paid off!
It matters if the array is large... or if the number runs on that small array is large
Started a new job a few months ago and have fallen in love with maps and sets in combination with the data array. Fuck the users RAM, they want SPEED :D
You should look into sparse sets. They are pretty cool. It's like a faster map
Fight Club is a good example of flawed programming-fighting a system to solve a personal problem (split personality) that exists only because of the fight itself ;)
You forgot to count backwards. Huge difference
these Rob Pike rules are perfect
Write C, not JavaScript! Or Fortran. JavaScript is one of those languages (aside from BASIC) that should never have been invented.
Deno is so winning!! B word could never 😂
I hope there will be a lifetime purchase option in the future as well.
5:16 -> Did you include the time to organise that array? If not, you can't compare the two.
The set lookup comparison to array contains isn't fully fair though. There should also be a benchmark version comparing the set creation + N lookups to just running the foreach array search. In most cases though, unless that whole context is in a loop or inner loop (in which case, dafuq?) the cost of making the hash set when N is small won't be noticeable as a part of the user experience, so guarding against a horrible bad perf case if there is scale comes at a low absolute cost when it isn't.
The traditional for loop btw that you wrote isn't that optimised. If you cache the length of the array as a variable it doesn't need to pre-calculate it. There are even crazier optimisations with while loops you can do.
Can you please do a programming terminology explanation video because I miss most of the information because of it and I find a lot of confusing and completely different definitions online
I've never had anything negative to say about any of these videos before, but I thought the explanation of why set lookups are faster than array.includes was kind of poor. Though he mentioned some setup overhead, I can see a lot of newer programmers after watching this video just replacing `array.includes` with `new Set(array).has` and thinking they're going to see a big improvement, and wondering why they didn't.
The overhead that he mentioned is going require iterating the array once, so for the set optimization to work, you need be creating the set once before doing multiple checks on it.
Think of how hashing is done behind the scenes, not optimal for small dataset
YAY new course!
When you're creating a new set of 10 000 elements, you're using a lot more memory in comparison to using "includes". It's still a much better solution, but I feel like it should be mentioned at least.
Why didn’t you include the array to set conversion in the benchmark for set?
Because it's part of setting up the test data, which is usually excluded from benchmarks. You wouldn't create a new set every time you want to search for something, so it doesn't make sense to include it in the benchmark.
5:40 ... don't tell me JS "devs" don't know about how lists, sets and dicts should be used...
5:40 the analogy between database indexes and O(1) Set lookups doesn't make sense to me. Most database indexes are stored as B-trees or B+trees, and the lookup complexity there is O(log n). So either the time complexity of a Set lookup is also O(log n), or the Set is actually stored as a hashmap - which is what I would've guessed - giving you an average case of ~ O(1).
amazing content!
Best video till date
code simple, test thoroughly, fix it
But if you're really looking to optimise a sorting algorithm, and have solid static typing, and don't want to learn a completely different language...
Use a systems language like assemblyscript.
That's not a system language lol
"scientifically faster" is the important part of the video title.
processors are so fast now'a'days that performance really isn't an issue....unless you're a nurd.