Linq makes a lot of stuff easy with not a lot of code. Premature optimization is the root of all evil ;) Write code with Linq first if Linq makes it easy and then if there are concerns about performance, then run a profiler to see *really* where the performance hotspots are and then worry about those in order of heat. But the focus on Linq here is interesting, thanks!
100% agree with that. I still use LINQ all over the place, it's awesome. I've only removed it in a few places that there is no IO to load level the processing and every microsecond matters, which I understand is a very small usecase.
@@Micz84 that's understandable. Performance and also not being able to be sure precisely enough as to how long it will take to execute in some cases. But for lots and lots of other areas (business, etc.), simplicity of code is a big deal. It's probably going to be a junior developer who looks at this code before long so they need to be able to figure it out easily and maintain it.
Interesting video, thanks. As a general rule I've found that the easier code is for most developers to read and write, the further away from the CPU the code is, the slower it runs, or the more memory it needs; or both.
It would be great if .NET could compile Linq down to efficient code the way that C++ and Rust and Java do with their iterators/streams. It would require breaking changes, I think it would be worth it.
The community has been requesting this for years now. It is clear that this is not a priority nor is it even planned for a specific release, unfortunately.
This would be useful People argue not to worry about this, but if you care about performance you should be conscious of easy come easy go waste in computing cycles. If just using one thing slows things down, you should at least give it some thought instead of discarding it and blaming I/O as the bigger fish to fry
Instead of just checking it yourself you will wait until someone else makes a video about it and then you will accept the presented conclusions as the revealed truth?
@@mabakay I have researched about it before and after writing that comment. I don't take anything as "the revealed truth" without proof/reference. whenever i find anything new i read at least a couple articles about it so i can get my own conclusions. I wanted Nick to make a video about AsParallel because (i think) a lot of people who are new to programming and C# watch his video and learning something that can help writing better code is good, both for viewers and for Nick (because he gets more views). and Nick does these showcases with benchmarking and deep explanation, so his presented conclusions are some of the best conclusions you can find.
6:10 You could do var listCount = _drinks.Count for(var i = 0; i< listCount; i++) to squeeze more performance when using List like collection (it won't help with plain non-wrapped array) . I have no idea why it isn't optimised by compiler.
It would be the same. .Count is a precomputed property in the list class so performance would be exactly the same and actually the compiler would lower the extraction to inline invocation
@@nickchapsas Yeah my exact thoughts! But then i run the test on .net core 31 release build in benchmark .net on 100_000 elements list of string and it took 72 ns for standard version and 48 ns for extracted count which is almost array like performance. I'm yet to find explanation. It look like for arrays it works exactly as expected and it yields the same results. But check it out yourself maybe I've messed something up during my tests!
@@nickchapsas I think it may not be optimised away because you could technically change the list _size in the for loop so when you change the collection in for loop it is probably bad idea to cache it but a good one when you are just iterating over the collection. It looks like micro optimisation though with huge collections the difference might be noticeable. That's only my theory as i couldn't find good explanation in google.
Great video! Because I know someone is going to misinterpret what you are saying, I would have liked to have seen a disclaimer about linq to objects, vs linq to sql.
Another thing to think about is to measure whole application and pipeline performance. Because these days most of our code runs inside highly complicated software solutions consisting of many different layers, as nearly any stack trace we see can tell. :) For example in ASP.NET Core by the time the request goes through Kestrel, then model binding, then routing, then authentication, then other middlewares and filters, including logging, countless scoped service allocations, etc., that your eventually executed business logic has to do plenty of work to make your choice of algorithm matter much. But of course some developers do work in low-level systems, or on specialized microservices with high workloads. Totally depends on the area. Fun fact is that I solved a few advanced HackerRank problems with Linq set operations, while other people were complaining that their hand-looping solutions timed out. So it can be fast too, it's just easy to run into traps, and one has to understand closures and generally what is happening in the background, which makes Linq into a leaky abstraction for those high-performance use-cases.
This is CRAZY!! This kind of seems like a bug to me. I feel like the 2 versions should perform the same. I have been making that assumption for years! Even when I do performance testing, it has never occurred to me that there would be any difference between these 2 statements! It's like you said, Rider (along with all other code suggestion tools) will tell you to simplify the statement.
Wow, I have always thought my handcrafted algorithm would be slower than LINQ just because it looks kinda messy. Thanks for showing us the benchmark stuff.
@@SixOThreeI think general method that takes Action and runs stopwatch on execution is fine to get the idea of performance characteristics. When you need something more accurate the sure go for Benchmark Dot
For me the bigger problem with excessive linq expressions are readability. Might be easy to write expressions but I find it very common that it sneaks in bugs. I have seen many cases where we had to rip the expression appart and rewrite it in order to fix bugs. So dont go overboard unless there is a very specific reason.
Lol, I have always used for(int) loops in my code only because I thought it consumes less time and memory. Although I have never researched on this topic as I thought it is the simplest form of code for the machine to work. As far as understanding a code is concerned, I interpret a for loop code quicker than its alternatives. And today just after watching this video, I now have a benchmark result to support my claim 😁.
As Nick pointed out repeatedly, the performance difference can only matter in hotspots. Most of your code is not a hotspot, so there is no excuse for using ugly, redundant hard to read, low level structures everywhere un-differentially. It's lazy programming.
Good video. I'd suggest that you also make a video about closure / lexical scoping, because these are hidden allocations that further reduce performance. I often see code where the author did not realize at all that Closure was actually used - because anonymous methods and lambdas so easily allow to use variables / parameters of the outer scope.
@@JackMott Yeah, those are situations where you shouldnt use Linq, but by that definition, everything that is not applicable in every scenario is a problem. So everythings a problem. Linq is no problem. Linq is also not the best solution in every situation.
I agree with you.. many places LINQ impact performance but this video would have been more interesting if you have used microsoft linq functions rather than writing custom definition for count..
Nick, you are amazing! Any thought on linq of nested objects vs nested for loops? It looks like there is some hash join logic that reduces the number of entities linq acts upon.
Wow, forget the linq, what I am surprised the most is that foreach takes about 70% more time. I kinda expected it being slower, since it have to figure out how many iterations it needs to do by itself, but this is huge. Also something that I learnt today is that using .where and then .select is not twice the work but is optimized.
I can't remember exactly but if I recall a foreach is just syntatic sugar for a for loop so it will always be slower but yeah the impact is huge depending on the load in the foreach.
This is true. Next video is about lowering and I will be showing that very thing, but it does actually work differently depending on the type of the enumerator.
@@nickchapsas Also when working with Span types you can do ref returns that will bust performance, from what I understand it's as fast as unsafe pointer loop.
Simple. Fast. Correct. In that order. Every time someone is like "is x slow, is y slow". You know what's slow? Me having to decipher your convoluted code mess for that bug that's production critical
Hey Nick. Great video. Is there any way to optimize LINQ queries without those packages you mentioned and without using for loops or foreach loops? Thanks
Not really. You can optimize the way you use them, but by design, if you use the built in linq, there is not much you can do. The count example shows how mix and matching different types of methods can have different performance results so if you care about time more than memory then you can optimize about that be using the time efficient LINQ approaches.
I want to start off with, I truly enjoy this type of benchmark conversations in general. Benchmarking in particular is a strange strange beast. Benchmarking a real application under load in production is much more difficult thing. There is a wonderful talk "Performance Matters" by Emery Berger given at StrangeLoop. There's a version on YT. It might give you some additional ideas how to benchmark the same code different ways. Generally "performance" isn't in the most important thing in an application. I'm just shy of saying if runtime performance is the most important thing why are you using C# to begin with? C, C++, Golang, Rust are making headlines for a reason. And yet languages like Python and Ruby which are MUCH slower are still quite easily building billion dollar companies? I'd rank several things ahead of performance. Most notably testing, readability, good process, timelines, user feedback. In fact when it comes to performance issues in real world applications generally where we see the slow downs is that there are N objects in development and eventually those objects grow. That missing db index becomes apparent eventually or we've got an O(n^2) algorithm that becomes apparent. It's almost never in a direct line O(n) alg. The next most notable thing I see are GC pauses. At that point I might question why are we trying to iterate over 1 million objects? Now, you may say - but we've highlighted a place where we need to improve performance, I sure hope you can reason about the existing code - because I've seen rats nests of code that were so badly written, non-changeable with no tests that any tweaks to it would break something in the system. The level of fear of changing code becomes a driving force in these projects. I've basically banned FOR loops unless truly necessary. We spend A LOT more time reading code than writing coding. Reading FOR loops to understand the looping construct is a huge time sink form a "understand what's going on quickly" aspect. Death by 1000 cuts. Past that, I know Stack Overflow uses static only methods with no (or very limited) dynamic memory allocation specifically for performance reasons - check out talks by Marco Cecconi on Stack Overflow Architecture that are truly awesome. They are definitely pushing the limits of .Net performance. StackOverflow can't render simple templates without a caching layer. They've got real problems. As a final note, in other languages I've seen people code "perf tweaks" for a bit of code that later was unnecessary due to VM upgrades. Is it truly worth it? Also check out TechEmpower Framework Benchmarks for the biggest and best list of web benchmarks.
I think, in the first half of your comment, you are mixing Performance testing and Benchmarking. They are wildly different and they should not be confused. This video only focuses on Benchmarking, not performance testing, only within the scope of a small piece of code that isn't talking to any IO concern. It is highly unlikely that unless you are doing thousands of requests per second, this this will ever be something you even need to look at, but it is good to know about it in case you have to, which is what this video is about. Performance is subjective and contextual. If I'm handling 5000 RPS with 5 instances is it worth fixing that memory allocation and that "slow" operation so I can handle 5025 RPS, or is it easier and cheaper to just scale out to an extra instance of your app and do 6000 RPS? In a the real world, with stateless microservices and containerized applications, scale and performance is really really easy to achieve.
@@nickchapsas For example does the order of the functions matter? Does the name of the method matter? Does the version of .Net matter? Does the number of executions matter? Does the order of the test invocations matter? It certainly might.
At 10:36 on line 62 you create a reference to the item in the array and then don't use it on the next line. Probably a minor/insignificant issue, but thought I'd mention it. Great video though.
i'm using linq for record a lot of data, (170,000, entries) in sqlite i spend 5 hours to record the database, how I can solve it if i want to record in specified column?
Although the main points stand ("It is situational" and "Benchmark to know") there is one thing I think is worth noting: If the situation is LinqToSql, things become very different. So different, it warrants a separate video. I see too many mistakes made (even by myself) related to what happens on the SQL server and what happens locally. Complicating it is how benchmarking on different sizes databases can seriously impact results. And that is yet another topic zo big you can make a separate video out of it.
Agreed that it is a totally different beast. But it looks the same and it smells the same. Some people might get the idea that it is the same. So a mention of it might have been good. Especialy because it illustrates the point "It is situational".
The thing that I always found odd is how the compiler doesn't know how to optimize the LINQ keywords beyond the LINQ functions, least from what I know, given that they're integrated into the compiler you'd think it would at least try to analyze the code, while its understood that every case makes a call to a LINQ function, it would be very easy to inline in a lot of common use cases.
This is only true for raw arrays. If the type you’re iterating is implementing IEnumerable then it will be slower because it is following the current/movenext approach
I was surprised with this. Resharper prompts you to change .Where(lambda).Count() to .Count(lambda). So in this case it is prompting for the worse solution
Your benchmark named ...DrinkNameForLoop assigns drink as _drinks[i] but then you index into the array to test to see if it is alcoholic rather than using the variable. I’m curious to know if the compiler/ jitter optimized this routine or if you left some performance on the table.
@@FilipCordas A foreach over an IEnumerable will not be optimized, but if it's over an array (e.g int[]) then it'll be optimized. There's an article by Ziad Salloum titled "Performance of the loops in C#" on Medium that goes into more detail.
Great video, but it would be nice if you reshoot bloopers instead of adding visual corrections for them. People may "watch" your videos without actually watching them =)
ToListAsync is a EF extension and in reality it is an asynchronous enumerator against the database so it doesn't really have much to do with LINQ as we know it.
The first test you have ".Where().Count()" and the second test is only ".Count()". To me, that says the first test will enumerate the list twice, where-as the second list enumerates it only once.
They are not. the foreach on lists are being lowered to IEnumerable's MoveNext and Current pattern which is totally different from how an array is being lowered, being a native data structure.
Application profiling is what decides what isnt fast enough. We could curse recursion or reflection but why would you when 90% of the time it's not even statistically relevant to your issues.
It is great video (as usual). Just curiocity... Why you explain what 1 micro second is 0.001 of millisecond? Your video for programmers we should know this things :) Thanks again for the great content!
Well having seen the comments in previous videos with benchmarks, you'd be surprised how many people don't know that, mainly because they never had to look for anything less than milliseconds
IMHO, you should always start with LINQ and only change it to more verbose code when absolutely necessary. All of that extra code will bite you in the butt down the road when someone else takes over your codebase or you're working on so many projects that you can't remember all of the nifty little tricks that you used to eek an extra 10 iterations out of each millisecond.
Yes. Performance is contextual. But programming effort is expensive. So I wish you had mentioned that you should almost never pre-optimize. Shoot for developer effort reduction. If something becomes slow then optimize it. 99% of the time you will not need to optimize. 99% of the time you'll benefit from development ease.
Linq makes a lot of stuff easy with not a lot of code. Premature optimization is the root of all evil ;) Write code with Linq first if Linq makes it easy and then if there are concerns about performance, then run a profiler to see *really* where the performance hotspots are and then worry about those in order of heat. But the focus on Linq here is interesting, thanks!
100% agree with that. I still use LINQ all over the place, it's awesome. I've only removed it in a few places that there is no IO to load level the processing and every microsecond matters, which I understand is a very small usecase.
Not in the world of game development. Linq is prohibited in the hot code path.
@@Micz84 that's understandable. Performance and also not being able to be sure precisely enough as to how long it will take to execute in some cases. But for lots and lots of other areas (business, etc.), simplicity of code is a big deal. It's probably going to be a junior developer who looks at this code before long so they need to be able to figure it out easily and maintain it.
And this is also true for everything else, not only Linq
The programmers over at Id Software have a different saying: "premature optimization is the root of all good".
Who is this guy who just keeps popping up in my feed?? This is awesome! Please continue with these types of videos, they're great!
felt the same hehe =) just subed!
Depends on the application, are you coding a satellite YES, are you serving customers at coffee shop NO
Interesting video, thanks. As a general rule I've found that the easier code is for most developers to read and write, the further away from the CPU the code is, the slower it runs, or the more memory it needs; or both.
It would be great if .NET could compile Linq down to efficient code the way that C++ and Rust and Java do with their iterators/streams. It would require breaking changes, I think it would be worth it.
The community has been requesting this for years now. It is clear that this is not a priority nor is it even planned for a specific release, unfortunately.
This would be useful
People argue not to worry about this, but if you care about performance you should be conscious of easy come easy go waste in computing cycles. If just using one thing slows things down, you should at least give it some thought instead of discarding it and blaming I/O as the bigger fish to fry
great video. you could also do another one talking about
AsParallel, how to properly use it and benchmark its performance.
Instead of just checking it yourself you will wait until someone else makes a video about it and then you will accept the presented conclusions as the revealed truth?
@@mabakay I have researched about it before and after writing that comment. I don't take anything as "the revealed truth" without proof/reference. whenever i find anything new i read at least a couple articles about it so i can get my own conclusions. I wanted Nick to make a video about AsParallel because (i think) a lot of people who are new to programming and C# watch his video and learning something that can help writing better code is good, both for viewers and for Nick (because he gets more views). and Nick does these showcases with benchmarking and deep explanation, so his presented conclusions are some of the best conclusions you can find.
It’s rare that I find actual good .NET content on UA-cam. Keep it up!
6:10 You could do var listCount = _drinks.Count for(var i = 0; i< listCount; i++) to squeeze more performance when using List like collection (it won't help with plain non-wrapped array) . I have no idea why it isn't optimised by compiler.
It would be the same. .Count is a precomputed property in the list class so performance would be exactly the same and actually the compiler would lower the extraction to inline invocation
@@nickchapsas Yeah my exact thoughts! But then i run the test on .net core 31 release build in benchmark .net on 100_000 elements list of string and it took 72 ns for standard version and 48 ns for extracted count which is almost array like performance. I'm yet to find explanation. It look like for arrays it works exactly as expected and it yields the same results. But check it out yourself maybe I've messed something up during my tests!
@@nickchapsas I think it may not be optimised away because you could technically change the list _size in the for loop so when you change the collection in for loop it is probably bad idea to cache it but a good one when you are just iterating over the collection. It looks like micro optimisation though with huge collections the difference might be noticeable. That's only my theory as i couldn't find good explanation in google.
Great video! Because I know someone is going to misinterpret what you are saying, I would have liked to have seen a disclaimer about linq to objects, vs linq to sql.
I have mentioned that individually when people asked in the comments. LINQ to SQL has nothing to do with this video.
Another thing to think about is to measure whole application and pipeline performance. Because these days most of our code runs inside highly complicated software solutions consisting of many different layers, as nearly any stack trace we see can tell. :) For example in ASP.NET Core by the time the request goes through Kestrel, then model binding, then routing, then authentication, then other middlewares and filters, including logging, countless scoped service allocations, etc., that your eventually executed business logic has to do plenty of work to make your choice of algorithm matter much. But of course some developers do work in low-level systems, or on specialized microservices with high workloads. Totally depends on the area.
Fun fact is that I solved a few advanced HackerRank problems with Linq set operations, while other people were complaining that their hand-looping solutions timed out. So it can be fast too, it's just easy to run into traps, and one has to understand closures and generally what is happening in the background, which makes Linq into a leaky abstraction for those high-performance use-cases.
Resharper really needs an option to recommend for performance instead of the default recommend for style
bingo. you could actually prefer their recommendation for maintainability or other code style metrics.
Great video and some good stuff in it. Glad to see my where clauses work even better then just using a predicate count
This is CRAZY!! This kind of seems like a bug to me. I feel like the 2 versions should perform the same. I have been making that assumption for years! Even when I do performance testing, it has never occurred to me that there would be any difference between these 2 statements! It's like you said, Rider (along with all other code suggestion tools) will tell you to simplify the statement.
Wow, I have always thought my handcrafted algorithm would be slower than LINQ just because it looks kinda messy. Thanks for showing us the benchmark stuff.
If nothing, I learned today that BenchmarkDotNet exists. I can stop writing my own stopwatch code now.
@@SixOThreeI think general method that takes Action and runs stopwatch on execution is fine to get the idea of performance characteristics. When you need something more accurate the sure go for Benchmark Dot
Talking about for loop performance, what if we use new List(70000) instead of dynamic definition. I believe linq has this optimization.
For me the bigger problem with excessive linq expressions are readability. Might be easy to write expressions but I find it very common that it sneaks in bugs. I have seen many cases where we had to rip the expression appart and rewrite it in order to fix bugs. So dont go overboard unless there is a very specific reason.
Can't wait for the closure video! Amazing content!
Lol, I have always used for(int) loops in my code only because I thought it consumes less time and memory. Although I have never researched on this topic as I thought it is the simplest form of code for the machine to work. As far as understanding a code is concerned, I interpret a for loop code quicker than its alternatives.
And today just after watching this video, I now have a benchmark result to support my claim 😁.
As Nick pointed out repeatedly, the performance difference can only matter in hotspots. Most of your code is not a hotspot, so there is no excuse for using ugly, redundant hard to read, low level structures everywhere un-differentially. It's lazy programming.
Great video, it helps me understand some performance issues we had in some project
Thank you :)
Good video. I'd suggest that you also make a video about closure / lexical scoping, because these are hidden allocations that further reduce performance. I often see code where the author did not realize at all that Closure was actually used - because anonymous methods and lambdas so easily allow to use variables / parameters of the outer scope.
There is a video coming about closures, lexical scoping and also boxing coming.
Awesome video, and very informative!
so, you have tradeoff between high perf and able better support code, and high perf doesn't always win
In games, perf always win. Non linq code is not that terrible readability wise.
This reminds me of Silicon Valley episode where everyone made fun of Richard for brute forcing sorted list. I was like.... I'm using linq...
Linq is no problem. Using the wrong data structure/ using the data structure wrong is a problem.
@@789blablajaja Linq is a problem in the middle of a game loop. Linq is a problem in a web api with sufficiently high traffic.
@@JackMott Yeah, those are situations where you shouldnt use Linq, but by that definition, everything that is not applicable in every scenario is a problem. So everythings a problem.
Linq is no problem. Linq is also not the best solution in every situation.
I agree with you.. many places LINQ impact performance but this video would have been more interesting if you have used microsoft linq functions rather than writing custom definition for count..
@@MiningForPies If you know c# well and your game doesnt need state of the art everything, unity with c# is totally fine.
Nick, you are amazing! Any thought on linq of nested objects vs nested for loops? It looks like there is some hash join logic that reduces the number of entities linq acts upon.
Wow, forget the linq, what I am surprised the most is that foreach takes about 70% more time. I kinda expected it being slower, since it have to figure out how many iterations it needs to do by itself, but this is huge. Also something that I learnt today is that using .where and then .select is not twice the work but is optimized.
I can't remember exactly but if I recall a foreach is just syntatic sugar for a for loop so it will always be slower but yeah the impact is huge depending on the load in the foreach.
This is true. Next video is about lowering and I will be showing that very thing, but it does actually work differently depending on the type of the enumerator.
@@nickchapsas Also when working with Span types you can do ref returns that will bust performance, from what I understand it's as fast as unsafe pointer loop.
Thanks for the insight !
Any chance you know if the count trick works for .Where(predicate).FirstOrDefault() too ?
(nice seed x)
You'd have to try and see. LINQ is full of surprises and I don't remember everything from the top of my head.
Simple. Fast. Correct. In that order. Every time someone is like "is x slow, is y slow". You know what's slow? Me having to decipher your convoluted code mess for that bug that's production critical
yep!
Uhm no. Simpel, correct, fast: in that order. is the original quote.
@@tomheijtink8688 hahaha oops you're right. I typed this when I was way too tired
A very good and interesting presentation.
Hey Nick. Great video. Is there any way to optimize LINQ queries without those packages you mentioned and without using for loops or foreach loops? Thanks
Not really. You can optimize the way you use them, but by design, if you use the built in linq, there is not much you can do. The count example shows how mix and matching different types of methods can have different performance results so if you care about time more than memory then you can optimize about that be using the time efficient LINQ approaches.
I want to start off with, I truly enjoy this type of benchmark conversations in general. Benchmarking in particular is a strange strange beast. Benchmarking a real application under load in production is much more difficult thing. There is a wonderful talk "Performance Matters" by Emery Berger given at StrangeLoop. There's a version on YT. It might give you some additional ideas how to benchmark the same code different ways.
Generally "performance" isn't in the most important thing in an application. I'm just shy of saying if runtime performance is the most important thing why are you using C# to begin with? C, C++, Golang, Rust are making headlines for a reason. And yet languages like Python and Ruby which are MUCH slower are still quite easily building billion dollar companies? I'd rank several things ahead of performance. Most notably testing, readability, good process, timelines, user feedback. In fact when it comes to performance issues in real world applications generally where we see the slow downs is that there are N objects in development and eventually those objects grow. That missing db index becomes apparent eventually or we've got an O(n^2) algorithm that becomes apparent. It's almost never in a direct line O(n) alg. The next most notable thing I see are GC pauses. At that point I might question why are we trying to iterate over 1 million objects?
Now, you may say - but we've highlighted a place where we need to improve performance, I sure hope you can reason about the existing code - because I've seen rats nests of code that were so badly written, non-changeable with no tests that any tweaks to it would break something in the system. The level of fear of changing code becomes a driving force in these projects. I've basically banned FOR loops unless truly necessary. We spend A LOT more time reading code than writing coding. Reading FOR loops to understand the looping construct is a huge time sink form a "understand what's going on quickly" aspect. Death by 1000 cuts.
Past that, I know Stack Overflow uses static only methods with no (or very limited) dynamic memory allocation specifically for performance reasons - check out talks by Marco Cecconi on Stack Overflow Architecture that are truly awesome. They are definitely pushing the limits of .Net performance. StackOverflow can't render simple templates without a caching layer. They've got real problems.
As a final note, in other languages I've seen people code "perf tweaks" for a bit of code that later was unnecessary due to VM upgrades. Is it truly worth it? Also check out TechEmpower Framework Benchmarks for the biggest and best list of web benchmarks.
I think, in the first half of your comment, you are mixing Performance testing and Benchmarking. They are wildly different and they should not be confused. This video only focuses on Benchmarking, not performance testing, only within the scope of a small piece of code that isn't talking to any IO concern. It is highly unlikely that unless you are doing thousands of requests per second, this this will ever be something you even need to look at, but it is good to know about it in case you have to, which is what this video is about.
Performance is subjective and contextual. If I'm handling 5000 RPS with 5 instances is it worth fixing that memory allocation and that "slow" operation so I can handle 5025 RPS, or is it easier and cheaper to just scale out to an extra instance of your app and do 6000 RPS?
In a the real world, with stateless microservices and containerized applications, scale and performance is really really easy to achieve.
@@nickchapsas For example does the order of the functions matter? Does the name of the method matter? Does the version of .Net matter? Does the number of executions matter? Does the order of the test invocations matter? It certainly might.
At 10:36 on line 62 you create a reference to the item in the array and then don't use it on the next line. Probably a minor/insignificant issue, but thought I'd mention it. Great video though.
Yeah I had a popup during that timestamp addressing this thing and I also mentioned the new performance after addressing it outside of the video.
@@nickchapsas D'oh! I didn't spot that - good work!
Hi. What about try catch perfermance.
Hi, on 3.14 u do something and for a second u can see informations over variables (x:Drink). How u do this magic ?
Hmm... raw assembler vs raw code vs ...
Which version of VS was that ? Can you also do a video of extensions that you use in your IDE.
This is not VS. This is JetBrains Rider
Hey Nick, Thank you for your all videos. I just wanted to know which machine do you use for development ? I mean Mac or Windows or Linux?
i'm using linq for record a lot of data, (170,000, entries) in sqlite i spend 5 hours to record the database, how I can solve it if i want to record in specified column?
Well it’s fast to read and write, that’s 75% of the way
Although the main points stand ("It is situational" and "Benchmark to know") there is one thing I think is worth noting: If the situation is LinqToSql, things become very different. So different, it warrants a separate video. I see too many mistakes made (even by myself) related to what happens on the SQL server and what happens locally.
Complicating it is how benchmarking on different sizes databases can seriously impact results. And that is yet another topic zo big you can make a separate video out of it.
LinqToSQL is a whole different thing. It is not actually LINQ but rather an interpreter so it's by no means part of this video.
Agreed that it is a totally different beast. But it looks the same and it smells the same. Some people might get the idea that it is the same.
So a mention of it might have been good. Especialy because it illustrates the point "It is situational".
Looking forward for linq in depth
Do you talk about someplace, or can you elaborate on, struct enumerators in foreach loops?
The thing that I always found odd is how the compiler doesn't know how to optimize the LINQ keywords beyond the LINQ functions, least from what I know, given that they're integrated into the compiler you'd think it would at least try to analyze the code, while its understood that every case makes a call to a LINQ function, it would be very easy to inline in a lot of common use cases.
Very good content!
Hey Nick, very informatvie video. What is this benchmark tool you use?
It's called BenchmarkDotnet. I have a video on it here in the channel
Why is a foreach loop so much slower than a for loop? I always thought that a foreach is also a for under the hood
This is only true for raw arrays. If the type you’re iterating is implementing IEnumerable then it will be slower because it is following the current/movenext approach
I was surprised with this. Resharper prompts you to change .Where(lambda).Count() to .Count(lambda). So in this case it is prompting for the worse solution
Great content, would love to see more indepth talk on this topic.
Your benchmark named ...DrinkNameForLoop assigns drink as _drinks[i] but then you index into the array to test to see if it is alcoholic rather than using the variable. I’m curious to know if the compiler/ jitter optimized this routine or if you left some performance on the table.
There is an annotation in the video addressing that. I rerun the test using the variable and added the results in the annotation
@@nickchapsas Thanks. I didn't notice that when watching on my phone previously.
Do you have any video where to explain about that benchmark tool and how to wire it out?
I sure do. Here you go: ua-cam.com/video/EWmufbVF2A4/v-deo.html
Do you have a repo for this code? Would love to see what packages you're using for data generation and benchmarking. TIA.
Hey Daniel, the source code is available to my Patreons. For data generation I’m using Bogus, AutoFixture and BenchmarkDotnet
@@nickchapsas fair enough. Thanks so much for answering my question! Loving the content.
I always assumed LINQ and foreach iterators were slower. Both are just con convenient.
Foreach is compiled to a for loop under the hood
@@ITR Foreach calls a function that returns the next element in a collection.
Look up the IEnumerable interface if you're interested to know more.
@@e-arafat Not really compiler will optimize a foreach loop when possible. Also it can return a reference pointer in case of Spans.
@@FilipCordas A foreach over an IEnumerable will not be optimized, but if it's over an array (e.g int[]) then it'll be optimized.
There's an article by Ziad Salloum titled "Performance of the loops in C#" on Medium that goes into more detail.
@@e-arafat True as I said the compiler will optimize when possible.
Where can I find theese benchamerk and memoryallocation tools?
I'm assuming JetBrains Rider
What IDE is this? I did some comment trawling but didn’t find an answer
It’s called JetBrains Rider
Best seed :)
absolute True :p
Using sp or linq? Most linq project were updated to calling sps
Now do it with Span for some real magic :)
How can we get the source code?
Check the video description
Great video, but it would be nice if you reshoot bloopers instead of adding visual corrections for them. People may "watch" your videos without actually watching them =)
Which one would be faster if you use ToListAsync() instead?
ToListAsync is a EF extension and in reality it is an asynchronous enumerator against the database so it doesn't really have much to do with LINQ as we know it.
The first test you have ".Where().Count()" and the second test is only ".Count()". To me, that says the first test will enumerate the list twice, where-as the second list enumerates it only once.
They will both use one enumerator and they will both be enumerated only once since there is only one point of enumeration for both.
thumbs up before i even start watching
How come foreach is slower than for? Considering they are both probably lowered to while loop...
They are not. the foreach on lists are being lowered to IEnumerable's MoveNext and Current pattern which is totally different from how an array is being lowered, being a native data structure.
@@nickchapsas Oh, ok. Did not realized that. Thanks for your help. Love your videos 👍
What is the name of the benchmarking library you are using?
BenchmarkDotnet. I have a video about it on the channel
LINQ is the next best thing since sliced bread
Hi Nick, have you considered livestreaming?
I have but it doesn’t work logistically for me. Way too much of a time commitment without any real return on time investment
Application profiling is what decides what isnt fast enough. We could curse recursion or reflection but why would you when 90% of the time it's not even statistically relevant to your issues.
What happen if the customer needs LINQ with Entity Framework
That’s a whole different story. That linq is actually converted to SQL queries so it looks like linq but it serves a different purpose
Hello, may I know which text editor/ IDE you're using?
It's JetBrains Rider
You can't purchase a perpetual license for this? Seems sketchy, Lol
@@alexhorton7845 No C# IDE has a perpetual license. Visual Studio's license is monthyl as well and it's significantly more expensive.
If you need that type of optimization you need c++
keeping those algorithms happy...
Just stumbled across this guy... Now I'm gonna be benchmarking everything..... ffs
Your not considering my performance when I see red and green squigglies lol
.where.count seems just as clean and easy to read as the count.
What benchmark framework is this?
It's called BenchmarkDotNet. I have a video for it if you're interesting in learning more about it.
holding a 73B of memory for 3 sec for example is better than holding 40B for 8 sec
Yeah basically that's exactly what the video is about. Mainly food for thought about something that we take for granted.
and... subscribed : -)
is this tutorial good for a newbie?
This is not a tutorial
@@nickchapsas yes it is you foul scammer!
Great video! You could also make the same comparison where/count using the EF, it'd be nice :)
It is great video (as usual). Just curiocity... Why you explain what 1 micro second is 0.001 of millisecond? Your video for programmers we should know this things :)
Thanks again for the great content!
Well having seen the comments in previous videos with benchmarks, you'd be surprised how many people don't know that, mainly because they never had to look for anything less than milliseconds
Nice video
16 Dislikes... pffffff, on of the Best Videos about why you shouldn't use Linq all the time!
5ms to return a simple result from database? It is not slow, it is glacial.
IMHO, you should always start with LINQ and only change it to more verbose code when absolutely necessary. All of that extra code will bite you in the butt down the road when someone else takes over your codebase or you're working on so many projects that you can't remember all of the nifty little tricks that you used to eek an extra 10 iterations out of each millisecond.
I agree 100%. There are usually so many things you can optimise before LINQ becomes the thing that "slows you down".
Do some project in Unity (engine) and you will quickly learn these things ;-)
shouldn't mix 420 and beer..
Yes. Performance is contextual. But programming effort is expensive. So I wish you had mentioned that you should almost never pre-optimize. Shoot for developer effort reduction. If something becomes slow then optimize it. 99% of the time you will not need to optimize. 99% of the time you'll benefit from development ease.
oh, nevermind. I guess you do say that later in the video.
Great video but please don't do the "Crazy face on the thumbnail" thing
To the microphone listeners, stop this!!!
Could you speak faster bro?
I love your videos and appreciate your effort but your thumbnails make me cringe
😢
@Rolo567 Clickbait thumbnails generate views. When you make your channel, then you can complain about other people's thumbnails.