Writing C# without allocating ANY memory
Вставка
- Опубліковано 30 бер 2022
- Check out my courses: dometrain.com
Become a Patreon and get source code access: / nickchapsas
Hello everybody I'm Nick and in this video I will show you how you can use features such Span, ReadOnlySpan and stackalloc to write allocation free C# that is very fast. This video will just focus on a single example but I will explain every decision as I go and hopefully you will find areas where you can use the technique you see here to optimize your own code.
Don't forget to comment, like and subscribe :)
Social Media:
Follow me on GitHub: bit.ly/ChapsasGitHub
Follow me on Twitter: bit.ly/ChapsasTwitter
Connect on LinkedIn: bit.ly/ChapsasLinkedIn
Keep coding merch: keepcoding.shop
#csharp #dotnet #performance
Definitely want to see more content like this, thanks Nick for constantly expanding my mind. I just want to say I'm probably about 10 years older than you but I have no shame in saying that you are a great teacher and I learn a lot from you, I always promote your videos at work.
I also promote his video's at work. 😁
Yep, I third this, lol
I equally do share his videos with my colleagues and other programmer friends.
Me Too, also promoting his courses
Great example! Well done Nick!
BTW: when your process runs in 32 Bit mode, the default thread stack size is 1MB. In 64 Bit mode it's 4MB.
Source?
I would be interested in seeing json serialization/deserialization techniques that are more memory efficient. Especially when receiving back large json payloads that need to be deserialized to an object.
exactly what I was thinking about!
I reckon you’d use System.Text.Json as it is built with similar optimisations as this video.
One thing the Primeagen always mentions about garbage collected languages is that the performance gains of having fewer allocations won't show up in spot tests like this, because the garbage-collector pass didn't get timed in the stopwatch.
So the optimized methods clocked in way faster but also shorten the timing on the garbage collector. Not sure if that also applies to C# like it does to JS
No matter how much I think I know, I always learn something from your videos! Thanks for continuing to put out great content!!
the new span and memory types are making the old C programmer in me happy
Prior to 2019, allocation-free code was *essential* in Unity (the game engine), which ran on a 12-year-old version of Mono with single-threaded garbage collection. Having too much garbage would result in stuttering.
To work around this issue, in 2015 Unity introduced "IL2CPP", a compiler which takes the compiled C# IL code and compiled it into C++, and then compiled *that* into a native binary. No, this is not a joke; it's a real thing that still exists.
However in 2019 they finally upgraded to a version of Mono with multi-threaded GC, so IL2CPP is less useful. It's still a good idea to minimize allocations, though.
I don't think that was or is the primary reason for IL2CPP....
Yeah definitely not the only reason for IL2CPP. iOS sticks out, as you can currently ONLY build Unity games for iOS using IL2CPP. Android supports the Mono runtime or IL2CPP.
still an older version of mono, but at least not ancient
Now you have burst wich converts IL/.NET bytecode to highly optimized native code using LLVM, but to get the most of it, you have to write in a subset of c# called high performance c#. It is c# but with no reference types.
Now with .NET 7 we have Native AOT!
Even without it, RyuJit can output some very optimized assembly if you know what you're doing.
Some tips on this video:
1) there is a Guid.TryWriteBytes() method that acts without allocation
2) you can use string.Create() to fast allocate the strings based on Span, instead of calling string constructor
This is amazing! Learnt a lot of things that I’ve heard about but didn’t really know, keep up the good work!
C# is starting to feel more and more like Rust. Thanks for taking the time to walk us through this.
Add first-class sum types and late interface implementation/orphan rules to the lang, and ideally offer niceties like the One True Constructor and a true no-null mode, and C# becomes very attractive indeed
@@berylliosis5250 i give another name to this, feature hell. The language is getting more and more complex, if i want rust, i use rust instead, is far more performant from scratch.
@@rafaelrosa3841 That's a fair point, but the whole "small and simple language" ship has _very_ much already sailed for C#.
As for using Rust instead - sure, but C# has different design constraints that are genuinely better at some things, so providing the "common" things that are really nice makes it nicer to use when necessary. (E.g. for use with Godot, which has a threading model that broadly disagrees with safe Rust)
@@rafaelrosa3841 Rust has the borrow checker and with that another memory management model. C# is and will always be a Garbage Collected Language. This makes for very different ergonomics, ever if they share features.
One's hell is the other persons heaven. The recent additions to the language have all been aimed at reducing boilerplate and reducing error potential. And for all of them, I think one thing applies: If you read the code using these features you'd be hard pressed to assume it does something else than what it does.. And that's imho the most important thing.
@@RedHotBagel I don't agree with this, more features to do the same things differently only will kill the homogeneous code base to a more extensive and hard to read language, for me give a different name to a Apple killing one p (aple) Wich mean now i writing less to describe the same thing will not help, I already know what apple means it's just now have two ways of describe it, and more ways to describe the same thing will increase the learning curve of the language.
I'd definitely like to see more videos like this. Great example in this video. Keep up the great content.
Nick, I really value your content. Every time I listen to you talk about some topic, I realize I don't know enough, but I finish the video knowing more than I did before. Cannot thank you enough
Great video. I learned something very valuable today! Now I've got to look up stack vs heap. Hopefully Nick Chapsas is in the results. :)
Brilliant as usual. Thank you. Your videos push people to make their minds more efficient
This is great video! please do more of those! I never thought about the size of the guid.
I am willing to pay for a course with this level of details!
I wanna see more of those kind of videos for sure, thanks for sharing.
Yes. More like this. Very good optimisation walkthrough.
Wow, I can already think of where to use it at work. Great stuff!
Thank you! For both this video and the one on the Span.
The most insightful UA-cam channel!!
Please more content like this, also Unsafe code :)
Thanks very very much Nick!!!
I never use structs but thanks to this video I will use them a lot more. I am starting to see the real power of structs when you just use them to compute stuff and don't store them in memory. thanks Nick great video!
Thank you for sharing your knowledge!
I really like using Span and Memory when appropriate. I remember using Memory for creating a zip archive from file streams. With a maximum compression I was able to make my application 2.5 times faster because of using Memory (2.5 seconds instead of 6-7)
First of all, I just discovered your channel and I think it is really good and informative. Kudos.
Now, as an old C developer (now a manager :P ) the "allocate memory" concept rang so many bells in my mind. I have a general comment on this. If you want speed, you need to go low. If you do all this you may as well go writting unsafe in my opinion. Still it is great for people to see these kind of tutorials in order to actually see how things work!
Props to you, you really make it look easy!
Another thing worth mentioning is if you want your URL-safe base64 to be RFC 4648 compliant, you need to swap the replacements. Pluses should be replaced with hyphens, not underscores. Slashes should be replaced with underscores.
extremely high level content, thanks!
Well done! Thank you!
Wonderful .. truly high value stuff ... keep it up. 👍
Thank you for teaching us this
This is absolutely great! This'd be an amazing way to optimize azure function apps
Cool content,
Thanks for your great videos
I don't use C# anymore but I'm impressed how far it's gotten since I used it many years ago.
About the constant characters; I'd even consider it worse to create character constants mapping them to their names. You already know that '=' is the equals sign, and it's the glyph itself you care about when replacing them in base 64. It would be good practice if that '=' character represented some special sentinel that acts like a magic value, a special value encoded in the same data type.
I was just going to say the same. It would make sense to define PaddingReplacement, PlusReplacement and SlashReplacement because that's where you're deviating from / overriding the standard and you may want to make it configurable. But this way you just make the reader doubt if the constants actually contain what their names claim to contain.
I assume we all know and agree that const’s provide some benefits including being included in the assembly at build time and also being able to view occurrences referencing the const, allowing easy renaming/ better refactoring as opposed to hard coded literals.
@@NedMorse I agree on the occurrence and refactoring argument, but it doesn't matter if you have a const field, const variable, or simply use the literal. The resulting assembly code is the same. IMO as these constants are private and they are only used once, it is a choice of style, and I tend prefer AlFas's and Benjamin's suggestion.
I'm going to concur with AlFas and Benjamin Schug as well. Characters are self-identifying constants already. If you're going to give them another name, give them the name of what they're representing. Replacing `'=`` with `Equals` is just obfuscation. Replacing it with `Base64PaddingCharacter` tells the reader why you're appending an equals without actually needing to tell them what the character even is.
This is very useful! Thank you for sharing..
I reallly like this kind of content, pitty i didnt run into this channel earlier
This is great! Thanks a lot.
Great video nick !!!!
Really videos like this opens yours eyes on you do wrong 😅. Thank you
Amazing content. More like this please
I write different type of C# apps, worked since VS 2001 with C#, I had to use strange memory allocations twice and that was for a multichannel (80+) audio streaming application.
Awesome content!!
Love how you emphasized "when you need to"...
Great video!
But what if I want to remove some characters from a Span?
And what if I want to replace a sequence of chars with one specific char?
You’re literally god of c#, thanks for existing
very informative!
wow! thats cool! thank you!
Hi Nick. Amazing video as always. Do you have any safe vs unsafe code video for C#? That would really help us folks out. Thanks.
Excelente content!!
well done!
Thanks for great video! I love Span and Memory.
Memory, array, and string builder pools
seeing chained replaces... i think i know where this is going lol. great content.
Great video. I was actually discussing url safe Id's with my professor today say..do you have tutorials for ASP.NET and Memory management in C# I really want to dive into this
Love it.
Thank for the video, Nick.
It’s a pity that Span doesn’t work with async methods. (Can’t be members of heap objects which async state machine is)
The state machine is a struct in release mode, it’s just not a ref struct
That's next level stuff
That youtube link example 🤦♂, it got me
Wow, you took the time to type that out and got punished 🤣
Nice!
Also it would be even faster if you convert to/from base64. It's just a lookup.. You were walking through the bytes/chars for replacement. You may use that time you to convert with a lookup table..
Amazing
MORE PLEASE
I would be interested in seeing if what difference replacing new string() with string.Create() would make in both memory and performance.
Of course it depends on hardware, OS and .NET version, but my tests (i7-9850H, win 10, .net 6) shown that string.Create is slower. No difference in memory allocation.
Hey Nick, How do we preserve benchmarked code and keep code duplication low? Do we create these methods in test project?
is it possible to use Benchmark in existing application to measure parts of code that are not so easy to extract into separate console test app? use case would be to collect measurements of code that you suspect is the root of all evil before refactoring that would allow to benchmark those parts in more isolated way. of course you can use dotTrace or similar profiling tools but maybe Benchmark or other package could be of use as well here?
and as always, this video was great, keep them coming!
I was waiting for a pointer I'm any time, but we still safe.
Tip for writing allocation free code: At least in Rider you can set a different highlight color for struct types. I have structs green and class orange.
Additionally the `Heap Allocations Viewer` plugin is really great.
Just because something is a value type it doesn't mean it will be a stack allocation so the colour highlighting advice is misleading. HAV is good though
@@nickchapsas no, but knowing if something is a value or a ref type is essential to be able to reason about allocation and performance characteristics and having different highlights makes this a lot easier.
I do this too. Most types are the standard teal color, but structs are a neon greenish color. (dark theme).
Will introducing a variable for indexers' calls in switch statements improve the performance?
Looking forward to seeing unsafe version of this optimization
Up until a few weeks ago, it'd been a long time since I'd touched C#, and I've only recently been doing Windows Forms in Mono on Linux. I'd spent a long time working on C/C++ and microcontroller assembly, so seeing these highly performant improvements to C# is very exciting! I'm loving the safe stackallocs to Span! If only it were implemented in Mono. 😢
You don't need Mono. .NET is fully cross platform.
@@nickchapsas Yeah but Windows Forms isn't. And yeah I know Avalonia is a thing, but I just wanted to quickly whip up this personal project in something I was familiar with. (In retrospect, maybe I should've spent the time to learn Avalonia and the newer .NET and C# features, but alas.)
@@c4ashley You can use MAUI instead of WinForms.
@@nickchapsas I wish. It's still unsupported on Linux, and the linux-maui community fork is a total bust in my case. Maybe one day soon. Or maybe if I spend long enough trying to solve installation problems. For now, Mono is fine. A bit buggy, a bit dated, but at least it actually works.
Anyway, this is all tangental; I was just saying I like the features you're pointing out in this video, and I'm excited at the prospect of using them for future (non-Mono) projects! 😀
Hey Nick,
I feel like I've become pretty proficient in safe memory optimizations (which usually is more than enough).
Especially since span got added.
However, I never really trained the unsafe muscle and I was wondering if you can point me towards some good videos / blogs to start looking into that?
Same for me :)
Le dot
Awesome
Tiktokers: Say something without saying it.
Nick: Access something without allocating it.
If i want to optimize my (real time)C# code i'd rather port it to C++. My boss doesnt think that way tho. Thanks for the knowledge
Could you make a video about how dispose works (IDisposable) and how to implement it correctly?
Keep in mind there's also IDisposableAsync. Dispose is used to release resources. Disposable objects should be put in a using statement when possible.
The whole point of IDisposable is to use it in a using statement so it will get called when the using statement ends, even if an exception gets thrown.
Is extracting the strings to constants necessary? The compiler performs automatic string interning on literals and should only allocate once for the life of the program.
It's a style thing. An ordinary idea in programming is to avoid "magic strings" in your methods, and have all constant values at the top. As you said it doesn't do anything at all to impact speed or memory consumption.
@@AndersJohansen92 As a style thing, this video shows exactly how NOT to do it correctly. A couple of points:
- Replacing '/' with `Slash` is akin to replacing `42` with `FourthyTwo`. It doesn't provide any value and definitely doesn't respect the idea behind the no-magic-literals rule which is to make sure that all of your literals are *described* in your code. A better name for `Slash` would be `Base64_Value63` which describes what the code expects the variable to contain.
- The cast to byte should also reuse this variable and not redeclare it from '/' again to highlight the relationship between the char and byte version. In this example, one could decide to replace the '/' with another char but forget to do it on the other way around. This would prevent decoding from working correctly.
- The fact that the compiler already handles this for you is not mentioned in the video. This means that most people will watch this video and think that those const extractions actually helps with the performance of this code, which is a very common myth.
- '+' comes before '/' in base 64, so they should be declared in that order as well. In fact, a proper variable name would also highlight this.
@@EtienneMaheu Awesome points man. This youtuber is a total noob, he should not have a youtube channel. You would do a much better job.
@@ZelenoJabko 🤡
This is because of rule #1: You write code for developers, not for computers. While defining Equals = '=' is kinda pointless, it comes from a good habit where you leave not constant unchecked, so you fan find wherever you use them. Try finding where use used = as a constant with "Find in Files"
1:24 Wasn't the URL I was expecting but still got the expected result.
Do we can update base64span array without creating finalChars? Iterate and update special chars.
I like your low_level optimization videos in particular, but why are there no built-in URI-safe GUID methods which use safe characters as their native lexicon?
very useful, but this way for reference type, have any way for value type ?
Cool!
Is such conversion of guid to bytes correct? What about BE/LE systems? Won't they mess bytes?
More!
Hey Nick, for some of your courses, if you have questions, what is the best way to reach out on those?
Linked in, Twitter DMs or email works
Why did you create a const variable for backslash and plus and such? is it for maintainability?
Great video, but I got a bit lost when you said Span was allocated on the stack (so I thought it generally was) and still used stackalloc (in order to achieve something I thought was already the case?). What am I misunderstanding?
Broadly speaking: Span only contains a 'pointer' to memory and bounds. That memory can still be on the stack or on the heap. Using a stackalloc you allocate memory on the stack, and then create new Span object (also on the stack) that points to that memory
About making this faster:
The conversion of the arrays in the optimised To and From, could be done in parallel..
I guess that needs testing and benchmarked to verify improvement in speed.
Adding multithreading or concurrency here would make it slower due to the workload being small. It's overhead with diminishing returns
Your intro sounded like a aprils fool - glad I knew about Spans and stuck around ;)
Would casting char to byte within the function allocate memory as well? Or you could have worked directly with const char?
Also when was that switch introduced? First time seeing it
Char and byte are value types so there would be no allocation. Switch expressions have been out for a few years now
Having a full grown api already i am curious if this would improve the performance so drastically.
I would think you have the string converted after each call of an endpoint, so basically every Guid you use now would be string something like B2yOjLyEZk0.
Then you'd convert it back to guid and use the guid for querying database. Would be interesting to see how you would implement that into an existing api.
I am also curious if the DI would slow things down once you implemented the function to convert guids.
edit: i just saw you're using static.. so forget the DI.. :D
You will really see a benefit if you are using the conversion in many areas. For example if you were allocating 200 bytes (like in the video) per conversion, then if you’re dealing with 1000 requests per second (which I do more of that) then you’re instantly on 20kb or wasted memory per second which adds up quickly and triggers collection by memory pressure. I think you should profile your apps memory and if that part is hot then optimise it otherwise, ignore it and go for the low hanging fruit. You wanna be pragmatic here
What does it mean "without allocating ANY memory"?
stackalloc initializes stack allocated buffers. Sure, it does not use heap.
"without allocating ANY memory" - it is like your Span/Span exists in virtual address space of virtual address space 😀
You don't allocate. You are using already allocated limited stack memory. No overhead for GC too.
There is obviously no programming without allocations of some sort, but in C# when we talk about memory allocations we talk about heap allocations. This video shows how you can use the Span struct to mostly or solely allocate memory on the stack which is fast and cheap.
@@buraktamturk286 when you are creating (placing) variable in stack it means that some stack memory will be allocated specially for this variable. Variable is unable to exist anywhere in "already allocated limited stack memory", this variable will have concrete address in stack memory, i.e. we have allocating of specially space for variable in stack. Keyword is stackalloc, isn't it? 😀
@@nickchapsas It is OK. Forgive me for grabbing that screaming headline. This is a very good video. Many thanks.
@@PolishchukMaxim well, the stack for running thread is already allocated when that thread is created. So you are using pre-allocated memory. It is just stack pointer that gets adjusted by the amount of data requested. (It is just a simple math operation on stack pointer). Name is for convention only.
17:20 "there is an overload, so this iwll alocate no memory again"
I'm not sure that this is the case. assuming it would NOT allocate memory again, then the string would point internally to memory on the stack, so it would point to memory which won't exist after the mehtod anymore. OR it would mean that this constructor implictly MOVES the memory from the span into the string.
ALSO: the benchmark shows that it will allocate the memory on the heap, as it resulted in 76 Bytes allocated which are: 32 Byte for the empty string + 22* 2 Bytes for each of the characters.
**edit** while listening again to you, i think I've understood you wrong the first time. I thought you meant it would only allocate the 32 Byte for the string, but would not allocate the content of the string.
why extract 'Plus' and 'Equals' to the constants?? can they ever change? Plus become minus or something?
Where can I learn stuff like this? I missed a lot of this kind of stuff in my degree.
Instead of copying the bytes in the Guid into a Span, can't you just cast it into a byte* in unsafe mode?
We don't wanna resort to unsafe code
What's the purpose of putting =, /, - in fields instead of just in the switch?
When using the default case in a switch you do a second array lookup. Would it be faster to use a binding match instead of an underscore? Like char i => i instead of _ => array[pos]
There is no array lookup at all with the switch. It's being lowered to if checks
@@nickchapsas I had the same reaction as @Michael Chen and I must admit I don't understand your answer...
@@Krimog Sorry I misunderstood the question. No that can't happen because list patterns don't work like that. You can't do i => i.
Can you do a video on improving Entity Framework performance?
It's interesting idea, how we can optimize for example update entity which has child and they have own child. What about batch operations and so on
Dapper :)
@@briankarcher8338 🤣🤣
Thanks for the video.
I really appreciated it that they introduced this modern style of the good old stack-based buffer programming into the "safe" area of C# programming language, it's only just a few years ago.
As somebody who also programs in the classical C and C++ languages at certain times, I know exactly what these things mean. Everything is stack-based there by default. Very different from .NET and that makes these languages incredibly performance efficient.
I suspect that many daily C# development may bennefit more often from these things than we are often aware of.
We are all so used to the traditional heap reference based based array's in .NET.
As you said at the end this could lead to stack overflow
This would not lead to stack overflow because the memory is deterministic and extremely small. You should worry about stack overflow on lengths that you cannot control. this example will never lead to a problem. Most of the .NET code behind the scenes uses this exact practice to optimise the code
@@nickchapsas thank you for the explanation, and thank you for that wonderful demo.
For sure this will be useful with embedded boards and less resources computers where the performance is a main part.
The part that was hard for me to understand was the switch { } with the lambda.