very interesting, thanks for doing rust! I wouldn’t have guessed what it was doing extra. now you are making me wonder what if we wrote the code without using the stdio library (which is just a wrapper to system calls) and did the system calls directly..
Forgetting to free memory during a crash does not leak it, the operating system clears it up. However Rust does try to enforce the guarantee that everything gets shutdown (dropped) properly to terminate network requests, clean up temporary files, clear security-related data and similar things.
I would be intrested in seeing if statically linking libc musl into rust and build with panic abort would reduce the amount of syscalls. Half the syscalls here appear to be done by ldd and not the actual program itself. A statically compiled programm does not use ldd.
Dang I wrote an ASM program that war only slightly more complex than this and it used 3 syscalls (fstat, mmap, print). EDIT: I recounted and it's 5, although 2 of them wouldn't have been needed for this (exit and close).
As a new grad (working for non system-related program), I have been always wondering how I can always get interested in these types of topics and get more understanding about these topics. How did you learn and improve through your life? I think this is more what I need to become better engineer or programmer to understand what it is exactly happen. Can you give me some advice?
To add on, I think recent university has more focus on ML not system so much... I think it is interesting, buuuut, if I understand that correctly. If you really make "new" model to change the world, we should make new equations or algorithms. That part is not fascinating to me to be honest because of the pressure I will get. That is like music or art area in my opinion. I love logics more than probability... So I think I didn't build good start point from university...
Hey Kimsdo!! you are making deep questions that require careful and comprehensive responses!! feel free to reach out through discord if you want to talk about this, but here's the simplest answer To the first part of your question "How did you learn and improve to your life?" I always focused on becoming good on the things that: 1. Paid well 2. Nobody wanted to do. This started when iOS came out, all the trolls said that iOS was a toy and that real software only existed on the blackberry... so I focused on iOS. The strategy worked and I was able to get a job at GM because I was among the best iOS developers. Then autonomous vehicles (AVs) came around, nobody wanted to learn because they said that AVs were only hype, and that people will always drive... so I learned about AVs. I was assigned to Cruise Automation a company in California which is among the most advanced companies in the world. Fast forward to 2020, I recognized the value of social media, so I learned. It took me longer to pick it up, but here we are. It took me longer than I wanted to monetize and learn the trade, but now I am increasing my income every month through UA-cam and hopefully helping developers all over the world. Regarding your second question, " If you really make "new" model to change the world, we should make new equations or algorithms." before trying to make new equations, get your ML basics right, understand perceptrons, simple regression, gradient decent, then neural networks. You wont be able to innovate in the field of ML until you understand the basics. lastly, " I love logics more than probability... So I think I didn't build good start point from university..." after you learn ML properly, you'll understand that the world is NOT deterministic, it is probabilistic, and that will help you to think about new models. I sincerely hope that this helped, feel free to reach out on discord to chat some more.
@@dario.lencina I thought the kernel was aware of the memory used by a process, so as soon of the process die, the memory can be retrieve (that why you don't want zombie). But I'm not an expert
@@julesoscar8921 I think we are both right, unwinding panics does return the memory to the OS, and also, the OS keeps track of all the memory allocation so it can reclaim all that memory back regardless if you free it or not
do these numbers count calls from the dynamic linker? (would that even matter for comparisons anyway?) edit: nvm, seconds after sending this he notes ld.so in some of the calls
frankly loading pthreads for no reason is a fail. modern c is pretty trash too, and elf binaries do no service to this problem. this program should be 3 words or less.
signals don't need threads in c you should try to add libpthread to your empty c bin and still compare the calls also the original video by Hussein makes no sense imo most of these system calls are only called once and the cost will be amortized over thte long run for something like a webserver a better approach would be to open and read form a file 10000 times tom compare the calls or onlt strace after the initial startup but it's harder to do
Dumb to think it’s ok to do a lot of stuff when it can be done with less. The good question was risen in the original video, isn’t it a good time to take a step back, sit and rethink the approaches. Combining calls? Adding some one-means-a-bunch calls to the api? Other ideas? I’m not a kernel developer and know nothing about that but those questions seem reasonable.
Very interesting analysis. I understand the need for system calls for Memory Management, Security, Loading modules, and basic IO streams during the startup. But if a large number of system calls are a performance hit, what we can do is reduce the number of system calls in the runtime. I am currently playing with IO_URING on Python's async await architecture, and it does reduce the number of system calls significantly for large IO loads like servers. Python's default asyncio library uses a select-based reactor model which I always felt useless because it doesn't work on files. This can be achieved with any language. I don't think any modern language is truly superior just based on the least amount of system calls while ignoring other benefits.
This is a fact ^^ as devs we need to analyze what we get from the runtime, else we would be all programming in assembly without an OS to squeeze the last cycle of the cpu
What you said doesnt make sense at all. Rusts memory safety happens in compile time not runtime. And tell me which system call improves safety? You just said bunch of nonsense. And additionally you didn't remove the includes in c code.
very interesting, thanks for doing rust! I wouldn’t have guessed what it was doing extra.
now you are making me wonder what if we wrote the code without using the stdio library (which is just a wrapper to system calls) and did the system calls directly..
You are right! not using stdio should save a few calls!!
Great videos @hnasr @dario.lencina!
I think we should have a repo with examples like these with syscall analysis for different languages.
@@kartik4792 I love your idea!!
Forgetting to free memory during a crash does not leak it, the operating system clears it up. However Rust does try to enforce the guarantee that everything gets shutdown (dropped) properly to terminate network requests, clean up temporary files, clear security-related data and similar things.
Excellent 😄
Keeping a socket or file ope, not flushing.
so it's just rust added more steps, to prevent the user from making the system crash, interesting :) 💡
Yes! Saving devs from
Themselves!!!
Maybe or maybe not. Can it happen compile time and not runtime if not everything but maybe some portion of it?
That's probably just garbage code generated by rust
I would be intrested in seeing if statically linking libc musl into rust and build with panic abort would reduce the amount of syscalls. Half the syscalls here appear to be done by ldd and not the actual program itself. A statically compiled programm does not use ldd.
I gotcha
Dang I wrote an ASM program that war only slightly more complex than this and it used 3 syscalls (fstat, mmap, print). EDIT: I recounted and it's 5, although 2 of them wouldn't have been needed for this (exit and close).
Did you write in sarcasm64 or did you actually write it?
Pretty sure you can at least half the number of system call in C
Aw that is for sure mate
What if you build a static binary instead ?
That is a good point, maybe it does not need to reach out for dynamically linked libs?
@@dario.lencina exactly, but maybe I'm wrong. I'm kinda curious to see the result
very interesting ! Thank you for the video
My pleasure!
As a new grad (working for non system-related program), I have been always wondering how I can always get interested in these types of topics and get more understanding about these topics. How did you learn and improve through your life? I think this is more what I need to become better engineer or programmer to understand what it is exactly happen. Can you give me some advice?
To add on, I think recent university has more focus on ML not system so much... I think it is interesting, buuuut, if I understand that correctly. If you really make "new" model to change the world, we should make new equations or algorithms. That part is not fascinating to me to be honest because of the pressure I will get. That is like music or art area in my opinion. I love logics more than probability... So I think I didn't build good start point from university...
Hey Kimsdo!! you are making deep questions that require careful and comprehensive responses!! feel free to reach out through discord if you want to talk about this, but here's the simplest answer
To the first part of your question "How did you learn and improve to your life?"
I always focused on becoming good on the things that:
1. Paid well
2. Nobody wanted to do.
This started when iOS came out, all the trolls said that iOS was a toy and that real software only existed on the blackberry... so I focused on iOS.
The strategy worked and I was able to get a job at GM because I was among the best iOS developers.
Then autonomous vehicles (AVs) came around, nobody wanted to learn because they said that AVs were only hype, and that people will always drive... so I learned about AVs. I was assigned to Cruise Automation a company in California which is among the most advanced companies in the world.
Fast forward to 2020, I recognized the value of social media, so I learned. It took me longer to pick it up, but here we are. It took me longer than I wanted to monetize and learn the trade, but now I am increasing my income every month through UA-cam and hopefully helping developers all over the world.
Regarding your second question, " If you really make "new" model to change the world, we should make new equations or algorithms." before trying to make new equations, get your ML basics right, understand perceptrons, simple regression, gradient decent, then neural networks.
You wont be able to innovate in the field of ML until you understand the basics.
lastly, " I love logics more than probability... So I think I didn't build good start point from university..." after you learn ML properly, you'll understand that the world is NOT deterministic, it is probabilistic, and that will help you to think about new models.
I sincerely hope that this helped, feel free to reach out on discord to chat some more.
Are you sure you need to unwind to retrieve the memory?
Yes, you do have to unwind the error to **return the memory to the os
@@dario.lencina I thought the kernel was aware of the memory used by a process, so as soon of the process die, the memory can be retrieve (that why you don't want zombie). But I'm not an expert
@@julesoscar8921 I think we are both right, unwinding panics does return the memory to the OS, and also, the OS keeps track of all the memory allocation so it can reclaim all that memory back regardless if you free it or not
do these numbers count calls from the dynamic linker? (would that even matter for comparisons anyway?)
edit: nvm, seconds after sending this he notes ld.so in some of the calls
That is right, in the bigger schema of things it doesn’t matter 😆
frankly loading pthreads for no reason is a fail. modern c is pretty trash too, and elf binaries do no service to this problem. this program should be 3 words or less.
What problem tho? syscalls are a solved issue. It isn't the cause of real slowness on a computer newer than 1991.
signals don't need threads in c
you should try to add libpthread to your empty c bin and still compare the calls
also the original video by Hussein makes no sense imo
most of these system calls are only called once and the cost will be amortized over thte long run for something like a webserver
a better approach would be to open and read form a file 10000 times tom compare the calls
or onlt strace after the initial startup but it's harder to do
Comprehensive response and even better profile pic 😄 did you see the part where I mention that less calls does not mean “better”?
Dumb to think it’s ok to do a lot of stuff when it can be done with less. The good question was risen in the original video, isn’t it a good time to take a step back, sit and rethink the approaches. Combining calls? Adding some one-means-a-bunch calls to the api? Other ideas?
I’m not a kernel developer and know nothing about that but those questions seem reasonable.
04:05 you literally didn't remove the includes, how is that an apple to apple comparison?
Good point, I’ll do it then report the result back
which version of rust are you using? i ran the same code and it only used 73
I think the latest stable, around 1.80 and Ubuntu 20
Which os and rust version are you running?
@@dario.lencina ubuntu 22.04, rustc 1.81.0, toolchain is stable-x86_64-unknown-linux-gnu
Very good explication. I follow Nasser, he has very interesting topics.
I follow him too! He is awesome!
Very interesting analysis. I understand the need for system calls for Memory Management, Security, Loading modules, and basic IO streams during the startup.
But if a large number of system calls are a performance hit, what we can do is reduce the number of system calls in the runtime.
I am currently playing with IO_URING on Python's async await architecture, and it does reduce the number of system calls significantly for large IO loads like servers.
Python's default asyncio library uses a select-based reactor model which I always felt useless because it doesn't work on files.
This can be achieved with any language. I don't think any modern language is truly superior just based on the least amount of system calls while ignoring other benefits.
This is a fact ^^ as devs we need to analyze what we get from the runtime, else we would be all programming in assembly without an OS to squeeze the last cycle of the cpu
So, what does this mean? program performance, execution speed, compile time, or...?
Startup time is a bit slower in rust. It doesn't matter at all
perhaps slower startup time (though it's questionable if it's measurable)
this ^^
What you said doesnt make sense at all. Rusts memory safety happens in compile time not runtime. And tell me which system call improves safety? You just said bunch of nonsense. And additionally you didn't remove the includes in c code.
I love how you judge my assessment without even researching 😆😆
@@dario.lencina rust devs are wild candidates u should have known 😂