Signals. I spent 2 years to understand this part.
Вставка
- Опубліковано 9 тра 2024
- A quick introduction to one of the interprocess communication mechanisms in linux.
00:00 Main idea. We want to execute a function when external signal arrives
00:50 While loop. We can't simply jump to the signal handler!
01:24 Interrupts. Breaking a natural instruction flow
02:30 Saving registers. Kernel preserves user register values
03:09 How pc is saved
04:03 The most important idea! Replacing the pc
04:40 Big Picture overview.
05:55 To the Source code!
07:16 Kernel entry. Disassembling my kernel binary
12:04 Replacing the program counter
12:35 Return to user.
13:28 Signal handler is finished. How to resume the main code?
14:14 How the stack works when enter the kernel
16:26 We need to keep main code's original registers!
16:49 Kernel stack has to be empty. Overflow. Nested signals
17:30 Saving original regs to user stack
18:48 Kernel trampoline. Sigreturn
20:45 Bonus! (about the compiler bug video) - Наука та технологія
![Interview with Senior JS Developer 2024 [NEW]](http://i.ytimg.com/vi/aWfYxg-Ypm4/mqdefault.jpg)
I did a double-take when I saw the AArch64 assembly... most UA-camrs go straight to x86/x64 when talking about assembly, even though it's really not as beginner-friendly as RISC sets, so it's a breath of fresh air seeing someone introducing beginners to ARM. :)
Yep, makes it easier to follow. Though would prefer RISC-V assembly even more - I think it should have all the equivalents just the same
ARM? More like LEG. Get 2 and spread em
'arg, you're twisting my AARM' :P
I'm on the side of the fence getting burned by AARM existing. (Technology I require is x64 only. Customers require AARM) :/
Glad to see it's being used, but the flashbacks from work are real 😂
@@TheOriginalBlueKirby so that's why they called it that in Turing Complete (the game)
It looks like my process miss a signal every now and then. After watching this video, I think the issue is on my side, but I triple checked all the code. What a life.
If you are missing a signal, it could be because you are forgetting to re-register the signal handler after it catches the first signal. If you forget to do this, it will only catch a signal once
@@xBiggs I use python `signal.signal(signal.SIGUSR1, on_usr1)` and python emulates BSD signals, which do not need re-registering.
Have fun going down the rabbit hole
also if you use multi-threading, it is undefined which thread gets the signal
The example code has a bug. signal_arrived needs to be declared as volatile. Else the optimizer can allocate a register for signal_arrive in main(), and the while loop won't notice that the signal_arrived in memory has changed. In fact, gcc with -O3 optimizes out the whole loop.
So many high quality low level educational videos are being recommended nowadays, your is definitely one of them! I really like the "raw" work with objdump disassembly
6:15 what i liked- what i LOVED about this succinct, anthropomorphic explanation is that it really helped frame for me (a software engineer) how to think about the contractual interface i have with the hardware engineers without having to know about the details of CPU architecture.
overall just TERRIFIC presentation and formatting.
I cant even imagine what it must be like to "map" the kernel code in your brain because your explanation here was spot on. I have always (ie almost 6 years in SDe industry and 6 years of education before that) at a very high level just "assumed" that signals and interrupts are a thing and I know for a fact that I cannot dive into Kernel code to save my life but this video explanation I can definitely say I understood everything. That's definitely a kudos to you sir. I wish to understand more low level systems stuff more in detail as I continue my journey.
I swear this channel will blow up someday. I really like the chosen topics and presentation on this channel.
00:00 💡 Programs need to communicate with each other, often by executing functions when signals are received.
01:24 🛠 Interrupts allow the CPU to break from its natural instruction flow to handle external events like keyboard inputs.
02:30 💾 Kernel preserves user register values when handling interrupts by saving them to memory.
04:03 🔄 Replacing the program counter at the right moment allows the CPU to jump to a signal handler function.
04:41 🌐 Programs can register signal handlers with the kernel to respond to specific signals.
05:57 📜 Examining kernel code reveals how the CPU knows where to jump when an interrupt occurs.
07:18 🕵 Disassembling kernel binaries uncovers the implementation details of interrupt handling.
09:00 🔍 Detailed exploration reveals how the kernel handles interrupts and resumes user processes.
10:49 🛑 Ensuring the kernel stack is empty before handling a signal prevents overflow and nested signals.
12:07 ↩ Saving original register values to the user stack enables restoration after the signal handler executes.
14:16 ⬆ Understanding stack behavior in kernel space is crucial for signal handling and process resumption.
19:00 🎭 A kernel trampoline allows the CPU to jump back to kernel code after the signal handler finishes.
20:45 📺 Bonus: An additional video discusses a compiler bug related to reading local variables beyond the stack pointer.
It's like I didn't understand anything but still watched it, thanks for the video
I will watch this again in like 6 months and go "ooohhh"
(actually, I already understood like 75% of the video without knowing a single thing about signals, just gotta get that remaining 25%)
Learnt more C and Assembly than I did in 4 years of college in 21 minutes, you're a legendary instructor mate thank you for your service ❤
I find it amazing that every time an interrupt happens the CPU does this and it’s still able to be so performant.
Totally agree, blew my mind. It seems so inefficient to store all the registers to memory, then load them back in, every time the user presses a key on the keyboard.
just saw the words 2 years and i clicked and subscribed. Knowledge takes time and effort. This title clearly says it too
This is some quality education. Puts many universities to shame. Thank you!
Could you do one one on re-entrant functions and signal safety? I don't think I completely understand why a thread-safe function isn't also re-entrant.
Thank you for the video, appreciate it! Liked and subscribed ❤
If it is made threadsafe by using a thread-local variable, it is still not re-entrant from the same thread.
Reentrant means that it makes no assumptions about state or the order it is called so the code can be called multiple times, sonetimes before the first call was ecen finished. Imagine having your function invoked unpredictibly at some random place inside your function. Youd have to make sure there are no conflicts. No memory leaks etc.
@@tripplefives1402 Wouldn't each call get its own stack though? And if I hold a lock somewhere, the other invocation can't, meaning I guess I expect it to be re-entrant? Also, why would a function start somewhere in the middle randomly without also having executed things above it at some point? I mean sure one can come with examples, but just talking about standard functions + signals, it is not clear to me why that would happen.
@@programming5274 That yea .... hmm. Okay I guess the source of my confusion is that how can there be 2 execution frames of the same function in the same thread without them also having their own stack? Wouldn't that make it re-entrant safe? Isn't that similar to stack-frames in recursion?
LoL I kinda feel ashamed of myself haha, I get paid to write code but not knowing this properly definitely is making me question my readiness and "skillset". I really need to do better.
@@dexterman6361 Yes the stack would be different each time, but the heap and everything else would be the same. A signal can happen at any time, and when it happens your program is stopped at wherever it is and then execution jumps immediately to the function you told it to invoke. So if it is already invoking your function for that signal and the same signal or another signal also gets invoked execution will immediately jump to that function again. So no global variables or anything like that, and you must check to make sure that if there is some global state that you test it to make sure it wasn't already changed prior and you can't assume that you are starting fresh each time. In other words don't double allocate or double free. As far as developing operating systems you cant assume that your interrupt handler wont get called multiple times at the same time because if you blindly turn off interrupts then your code will miss input and the hardware will not keep a backlog for you.
Your videos, explanations, animations, are all absolutely stellar. So clear, so efficient. PLEASE keep sharing the knowledge, this is truly remarkable.
Very nice teaching method, by resolving problems one by one. Great video !!!
Respect to the knowledge level. I don't know if many of us will need it daily, but i hope many of us who want to know how it is done inside, will gather to such
level that you will keep sharing your wisodm with us.
the video's quality and information are truly stunning
Great video! I like your choice to explain everything with ARM assembly, instead of x86. You earned a new subscriber :)
The layout is just beautiful and the instructions are crystal clear.
Keep it up man, worth watching. Thanks 👍
Your channel is so good! Outstanding! Keep up the good work.
I just got my head around 14-bit PIC interrupts (at least on some level), and this absolutely blew my mind.
Excellent explanation. I now want to add interrupt functionality to my cpu architecture. Im also thinking about adding flags.
what exactly does "add interrupt functionaliry to my cpu architecture" mean? Are you writing a software emulator for some hypothetical cpu you're designing?
Wow great timing, this is literally exactly what I am battling with at the moment.
You make it amazingly clear!!! You deserve more subscribers!
Time well spend and explained in depth
Very good explanation. Keep up the good work!
If only had this video a few years ago when had to learn arm64 for the job, specifically port our kernel(OS) to it. Took a hot minute, to say the least, to understand concept of interrupt vector tables and under workings of the assembly
Also tpidr is kinda scratch register where you can store anything but it's cannonical use is to store id of current thread
Are you telling me that you got a job to port a kernel to another arch, and you didn't even knew the concept of interrupt vector tables? Wow, must have been a really hard and eye opening experience.
@@framegrace1especially because x86 has the same concept.
Cool. Good to know that we can use the bonus part to possibly get some escalated data into userspace until interrupt gets executed.
Excellent explanation!
this is perfect asmr for my brain
Awesome explanation!!!
a very good explanation and hard work , good animation , keep going
Beautiful Description
Such a high quality video, thank you very much
Wow very impressive, thank you and keep doing a great work please!!
This is such a great video! thank you
The internet certainly does not have enough low level programming information available, so things like that are always appreciated. I never worked on ARM but many other risc micro controllers. Also implementing preemptive / cooperative multitasking based on interrupts on x86 is somewhat simpler. This can even be done in the classic real mode. Since x86 stores the interrupt return address on the normal stack, in order to do a context switch you really just need to exchange the stack pointer / stack frame. Each "process" would have it's own stack area. So a process could invoke a software interrupt (cooperative) or a hardware interrupt like a timer (preemptive) could interrupt the execution of a process and the interrupt handler takes over. Besides the automaically stored return address, the handler would save the registers in the user stack frame and then just exchange the stack to the stack of a different process. As a result when the interrupt handler does it's iret, it returns to where this process has left off the last time.
My dad actually developed quite complex machine control software back in the 80s and 90s and they used a DOS powered machine that ran over 2000 "tasks". Many task where really small, did just one or two checks and then yield again. They circled through all tasks hundreds of times per second. The actual pascal library mtask is really simple, just around 515 lines of code with many comments (with the comments stripped out it's just around 320 and with a bit different syntax style just 230 lines :)). Quite an impressive library from an ancient time. Of course it does not really apply to any modern system except embedded and microcontroller development where you don't really have a separate kernel. Though the fundamental concepts are still the same.
Fantastic video! 😁
Thank you
Great video, I always like low level OS stuff, always fun.
But most programmers would have a different idea in mind when discussing interprocess communication.
named and anonymous mutex, semafores, pipes, sockets,shared memory, stdin/stdout/stderr and Maybe even COM on windows.
All of them start with an interrupt.. just saying. This is exactly how any async call works, deep inside any language or framework you may use there's something like this.
@@framegrace1interupts are optional with things like mutex. You can pause until signaled and then acquire the mutex but you could also just check periodically (not always in a loop). Its more or less memory that has been mapped into both processes so each can read each others writes.
What software do you use for this kind of animation?
An excellent video!
amazing visuals
I'm very curious about this msr and mrs instructions that are skipped by the branch. @ 8:50
I looked it up, and Move System to Register, and Move Register to System, seem like they would be useful as part of the interrupt process.
I wonder if there is special kernel code that can conditionally call into this interupt offset to skip the branch instruction?
This is great, so much pleasure watching this, thank you for the great work! LOVED your presentation being "everything happens everywhere at once" diagrams! I wish my debuggers could do that, haha. I'm gonna rewatch it again, making sure I've looked at every detail in your illustrations.
Have you considered looking into the more general idea? After all, Linux is conceptually UNIX, and UNIX is all about portability, thus signals is just a portable way of implementing interrupts-driven compute. (Then again Windows has it differently with its SEH.)
The general idea of interrupts is fascinating in their own right, like Dijkstra famously said: "It was a great invention, but also a Box of Pandora. Because the exact moments of the interrupts were unpredictable and outside our control, the interrupt mechanism turned the computer into a nondeterministic machine with a nonreproducible behavior, and could we control such a beast?"
If you have, and in the chance you have time on your hands, I'd LOVE to see your amazingly enlightening approach applied to visualization of more than just signals :) I know, it's a lot to ask, but one can hope, haha.
UA-cam likes to eat comments containing URL-like strings, but I'll try to put a link a response to my own comment to a wonderful page on interrupts history you very probably have seen already, from Mark Smotherman.
* people dot computing dot clemson dot edu/~mark/interrupts dot html
* Irfan Ahmad, his talks and paper on interrupts coalescing.
Great video! I think it could have underlined a bit more that these details are specifically about AArch64. The x86-64 implements some details a bit differently but knowing either one helps a lot in understanding the another.
Such a good video.
This was such a great video, keep it up man you earned a sub from me.
Amazing ❤
without OS, this is way easier to do, absolute control of the Processor counter xD
I love the animation style of your videos. How did you come up with this? I’d love to use this to present topics to colleagues
Amazing video :)
Thanks!
Bro cooked on this video. Respect.
Great video, I think that only thing that is missing is explanation about EL0_EL1 - not every arch has different banks for them.
Great content! Can you please share what tools do you use for animations?
Would it be useful for a CPU to have kernel only registers? That way the kernel wouldn't have to save and restore user-space registers all the time. Although, I guess it wouldn't really save much since the kernel has to do that a bunch for application scheduling anyway.
The kernel stores the registers in the stack so it can switch threads by just changing stack pointers and returning. To change processes it changes the page table loaded on the cpu.
@@tripplefives1402 it would have to restore the cpu registers at some point surely? User space programs still use them.
really good video ngl
What happens if another interrupt replaces the hardware "saved_user_pc" before the kernel had time to save the state from the first interrupt?
This is a very good question! Indeed, such a problem exists and hardware is well-aware about it.
When a systemcall (svc arm instruction) or an interrupt occurrs, the CPU hardware immediately disables all the interrupts. Since that moment it is up to the kernel code when to enable interrupts back. So, the kernel is safe while saving registers and program counter - another interrupt will never occur until the kernel explicitly enables interrupts back.
I think the first branch and the bytes skipped is for instrumentation, either debugging/probes or hot code patch
Дякую!
4:48 Hey, this example is really bad. Your signal handler does not follow the correct prototype of void (*handler)(int). Also, the signal handler is required to be reentrant or async signal-safe. Your function is not because it uses global state int signal_arrived instead of some volatile sig_atomic_t or other lock-free atomic global. In this case, you should probably use atomic_flag because it is guaranteed to be lock-free and your example doesn't care about what the value of the signal is. Furthermore, it calls printf() which is also not reentrant. Though, on a posix system you could replace it with a call to write()
Thanks! Yes, the example could have been more accurate.
K'mon, is just an example of how this works.... No one here is expected to jump directly to write signal handlers for real OS's. All that minutia would make just the concept more obscure. He explained all important parts.
Isnt this similar to the pipe function in ĺinux? @@kimylamp
@@framegrace1you use signal handlers in user mode code as well if you work with code that uses two or more processes or if you pause/sleep until some shared resource is free to acquire.
❤
A UA-cam gold 🎉
how do you make these videos? they are beautiful!
Love your videos! Though I think they might age better if you did Rust instead of C.
Program A runs on boot. Program B causes a kernel panick immediatly rebooting the pc.
problem solved
Can you share your color scheme? Thank you
Sure. It's Gruvbox Dark Medium.
Thank you! Great video btw
1:00 while not particularly relevant to the discussion (it's just an example), one should be careful with infinite loops...
int main(){
while(true){};
return 0;
}
C++ x86-64 clang (trunk) -O3
main: # @main
this seems to be a bug with LLVM backend.
now, if you use gcc, you might end up with the process hogging as much cpu as it can. so you really should sleep there.
1:25 it's only like that with old pc's using ps/2 plug. usb uses polling afaik.
Set the variable to voltaile so its not optimized out.
Too complex, imma build spaghetti with boost::interprocess instead.
AI ART LMFAO
Bad example. You CANNOT call printf() from within a signal handler. printf is not async-safe [it may call malloc() internally while interrupting inside of malloc()]
Busy waiting for a signal???
ngl it felt cool didnt understand anything
why use AI for the thumbnail?
what's the point of those strange characterisations, such as "the kernel VERY CAREFULLY saves all the registers..."?
Why not just say "saves" instead? It's an inanimate abstract interface, it's unable to CARE. I get it that you're trying to make it sound like a story from a book for kids (or something), but this just doesn't sound good.
Thanks!
Woooooow