@@overlisted Nah, if your name's Bill then you can only buy OSs from other people and turn them into unstable Frankensteins that BSOD all the time. But at least that'll make you rich. Now, I wonder what you should do if your name's Rich?
@@ThePC007 The reason Windows is less stable actually has a lot to do with vendors more than MS being incompetent, If you've ever seen the source code you'll see the comments on how they detail bugs that were fixed but the vendors who MS are obligated to support often asked them to be put back as the patches often break 3rd party software/hardware functionality. With Linux/BSD that relationship is the polar opposite , with everything being homogeneously developed under that relationship of having to make things work, shit is going to objectively run better and be infinitely more extensible, especially if you stay within the Unix ecosystem.
BSP (board support packages) has the real drivers for the work. BSP is hardware-specific and also vendor-specific. Many drivers here have the writes to the registers which are mentioned in the User manual of that SoC.
16:25 - The kernel check permissions in modern OSes; back in the DOS days, we read from and wrote to ports directly, so reading the keyboard controller from port 60h was a common task (well, maybe not as common as using interrupts). In those days, the computer _was_ essentially an Arduino. I miss the days of displaying stuff on screen by just writing directly to video RAM.
You could always do the same with linux. Just pipe the /dev/random to the screen buffer! Even better, pipe it to sound output so you could listen to "random"! :D
@@YandiBanyu Hey can you expand a little on how I would pipe random dev output to a screen buffer or audio buffer. Like which device I should be piping to?
@@kunaljoshi8038 That depends on the detail. But one thing is for sure, you need to make sure that the linux kernel create the framebuffer node (usually /dev/fb# where # is any number). After that, just do the usual "cat /dev/random > /dev/fb0". Of course it also hard to notice since your screen will likely refresh at least 24Hz. Audio buffer however is a little bit easier. Most Linux uses ALSA, you could always do "aplay -c 2 -f S16_LE -r 44100 /dev/random"
One thing worth mentioning is that the port io instructions are very old, dating back to the original 8088 CPU and it still exist today for backwards compatibility. Just like these segment registers (cs ss etc.) You mentioned in this video. Most all hardware on x86 is in fact memory mapped
Check out this Playlist by Ben Eater on building an 8bit breadboard computer for something even more in depth :) Edit: forgot the link lol ua-cam.com/play/PLowKtXNTBypGqImE405J2565dvjafglHU.html
Isn't that all of hacking? Just fool the application into accepting your malicious input, either due to its own fault not being wise enough, or by exploiting logical loopholes to inject data straight into the app.
There's a book called Understanding the Linux Kernel that I'm reading through at the moment that you might be interested in. It has chapters that go through things like interrupt request controllers and whatnot and interacting with hardware. It's not the very latest thing in the world, so a lot may have changed with the newer Linux kernels, but I would expect most of the very basic stuff to be the same since it still needs to work with the physical way motherboards and chips work, including older modules - so even if newer systems have been implemented both in hardware and software, this mechanism at least should still be in the kernel for backwards compatibility
Thanks fot the suggestion! Look at this: www.amazon.com/Practical-Binary-Analysis-Instrumentation-Disassembly-ebook/dp/B07BPKWJVT The position seems quite interesting, and a little about the topic. It's important to have a basic knowledge about the binary files. I haven't bought and read it yet, but i think, i will get one for me.
Absolutely fantastic. Watching this was *exactly* like finding a missing puzzle piece. It isn't deeply technical enough to explain everything (which I'm glad since that'd be wayyy too long) but it gives me a guiding light in my quest to almost understand computers. I really enjoyed this, thanks for the great video.
Not all IO must happen through these ports. Many times hardware (especially modern hardware) write directly into memory and then inform the CPU via an interrupt and so CPU doesn't have to synchronously read from ports. Part of the PCIe protocol is an algorithm for the CPU and a piece of hardware (like GPU or network card) to decide where exactly in CPU's memory address space they can communicate
@@victornpb yeah, sort of. DMA is a generic term for the ability of other hardware to access the main RAM without going through the CPU (i.e. not through port IO). PCI is a much more complicated protocol
I've been watching your videos for years now. Im about halfway through my embedded systems class in university now and I finally feel like I can watch without feeling lost.
Also, for your microprocessor and computer bridge. You can't really compare them. The I/O on the microcontroller is usually not directly connected to the CPU. Microcontroller has CPU, RAM, ROM, and I/O INSIDE the chip so many implementation is still abstracted away. And Ben Eater 8-bit computer is great, but it only cover the surface. I suggest maybe check out Ben Eater next series about Hello World from scratch. Or the (once) popular Z80 CPU.
Well microcontrollers still just output on the wire as you told it. In modern CPUs, saying _out 0x60 0xDE_ doesn't necessarily mean 1101_1110 is written to the wire, it has to follow all the different protocols like (S)ATA, PCI-e, and even the non-trivial protocol used to fetch data out of the RAM.
Heyy, I would like to request one thing to you could you please also add the prerequisites people need to know in order to understand the above thing, so that people like me who don't anything what's going on, can look into the prerequisites and go through it before going direct to the presented video. It would be really helpful!! Thanks a ton for these videos! (Hope, I did not bother you with my request :) :D
Most people already can’t understand the software part of computer science (they talk about power and compare numbers they can’t understand) so going deeper with the link between software and hardware is impossible to grasp to even a greater group of people. Just the algorithms needed to ensure the integrity of some operations in memory is hard to grasp , I won’t even go as far as what is needed to « ensure » that data is written on disk and that you didn’t destroy all your filesystem just by writing data in it. These are fascinating. Concurrent programming was one of my mastered course 30 years ago, so I have less problems than most understanding these.
Same. I Tried to find out how to read the Flash where the BIOS is stored. Came to the SPI stuff but then some other projects came up. I need to look into it again.
So I never comment on anything ever but I have been trying to get my head around x86 enough to try and write some OS code in my free time and this video literally gave me the biggest CLICK moment ever. Finally I can understand how a modern CPU interacts with things in a way that I feel comfortable exploring further. Seriously I was already binging your content but I felt the need to actually thank you for this, so much.
x86 (and relatives, like Z80) IO ports are just another address space -- much like memory, there are registers located at addresses, indeed it's usually multiplexed on the same address/data bus, just strobed with different signals (IORD/IOWR). And the accessing instructions suck (like x86 only putting operands through AL/AX/... and (E)DX!). The IBM PC had a quirk that only 10 bits of IO address was decoded (to the ISA bus, from the CPU; I forget how this happens), so addresses tended to be used a bit carefully. Like the keyboard port registers sometimes having extra bits for system flags (like DIP switches, or A20 enable). Or the paired graphics registers, where one serves as a pointer for the other, into a device-internal register file. As far as I know, this remains much the same today, with the quirk that wider (WORD+) IO accesses are possible (186+) (which probably isn't a good idea to access the above type of registers?!), and that everything propagates through caches and bus interfaces (>Pentium?) -- there's a lot of abstraction in the hardware alone, a lot of work has been done to keep things transparent to the CPU instruction set.
3 роки тому+1
In university (a long time ago) we had to breadboard a programable controller using a z80, some sram, eeprom and port expanders, everything "glued" with discreet logic. It helped greatly in understanding memory mapping, and separate memory and I/O spaces, not too different from way the x86 handles it.
When I watched your "How Do Linux Kernel Drivers Work?" video, I did expect something like you showed in this video. After watching that video I was a little bit disappointed that you were only talking about "high level" stuff ( or userland ).
I have to tell my 3 cents. IO Ports is not the only way to talk to the hardware. There is also something called Memory Mapped IO (MMIO) - registers and memory of the device's controller is mapped directly into virtual adress space, most often above kernel space. You need some type of documentation to know which byte in virtual adress space refers to which registers in the device controller on motherboard. When the virtual adress flies (from CPU) through the MMU it recognizes that adress should be directed to the certain controller register. X86 supports both methods of accessing peripherals but MMIO is currently recommended. Ports IO is some kind of legacy stuff. To be honest I don't know why MMIO is better than IO Ports and I'm not sure when to use what method so feel free to help me understand that. I read that advantage of MMIO is that we can use exactly the same instruction set as for normal memory when with IO Ports we can only use IN/OUT instructions to read something from controller and then dealing with recived data. So if we want to add 1 to controller's register value we have to: read data from controller, add 1, send data to controller instead of using just "INC [memory]". The same situation of every other more sophisticated instruction. I heard also that IO Ports have some performance problems so USB and PCI devices are mapped by MMIO method but I don't know if it's true. I still wait for somone experienced enough to dispel my doubts :) Btw. isn't it funny that we have to "reverse engineer" these facts which are probably obvious for every kernel developer? XD
My intuition tells me that MMIO would be more efficient and would have way higher throughput, because devices can write its buffer directly on memory without having the cpu bit banging every byte
Side note, if your keyboard supports usb 3, it can use async notifications instead of polling, so that would be the "modern" replacement of this interface
Shhh, nobody tell them that PS/2 is not a direct interface, at least on PCs - there is traditionally a bus connected microcontroller in the middle (and it does more than just keyboard functions, you can also ask it to reboot the system or mask off the A20 line) - though in modern implementations it is probably no longer a real microcontroller core involved but it tries to act similarly for compatibility. The interface is still much simpler than USB, however, and PS/2 devices can be direct connected to anything with GPIO pins.
What an awesome video. I love this channel. In this video you've answered something I've wondered for a really long time... exactly how does a modern desktop CPU access hardware. I too was familiar with writing little pi and arduino programs that toggle LEDs using memory mapped functions, but even there I was left scratching my head as to how those functions were actually working. This video explained it all! Thank you!
So a port is just a register of some hardware mapped to a static/predefined memory location? I never fully understood memory mapped io back at my university time. Thanks for closing the gap. There is still so much to learn about Linux and I'm really excited about the journey. And keep going with these meta videos about the progress of learning!
I wouldn't say a port is a register. Rather a port is an ADDRESS which the CPU or hardware designer defined to be connected to some hardware. That is why in assembler you could use IN/OUT assembly to write or read from those ADDRESS
Love it. One of this rare videos where you explain you learned something new i already know for a long time. It makes me feel less stupid. When i watch your other videos, i see that you know so much more than i do.
Modern RISC processors (like RISC-V) tend to use memory-mapped I/O rather than port-mapped I/O, it's more flexible because: it avoids adding specific instructions (classic load/store instructions do the job) and it removes the constraint of supporting obsolete technology (like PS/2 port). By the way, x86 also use memory mapped IOs for modern devices (like the PCI bus). For memory-mapped I/O on linux we use a "device-tree" file that describes which physical address corresponds to which device. This allows linux, at boot time, to know the physical addresses of the devices for the kernel drivers and to define a virtual address space for each driver. To prevent users from using the I/O directly, the kernel simply does not map the I/O memory in the virtual address space of user processes (so he doesn't need to do anything). And a user can under certain circumstances ask the kernel to map the address space of a device in its virtual memory for faster access (bypassing syscalls) and implement a driver in the user world. This would be much more complex with port-mapped I/O. Memory-mapped I/O are a much more elegant solution than port-mapped I/O. =)
The middle part that you say you are having trouble with can be summed up with one word: registers. You toggle a pin on an MCU by writing to a register. Basically, you read the register corresponding to the pin bank, change a single bit, and write that changed value to it (because you can't address single pins directly). Then the logic inside the MCU will read the register, send it to the output buffer, and the pin will be set accordingly. This is typically done once every four clock cycles. Edit: I thought you were asking about how pins are set in a MCU. But it seems that you have that knowledge concerning CPUs. MCUs are not different. An MCU is a CPU with output banks and other stuff (namely RAM, DACs, ADCs...).
Fun fact on hardware vs network ports: Port 0x80 is used for firmware debug output that is sent out through the LPC bus, what a POST card displays (or super I/O / EC / BMC catches today). Not to be confused with the HTTP port. In spoken language, both are referred to as "port eighty". And "to POST" (power-on self-test) in firmware not to be mixed up with the HTTP verb. ;)
would be nice if he explains what a port is, like, I get the idea, modern CPU, arduino and other CPU share the concepts, and use IO ports to communicate to hardware, but WHAT exactly these IO ports are? I still missing something :(
Bus vs MMIO : if the process threads get accessed by /dev/bus and altered information... How does that kinda hack happens? How does bus of the soundports get hooked on by another device? (Thank you for the arduino and PC comparison ♡ I learned something I couldnt understand for years on ports and pins memory address)
I actually had that exact same eureka moment a few months back (without watching this video but some others on how pcs work) and it was one of the best things to experience such moments, you suddenly feel like you know EVERYTHING (which obviously isn't true)
i think the intel or amd cpu dont know about ring's (kernel) i think it is the CISC instructuction set that makes it hard to understand what goes on. today you can get the same result with many ways, like mmx, sse, fpu, gpu, or native x86(286) code. an olodscool 8bit cpu otherwise is understandable enouhgt to get the feeling of real understand whats going on. like the 6510/02 it is easy to know whats going on, but yeah it gives tricks to hide a bit but if you know the cpu you know whats going on. i dont thinkt so about cpus above a 386 :/
I don't know much about SerenityOS but what's bothering me is that (if I understood the video correctly) that it's kernel is running with ring 0 privileges. Is there any particular reason/advantages in doing that?
i once (kinda) wrote an operating system (for an atmega 2560) that can (sort of) deal with multitasking and i'd say it's more complex then youe standard c programm for an arduino, but still nowhrere near a propper os like Linux. but i learned a lot kn writing it
I'm too nooby to understand a lot of this but it's definitely interesting how using cpu processes can be used to bypass restrictions placed on an operating system.
bro you need to compile everything from hardware exploitation to software exploitation in a course would be very grateful I can get a good hold I have Arduino and learnt c and C++ since they are used for embedded systems but still feel I haven't got anywhere please help me
I think the title is more of an expression of his though train you know? He explain it that he doesn't really know the bridge between the hardware and the kernel. And then when he understand the exploit, he finally understand how CPU Access Hardware
daaaamn... you asking the same questions as I do about how the computer works! - That goddamn keyboard...HOW DOES IT WORK!?!? ps. I am only 5:15 into the vid - hopes you will enlighten me with all the answers I need !!
For a better feel of x86 hardware I/O basics, you could always spin up an 8086 emulator and run DOS in it. Since back then there wasn't any hardware privilege isolation and it was closer to today's MCUs. Later processor versions just layered more features on top.
I really appreciate your kind of explaining things in your videos in a really good way. I love your videos and just wanna say thanks 🙏 to you ☺️. I really like your character, too!
Very nice and informative video! Just a couple of mistakes about the AVR assembly: @14:02: The arguments of “sbi” are backwards: first comes the port number, then the bit number. @14:13: The second argument of “out” is not an immediate value to be written to the port, it's the index of the CPU register that will be copied there.
Actually even some microcontrollers have the concept of hardware privilege levels. For example all ARM Cortex-M cores have two privilege levels, when a core is also equipped with a MPU (Memory Protection Unit) you can also restrict access to some memory areas for unprivileged code. They all also have syscall instruction, so you could implement a basic OS on an ARM microcontroller - which I definitely recommend - it's great learning experience especially since the architecture is so much simpler from x86.
Please keep in mind, that PS/2 is not only the "old-school keyboard port". It is still being used in many modern notebooks since it is interrupt-driven. Therefore it doesn't have to poll the USB interface all the time. This approach can save some power on mobile devices.
Coming from eletronic engineering I had the (wrong) impression that I/O on ports was a microcontroller thing and microprocessors used memory mapping. I see now I was wrong. Thank you very much for clarifying that, it was really useful!!
Huh, interesting, I never knew about the IN and OUT instructions in AVR assembly, or any assembly for that matter. When I wrote AVR code to access GPIO pins without the arduino stuff, I always just read and wrote directly from/to the PORTA and PORTB registers, which would to the same as digitalRead/digitalWrite. I'm guessing this is the equivalent to "memory mapped access", compared to port based access with the IN and OUT instructions?
“OUT” is a machine instruction you use when programming in assembly. “PORTB” is a macro of the avr-libc you use when programming in C. The compiler then translates that to whatever it thinks is suitable, most of the time to port-mapped I/O.
For anyone dealing with these kind of questions: Write a basic operating system for a raspberrypi. If you have a certain amount of experience, it's not that hard and you don't need to implement everything. Just a uart driver maybe that you can send messages through. You will read a technical manuals and data sheets. And you will learn a lot. Reading a cpu manual for the first time, it is surreal. I remember having this weird bug in the beginning where I wasn't realizing that an interrupt changes the processor mode and different processor modes have different stacks, so my interrupt stack wasn't initiallized. That's a bug where google won't help haha. Anyway, I can recommend it dearly for a side project.
I too had a "what the heck?" moment at the hardware/software interface. In my case it is sort of "why does software sort of 'disappear' from view at a certain point, and what is it that the abstraction is removing?" As for the exploit, interesting, but: given the IO ports, does this exploit assume that there is a ATA type disk available? In particular, would one have to rewrite it on a SCSI system (for example)?
I knew about this concept existing in old game consoles thanks to Retro Game Mechanics Explained (game controllers, audio chip, cartridge, etc), but never thought this still applies to modern CPUs today!
I also understand what it's like to know the low level of computers (more or less) and I know how to make programs for operating systems, and in this video I identified myself a lot. I realized that people who develop operating systems (especially if it's from scratch ) are people who could build (theoretically) a computer from silicon to make an operating system, and then make the software for own OS. This is amazing and i want to do that too.
I can really recommend building a risc v core on an FPGA. It is very interesting an you have the opportunity to build Interface logic yourself (like adressable hardware)
I had no clue assembly had a command to directly do port I/O, I assumed everything was memory mapped (and that data magically got to the desired device after writing to memory)
Well. The concept is not that hard to understand. But i still have not much of about how to start implementing it into an actual exploit^^ I guess i have to write to some executable file (bash or whatever) and set a uid bit or something. Without messing up the file system. How does accessing the first sector help me out on that?
This was so good! Thanks for sharing this. Still not 100% sure whats going on between the userland and hardware, but way smarter now than before I watched this video.
If your name is Linus, you should do something with operating systems. It will turn out well.
bill too
@@overlisted Nah, if your name's Bill then you can only buy OSs from other people and turn them into unstable Frankensteins that BSOD all the time. But at least that'll make you rich.
Now, I wonder what you should do if your name's Rich?
There’s also Linus from Linus Tech Tips, lol.
Sure, he didn’t write an OS, but still.
@@austinfay6287 He builds the hardware, he’s like the other linus but he’s more hardcore and doesn’t have an undo button on his code
@@ThePC007 The reason Windows is less stable actually has a lot to do with vendors more than MS being incompetent, If you've ever seen the source code you'll see the comments on how they detail bugs that were fixed but the vendors who MS are obligated to support often asked them to be put back as the patches often break 3rd party software/hardware functionality.
With Linux/BSD that relationship is the polar opposite , with everything being homogeneously developed under that relationship of having to make things work, shit is going to objectively run better and be infinitely more extensible, especially if you stay within the Unix ecosystem.
BSP (board support packages) has the real drivers for the work. BSP is hardware-specific and also vendor-specific. Many drivers here have the writes to the registers which are mentioned in the User manual of that SoC.
Sounds really cool, even though what I am really familiar with are only { , if ( , } and ) 😁.
So it is actually possible to control port-mapped device on userland?
16:25 - The kernel check permissions in modern OSes; back in the DOS days, we read from and wrote to ports directly, so reading the keyboard controller from port 60h was a common task (well, maybe not as common as using interrupts). In those days, the computer _was_ essentially an Arduino. I miss the days of displaying stuff on screen by just writing directly to video RAM.
You could always do the same with linux. Just pipe the /dev/random to the screen buffer! Even better, pipe it to sound output so you could listen to "random"! :D
@@YandiBanyu Hey can you expand a little on how I would pipe random dev output to a screen buffer or audio buffer. Like which device I should be piping to?
@@kunaljoshi8038 That depends on the detail. But one thing is for sure, you need to make sure that the linux kernel create the framebuffer node (usually /dev/fb# where # is any number). After that, just do the usual "cat /dev/random > /dev/fb0". Of course it also hard to notice since your screen will likely refresh at least 24Hz. Audio buffer however is a little bit easier. Most Linux uses ALSA, you could always do "aplay -c 2 -f S16_LE -r 44100 /dev/random"
@@YandiBanyu ok thank you. Will try this out 👍
@@YandiBanyu I don't know what I was expecting when I played random through the audio buffer
holy shit I think you tranfered your eureka moment to me really well
+1
+1
thanks 68th liker.
One thing worth mentioning is that the port io instructions are very old, dating back to the original 8088 CPU and it still exist today for backwards compatibility. Just like these segment registers (cs ss etc.) You mentioned in this video. Most all hardware on x86 is in fact memory mapped
I really like the low-level interaction between software and hardware. I'd love more videos about this!
Check out this Playlist by Ben Eater on building an 8bit breadboard computer for something even more in depth :)
Edit: forgot the link lol
ua-cam.com/play/PLowKtXNTBypGqImE405J2565dvjafglHU.html
Kernel exploitation is like social engineering, just ask the kernel nicely to do something for you!
And you ask in such a way that the kernel doesn’t realize what you actually want to do, and doesn’t refuse.
@@chri-k 😭😭😭
ua-cam.com/video/PxThRel8OyE/v-deo.html
Isn't that all of hacking? Just fool the application into accepting your malicious input, either due to its own fault not being wise enough, or by exploiting logical loopholes to inject data straight into the app.
There's a book called Understanding the Linux Kernel that I'm reading through at the moment that you might be interested in. It has chapters that go through things like interrupt request controllers and whatnot and interacting with hardware. It's not the very latest thing in the world, so a lot may have changed with the newer Linux kernels, but I would expect most of the very basic stuff to be the same since it still needs to work with the physical way motherboards and chips work, including older modules - so even if newer systems have been implemented both in hardware and software, this mechanism at least should still be in the kernel for backwards compatibility
Thanks fot the suggestion! Look at this:
www.amazon.com/Practical-Binary-Analysis-Instrumentation-Disassembly-ebook/dp/B07BPKWJVT
The position seems quite interesting, and a little about the topic. It's important to have a basic knowledge about the binary files. I haven't bought and read it yet, but i think, i will get one for me.
@@damianc8738 Certainly a title that sounds like it could be a good resource :)
been reading it for 5yrs now. great book
As a person who has the same confusion regarding the communication between kernel and hardware I really appreciate this video!
Absolutely fantastic. Watching this was *exactly* like finding a missing puzzle piece. It isn't deeply technical enough to explain everything (which I'm glad since that'd be wayyy too long) but it gives me a guiding light in my quest to almost understand computers. I really enjoyed this, thanks for the great video.
I understand this is more about the hardware I/O and all, but can we all actually take some time and appreciate how beautiful Linus exploit was?
I know right? I was amazed when I saw it! So simple yet so clever :)
Not all IO must happen through these ports. Many times hardware (especially modern hardware) write directly into memory and then inform the CPU via an interrupt and so CPU doesn't have to synchronously read from ports. Part of the PCIe protocol is an algorithm for the CPU and a piece of hardware (like GPU or network card) to decide where exactly in CPU's memory address space they can communicate
You mean something like DMA from the peripheral side?
@@victornpb yeah, sort of. DMA is a generic term for the ability of other hardware to access the main RAM without going through the CPU (i.e. not through port IO). PCI is a much more complicated protocol
I've been watching your videos for years now. Im about halfway through my embedded systems class in university now and I finally feel like I can watch without feeling lost.
Also, for your microprocessor and computer bridge. You can't really compare them. The I/O on the microcontroller is usually not directly connected to the CPU. Microcontroller has CPU, RAM, ROM, and I/O INSIDE the chip so many implementation is still abstracted away. And Ben Eater 8-bit computer is great, but it only cover the surface. I suggest maybe check out Ben Eater next series about Hello World from scratch. Or the (once) popular Z80 CPU.
ALLES is possible for you guys geez (sorry for the bad language joke)
lmaoo das verstehen jetzt aber auch nur deutsche xd
@@icarus1656 Afrikaans...
Andreas Kling is Tyrell Wellick
Jajajajaja siii
Lol this is fun hahahah
Ciro Santili, the Stackoverflow legend?
Man it's scary, he appears everywhere, I started to think it was some kind of cult
Ohh, such a simple exploit.
Well microcontrollers still just output on the wire as you told it. In modern CPUs, saying _out 0x60 0xDE_ doesn't necessarily mean 1101_1110 is written to the wire, it has to follow all the different protocols like (S)ATA, PCI-e, and even the non-trivial protocol used to fetch data out of the RAM.
Heyy, I would like to request one thing to you
could you please also add the prerequisites people need to know in order to understand the above thing, so that people like me who don't anything what's going on, can look into the prerequisites and go through it before going direct to the presented video.
It would be really helpful!!
Thanks a ton for these videos!
(Hope, I did not bother you with my request :) :D
Most people already can’t understand the software part of computer science (they talk about power and compare numbers they can’t understand) so going deeper with the link between software and hardware is impossible to grasp to even a greater group of people. Just the algorithms needed to ensure the integrity of some operations in memory is hard to grasp , I won’t even go as far as what is needed to « ensure » that data is written on disk and that you didn’t destroy all your filesystem just by writing data in it. These are fascinating. Concurrent programming was one of my mastered course 30 years ago, so I have less problems than most understanding these.
thats a question ive been having too for a while
Same. I Tried to find out how to read the Flash where the BIOS is stored. Came to the SPI stuff but then some other projects came up. I need to look into it again.
i wonder how EFI works as well, do you have any simple resources?
@@overlisted Me too
i tried to syscall in C and wonderd what if i run C code with no OS how do i talk to IO Devices.
ua-cam.com/video/PxThRel8OyE/v-deo.html
So I never comment on anything ever but I have been trying to get my head around x86 enough to try and write some OS code in my free time and this video literally gave me the biggest CLICK moment ever. Finally I can understand how a modern CPU interacts with things in a way that I feel comfortable exploring further. Seriously I was already binging your content but I felt the need to actually thank you for this, so much.
next steps (ie. next headache): DMA
15:53 lmao why is andreas there
¯\_(ツ)_/¯
there goes the clickbait title xd
x86 (and relatives, like Z80) IO ports are just another address space -- much like memory, there are registers located at addresses, indeed it's usually multiplexed on the same address/data bus, just strobed with different signals (IORD/IOWR). And the accessing instructions suck (like x86 only putting operands through AL/AX/... and (E)DX!).
The IBM PC had a quirk that only 10 bits of IO address was decoded (to the ISA bus, from the CPU; I forget how this happens), so addresses tended to be used a bit carefully. Like the keyboard port registers sometimes having extra bits for system flags (like DIP switches, or A20 enable). Or the paired graphics registers, where one serves as a pointer for the other, into a device-internal register file.
As far as I know, this remains much the same today, with the quirk that wider (WORD+) IO accesses are possible (186+) (which probably isn't a good idea to access the above type of registers?!), and that everything propagates through caches and bus interfaces (>Pentium?) -- there's a lot of abstraction in the hardware alone, a lot of work has been done to keep things transparent to the CPU instruction set.
In university (a long time ago) we had to breadboard a programable controller using a z80, some sram, eeprom and port expanders, everything "glued" with discreet logic. It helped greatly in understanding memory mapping, and separate memory and I/O spaces, not too different from way the x86 handles it.
When I watched your "How Do Linux Kernel Drivers Work?" video, I did expect something like you showed in this video. After watching that video I was a little bit disappointed that you were only talking about "high level" stuff ( or userland ).
I have to tell my 3 cents. IO Ports is not the only way to talk to the hardware. There is also something called Memory Mapped IO (MMIO) - registers and memory of the device's controller is mapped directly into virtual adress space, most often above kernel space. You need some type of documentation to know which byte in virtual adress space refers to which registers in the device controller on motherboard. When the virtual adress flies (from CPU) through the MMU it recognizes that adress should be directed to the certain controller register. X86 supports both methods of accessing peripherals but MMIO is currently recommended. Ports IO is some kind of legacy stuff. To be honest I don't know why MMIO is better than IO Ports and I'm not sure when to use what method so feel free to help me understand that. I read that advantage of MMIO is that we can use exactly the same instruction set as for normal memory when with IO Ports we can only use IN/OUT instructions to read something from controller and then dealing with recived data. So if we want to add 1 to controller's register value we have to: read data from controller, add 1, send data to controller instead of using just "INC [memory]". The same situation of every other more sophisticated instruction. I heard also that IO Ports have some performance problems so USB and PCI devices are mapped by MMIO method but I don't know if it's true. I still wait for somone experienced enough to dispel my doubts :) Btw. isn't it funny that we have to "reverse engineer" these facts which are probably obvious for every kernel developer? XD
My intuition tells me that MMIO would be more efficient and would have way higher throughput, because devices can write its buffer directly on memory without having the cpu bit banging every byte
At this point, I just think ALLES! wants to make sure wisdom3 doesn’t happen.
Direct CPU interface rather than going through a USB bus is exactly why I use a PS/2 keyboard and mouse. Would love to see you explore data busses :)
why deal with USB controllers and queues when you can just interrupt the CPU and have it handle your input!
Side note, if your keyboard supports usb 3, it can use async notifications instead of polling, so that would be the "modern" replacement of this interface
Shhh, nobody tell them that PS/2 is not a direct interface, at least on PCs - there is traditionally a bus connected microcontroller in the middle (and it does more than just keyboard functions, you can also ask it to reboot the system or mask off the A20 line) - though in modern implementations it is probably no longer a real microcontroller core involved but it tries to act similarly for compatibility. The interface is still much simpler than USB, however, and PS/2 devices can be direct connected to anything with GPIO pins.
@@big0bad0brad woah really? I didn't know that, thanks
What an awesome video. I love this channel. In this video you've answered something I've wondered for a really long time... exactly how does a modern desktop CPU access hardware. I too was familiar with writing little pi and arduino programs that toggle LEDs using memory mapped functions, but even there I was left scratching my head as to how those functions were actually working. This video explained it all! Thank you!
I haven't been this engaged in a long time
I can't believe I knew one thing you did not all this time :)
Liveoverflow’s beard grows proportionally to his hacking skills
I loved your role in SuperBad!
This is the reason I started writing my own toy OS.
Same
On x86, maybe a few legacy devices use port based IO, but modern devices still use MMIO.
Danke für deine Videos. Sie kommen immer wenn meine Motivation abnimmt und geben mir einen gigantischen Motivations boost
So a port is just a register of some hardware mapped to a static/predefined memory location?
I never fully understood memory mapped io back at my university time. Thanks for closing the gap. There is still so much to learn about Linux and I'm really excited about the journey.
And keep going with these meta videos about the progress of learning!
I wouldn't say a port is a register. Rather a port is an ADDRESS which the CPU or hardware designer defined to be connected to some hardware. That is why in assembler you could use IN/OUT assembly to write or read from those ADDRESS
please, set up a back light. It's kinda difficult to watch you blending with the background
0:30 watch at 0.25
Love it. One of this rare videos where you explain you learned something new i already know for a long time.
It makes me feel less stupid. When i watch your other videos, i see that you know so much more than i do.
Modern RISC processors (like RISC-V) tend to use memory-mapped I/O rather than port-mapped I/O, it's more flexible because: it avoids adding specific instructions (classic load/store instructions do the job) and it removes the constraint of supporting obsolete technology (like PS/2 port).
By the way, x86 also use memory mapped IOs for modern devices (like the PCI bus).
For memory-mapped I/O on linux we use a "device-tree" file that describes which physical address corresponds to which device. This allows linux, at boot time, to know the physical addresses of the devices for the kernel drivers and to define a virtual address space for each driver.
To prevent users from using the I/O directly, the kernel simply does not map the I/O memory in the virtual address space of user processes (so he doesn't need to do anything).
And a user can under certain circumstances ask the kernel to map the address space of a device in its virtual memory for faster access (bypassing syscalls) and implement a driver in the user world. This would be much more complex with port-mapped I/O.
Memory-mapped I/O are a much more elegant solution than port-mapped I/O. =)
It's free real estate!
The middle part that you say you are having trouble with can be summed up with one word: registers. You toggle a pin on an MCU by writing to a register. Basically, you read the register corresponding to the pin bank, change a single bit, and write that changed value to it (because you can't address single pins directly). Then the logic inside the MCU will read the register, send it to the output buffer, and the pin will be set accordingly. This is typically done once every four clock cycles.
Edit: I thought you were asking about how pins are set in a MCU. But it seems that you have that knowledge concerning CPUs. MCUs are not different. An MCU is a CPU with output banks and other stuff (namely RAM, DACs, ADCs...).
Fun fact on hardware vs network ports: Port 0x80 is used for firmware debug output that is sent out through the LPC bus, what a POST card displays (or super I/O / EC / BMC catches today). Not to be confused with the HTTP port. In spoken language, both are referred to as "port eighty". And "to POST" (power-on self-test) in firmware not to be mixed up with the HTTP verb. ;)
would be nice if he explains what a port is, like, I get the idea, modern CPU, arduino and other CPU share the concepts, and use IO ports to communicate to hardware, but WHAT exactly these IO ports are? I still missing something :(
Bus vs MMIO : if the process threads get accessed by /dev/bus and altered information...
How does that kinda hack happens?
How does bus of the soundports get hooked on by another device?
(Thank you for the arduino and PC comparison ♡ I learned something I couldnt understand for years on ports and pins memory address)
I actually had that exact same eureka moment a few months back (without watching this video but some others on how pcs work) and it was one of the best things to experience such moments, you suddenly feel like you know EVERYTHING (which obviously isn't true)
i think the intel or amd cpu dont know about ring's (kernel) i think it is the CISC instructuction set that makes it hard to understand what goes on. today you can get the same result with many ways, like mmx, sse, fpu, gpu, or native x86(286) code. an olodscool 8bit cpu otherwise is understandable enouhgt to get the feeling of real understand whats going on. like the 6510/02 it is easy to know whats going on, but yeah it gives tricks to hide a bit but if you know the cpu you know whats going on. i dont thinkt so about cpus above a 386 :/
I don't know much about SerenityOS but what's bothering me is that (if I understood the video correctly) that it's kernel is running with ring 0 privileges. Is there any particular reason/advantages in doing that?
i once (kinda) wrote an operating system (for an atmega 2560) that can (sort of) deal with multitasking and i'd say it's more complex then youe standard c programm for an arduino, but still nowhrere near a propper os like Linux.
but i learned a lot kn writing it
I'm too nooby to understand a lot of this but it's definitely interesting how using cpu processes can be used to bypass restrictions placed on an operating system.
bro you need to compile everything from hardware exploitation to software exploitation in a course would be very grateful I can get a good hold I have Arduino and learnt c and C++ since they are used for embedded systems but still feel I haven't got anywhere please help me
seems like you would enjoy programming PIC microcontrollers
That makes sense. I know you can write inline ASM in C but I have always wondered how to talk to hardware.
I can't even understand a bit of what ya are doing 😅I am just watching the video cause you sound Hyped :P
that's literally what i was facing at the moment. thank you so much for making this video
holy shit I tought it was Michael Cera
I don't know why only 2 comments mentioned that. It should like half lol
Did anyone else pause a lot for free real estate?
Yes I'm aware of the slow playback function, but it feels more like a video game.
That's shockingly bad security.
Neat. So like how to drive a TESLA of the road and into a ditch LMAO.
Wow, this is awesome, thanks for taking the time to explain!
Title suggestion: "But what *is* kernel I/O? - a SerenityOS exploit"
I think the title is more of an expression of his though train you know? He explain it that he doesn't really know the bridge between the hardware and the kernel. And then when he understand the exploit, he finally understand how CPU Access Hardware
One of my favourite videos of LiveOverflow!
Great title
this was perfect for me. thank you
I missed this exact link just like you.
Bro have you heard of manscapped?
nobody :
liveoverflow : leet - leet = Ø
daaaamn... you asking the same questions as I do about how the computer works! - That goddamn keyboard...HOW DOES IT WORK!?!? ps. I am only 5:15 into the vid - hopes you will enlighten me with all the answers I need !!
For a better feel of x86 hardware I/O basics, you could always spin up an 8086 emulator and run DOS in it. Since back then there wasn't any hardware privilege isolation and it was closer to today's MCUs. Later processor versions just layered more features on top.
I really appreciate your kind of explaining things in your videos in a really good way. I love your videos and just wanna say thanks 🙏 to you ☺️. I really like your character, too!
Very nice and informative video! Just a couple of mistakes about the AVR assembly:
@14:02: The arguments of “sbi” are backwards: first comes the port number, then the bit number.
@14:13: The second argument of “out” is not an immediate value to be written to the port, it's the index of the CPU register that will be copied there.
Actually even some microcontrollers have the concept of hardware privilege levels. For example all ARM Cortex-M cores have two privilege levels, when a core is also equipped with a MPU (Memory Protection Unit) you can also restrict access to some memory areas for unprivileged code. They all also have syscall instruction, so you could implement a basic OS on an ARM microcontroller - which I definitely recommend - it's great learning experience especially since the architecture is so much simpler from x86.
Wow 😍😍😍 this was very enlightening on how x86 processors work and that missing gap I also didn’t understand very much
Please keep in mind, that PS/2 is not only the "old-school keyboard port". It is still being used in many modern notebooks since it is interrupt-driven. Therefore it doesn't have to poll the USB interface all the time. This approach can save some power on mobile devices.
Coming from eletronic engineering I had the (wrong) impression that I/O on ports was a microcontroller thing and microprocessors used memory mapping.
I see now I was wrong.
Thank you very much for clarifying that, it was really useful!!
Huh, interesting, I never knew about the IN and OUT instructions in AVR assembly, or any assembly for that matter. When I wrote AVR code to access GPIO pins without the arduino stuff, I always just read and wrote directly from/to the PORTA and PORTB registers, which would to the same as digitalRead/digitalWrite. I'm guessing this is the equivalent to "memory mapped access", compared to port based access with the IN and OUT instructions?
“OUT” is a machine instruction you use when programming in assembly. “PORTB” is a macro of the avr-libc you use when programming in C. The compiler then translates that to whatever it thinks is suitable, most of the time to port-mapped I/O.
Redstone Computers got me started
Awesome work ...was still trying to understand kernel ,BIOS and processor plus hardwares work.
Ptrace is the key.
For anyone dealing with these kind of questions: Write a basic operating system for a raspberrypi. If you have a certain amount of experience, it's not that hard and you don't need to implement everything. Just a uart driver maybe that you can send messages through. You will read a technical manuals and data sheets. And you will learn a lot. Reading a cpu manual for the first time, it is surreal. I remember having this weird bug in the beginning where I wasn't realizing that an interrupt changes the processor mode and different processor modes have different stacks, so my interrupt stack wasn't initiallized. That's a bug where google won't help haha. Anyway, I can recommend it dearly for a side project.
I too had a "what the heck?" moment at the hardware/software interface. In my case it is sort of "why does software sort of 'disappear' from view at a certain point, and what is it that the abstraction is removing?" As for the exploit, interesting, but: given the IO ports, does this exploit assume that there is a ATA type disk available? In particular, would one have to rewrite it on a SCSI system (for example)?
I knew about this concept existing in old game consoles thanks to Retro Game Mechanics Explained (game controllers, audio chip, cartridge, etc), but never thought this still applies to modern CPUs today!
i just read something about IOPL and IOPERM and forgot what it was, and now I see it mentioned as a crucial point of this video and wish I remembered.
Seriously, how long did it take for you to prepare such a topic/video 🤨?
That missing link between user space programs and hardware...
I feel exactly the same about it !
How to learn hacking...
Is there a place to make video requests? I'd love more ghidra videos, I've been able to find very few quality resources for it.
I also understand what it's like to know the low level of computers (more or less) and I know how to make programs for operating systems, and in this video I identified myself a lot. I realized that people who develop operating systems (especially if it's from scratch
) are people who could build (theoretically) a computer from silicon to make an operating system, and then make the software for own OS. This is amazing and i want to do that too.
I can really recommend building a risc v core on an FPGA. It is very interesting an you have the opportunity to build Interface logic yourself (like adressable hardware)
I had no clue assembly had a command to directly do port I/O, I assumed everything was memory mapped (and that data magically got to the desired device after writing to memory)
Your content is awesome. Thank you for it. You got me into my cybersecurity career. Thank you Liveoverflow.
Well. The concept is not that hard to understand. But i still have not much of about how to start implementing it into an actual exploit^^ I guess i have to write to some executable file (bash or whatever) and set a uid bit or something. Without messing up the file system. How does accessing the first sector help me out on that?
This was so good! Thanks for sharing this. Still not 100% sure whats going on between the userland and hardware, but way smarter now than before I watched this video.
This is so cool, I've just been spending the last couple days wondering this same thing.
Thank you for making such high quality and well edited videos :)