@__kvik Pointed out that even smaller executable sizes can be achieved by using Zig cc with musl. With what I have tested, sizes with Zig are 7.8K on the only-C version and 6.4K with the C & Asm mixed version. Command to build with Zig: "zig cc -Os -target x86_64-linux-musl -static sys.S shell.c", I'll update this comment if I get the size of the shell any lower (and also welcome to offer suggestions in the comments)
zig might be doing this already but I think link time optimization would probably help reduce size without having to go without libc? Alternatively with the assembly version, I wonder if you can get it to be smaller by using a non standard executable format (like a.out?), but I won't bother to check since setting up the tooling is quite a hassle
Entirely pointless to use Zig for this. Zig is just using LLVM, which is a drop-in replacement for GCC in most cases, which you already use. You just don't pass the flags necessary to create an even smaller executable. Highly recommend reading Chris Wellons' blog, where he mostly focuses on Windows. The smallest possible Windows executable to print something without an import table and anything fancy beyond ASM entrypoint + normal C code for example is 1KB. You can get similar results on Linux as well.
@@terraria9934 Yeah, no reason why not. I guess loading a default script would be a bit trickier, but you could write a shell in lua and have it available by running require "shell"
As someone who daily drives Linux From Scratch, I have to say this is one of the clearest and most to-the-point videos I've seen on low-level Linux concepts-beyond interesting.
@@vaibhavmishra5179 LFS is a hugely fun project that takes shockingly little time to complete, considering what you get. I remember the first time I spun up Xorg I had to first reread the docs a few times since I had a hard time believing it was that straightforward. It's amazing how modular Linux is.
@@vaibhavmishra5179I believe @PerfectT2ste, I've done daily driving LFS also, many times since about LFS 6. I make myself do it again every few years, then the novelty wears off and I go back to the normal stuff (i3 on arch). I learn an insane amount of new stuff each time, there's no better way to catch up on what's going on in this space. Your reaction is warranted, it's not for the faint of heart. Dependency hell is real. :)
@@filip_sedlak This not the same thing, but I used to run Doom 3 as my xinit process when I wanted to play because it would significantly improve the framerate over launching from Gnome.
@@DimDima09 i made a small iso that boots doom made using busybox and also a version with buildroot. You can patch the init file to run fbdoom but in my Experience i cant run it as the main init process
Really cool video. Very interesting to see the different steps and it helps you understand how the kernel is built and how the whole system works in general.
"in just a handful of KB, not MB" -- it's called BASIC for a reason ;3 Then again Fabrice Bellard (same madlad who created ffmpeg and JSLinux) also created QuickJS, which supports the ES2023 standard in ~210 KB of code, which is smaller than the Lua interpreter, a notoriously bare-bones language. 👀
Bro no way, just about a week back I was learning how to compile Linux kernel from scratch for RISC-V and now I see this vid. This vid is so nice as a minimal example for playing with a kernel. I learned a lot, thanks!
Mad props for going right down to assembly and detailing really every step, and not glossing over minor mistakes either, but rather showing it as it is. I especially enjoyed it being mostly one-terminal with vim and tmux A suggestion: Please look into keypressOSD: I would love to see work done like this with a ticker-feed of actual keystrokes. No need to go slow if framestepping through the video is enough to figure out what the keyboard was getting, and I think that would add a lot to the pedagogal value of content like this.
Holy shit! This is the most effective linux build tutorials I've ever laid my eyes on! I needed this kind of starter tutorial to start playing around instead of elaborate detailing on all the bells and whistles of linux build tooling or some other thing! Thank you OP!
This Video is great! I have never seen such a simple tutorial on kernal building. No overhead and straight to the point. Keep going! Minor note: you could change the flags in make from the command line without changing the Makefile but maybe you are right by not making it more complex.
We didnt just get a video on tiny linux, we got a video on creating a shell, learning to mitigate the C stdlib AND how to build lua for said distro that you also build to iso....A fucking banger, well done.
Great content! I have deployed some super minimal edge computers in a manufacturing setting to run single programs. (communications back to ERP) Don't have screen output, take in character input through a barcode scanner, play sounds for user feedback, and use the gpio pins to have some status lights. Used the lepotato platform, but could easily do it on a raspberry pi or any other SBC provided there is networking support. Deployments like this aren't my specialty, so I just used Debian stripped down to the basics and it works great. But now I can see another level deeper would be I could write a custom init process for my application and use a custom kernel. I would have to handle DHCP with init as well, but otherwise would be very direct. Application was developed in Rust (yea, I know crucify me), so already statically linked. Debian was nice for initial roll out for debugging in the field. I could ssh into the endpoints, read log files, upload code changes, etc. But now that the solution is in a stable state these functions are vestigial. Though the setup is a bit more involved, I could see the advantage of this being a resultant simpler execution environment and a stronger default security stance (less running means less attack surface). While my deployment scale doesn't call for this level of effort, none-the-less its a stimulating thought experiment in deployment engineering. Thanks for the great video!
@@mipsters A bit tongue in cheek, but there has been spicy controversy around Rust especially in the Linux Kernel developer world with attempts to integrate Rust into Kernel development. Mostly Rust gets pitted against C and a lot of the arguments thrown around are charactures or badly informed on both sides. There are some vocal people against Rust. You can lookup some videos from the recent Linux Plumbers conference if you are interested. Anywho, thanks for the boost. It was a fun project really helped with executing business operations. I have found Rust pretty great for application development. The compile time guarantees, static typing, concurrency model, and trait system in my experience are top notch.I built a couple hyper stable and fast applications quickly with it.
This is great. The explanation makes it so much simpler than I thought it would be! This video has given me many ideas for potential single purpose images I could run to deploy little containerised apps. Super cool!
Interesting video, but i have three things to say. 1. For your minimal shell, I think it would be better to write the entire thing in assembler. You're already writing assembler to create C bindings for the syscalls. You also get more control over eliminating suboptimal instruction encodings and useless instructions. I reimplemented your shell in assembler and was able to get it down to 760 bytes, versus your 9.1KB. 2. If you're going to compile libc and link it statically with lua, what was the point of eliminating libc from your shell? You could just link dynamically across both binaries and save space through the shared code. I'd recommend Musl libc for a tiny libc. 3. If the only program you're able to run from your shell is the lua interpreter, why not just make that your init process? You could eliminate the shell entirely, then static linking of libc would make sense since it's the only program on the initramfs.
dynamically linking shared libc probably a good idea although you then need to include ld.so as well, bur I guess it pays off if you need to add more programs
I agree on most of this. This seems like a wasted opportunity to give Musl more exposure. I can't use it because I use many glibc-dependent programs, but in this case, it would be the superior choice. On the other hand, the issue with making regular programs the init process is that init needs to have the ability to reap dead processes. If, for some reason, the application-as-init ever ends up spawning subprocesses, I suspect the OS won't have a very long runtime.
Awesome video, it's always been hard figuring out how to do some of this stuff, but this explains everything pretty clearly. If you're ever in the mood, I have been struggling to find clear explanation how to boot something "bare-metal," with no kernel. Like using qemu to run a program that writes to the serial address of an esp32 to print something.
I love this. Is bash that big? Maybe just use lua with a run/exec command? Add a few drivers, some basic service support (a dir of shell scripts that run on startup?), busybox, something like mingetty for a basic login, a basic wayland window manager (to not worry about a separate x server) networkmanager and flatpak and then you should be able to run stuff like firefox and steam. That's like 5 core packages + kernel that you'd recompile when you want a system update, that would surely make for a fairly minimal but also fully featured distro to rival tinycore.
Yes Bash is big compared to shells in Busybox. Flatpak is humongous in comparison to downloading shared objects from the right git repo to the right /lib directory, or taking configuration from your server's /etc to your system's /etc. Having a server system to compile the software any way you like is way more efficient, but would take effort.
@@tiffcz3863 Yeah, I can see it defeating the point if size on disk was the one goal, but I'm kind of just looking at going from zero to fully featured in a minimum number of moving parts if that makes any sense.
initramfs support is required since I use it to load up the init cpio which holds the userspace. it is true that initramfs is useful for dynamically loading drivers before the main root fs is loaded from disk, but here I use it for the whole system (and there is no root fs that comes from disk)
Best I've seen is ~2kb with GeckOS, although that's more another Unix-like targeting the 6502. IIRC, older Linux versions could handle as low as 4MB of RAM too. Maybe update the title as "Modern Linux" to avoid confusion! :D I kid, of course. Great vid, I'm saving for later.
1) surprised you didn't start with adding some gcc flags to make it smaller, and jumped straight to assembly, like just add -Oz -ffunction-sections -fdata-sections -Wl,--gc-sections and maybe also -flto and you'll see it can get a LOT smaller 2) zig is irrelevant here, zig's using LLVM behind the scenes, so it's same as clang basically, all the magic is in using musl, you can use "mussel" project to easily make a gcc toolchain that uses musl (gcc also makes sliiiightly smaller binaries), there's also even smaller libc variants, but with those it gets farther and farther from "just write code" and more into "write code for this specific libc" there's a loot of fiddling that can be done for "smallest" stuff, including the fact you can use UPX to "pack" most elf executables smaller, with a tradeoff that you basically end up 2x the size of the binary, in ram (it has to unpack it and execute it, but the physical file is also in ram), which is neglegible in this case
Wow!!! Your videos are the best!!! Awesome!!!! By the way, you could use strip utility to get a smaller lua executable. Thank you and keep making this kind of work!!!!
That’s great! Especially for embedded HW with limited resources/running a single task. I need to dust off the Milk-V Duo board for this… (not x64, but at least some challenge in figuring out the assembly)
It's too bad gcc doesn't have a "-minimal" flag or similar to complement the -static flag which would build using only using the necessary functions and symbols as opposed to embedding the included libraries whole.
Okay, ignorant question here but: doesnt the kernel have a minimal libc nowadays? Could you link against that, or is it just not usable for something like this?
You can do this: gcc -o shell shell.c sys.S --entry main -z noexecstack -static (gcc can compile C, and GNU Assembly, and then link it together.) you can also use -Os to optimize for size. Also, to remove all libc, you can use the -nostdlib argument You can also remove unnecessary information using the strip command. Also, here is an implementation of your basic shell in only 435 bytes (I wrote it in fasm as I am more familiar with that one.) format ELF64 executable ; I/O STDOUT = 1 STDIN = 0 ; Wait P_ALL = 0 WEXITED = 4 ; Syscalls SYS_read = 0 SYS_write = 1 SYS_fork = 57 SYS_execve = 59 SYS_exit = 60 SYS_waitid = 247 ; Syscall helpers macro syscall0 number { mov rax, number syscall } macro syscall1 number, arg1 { mov rax, number mov rdi, arg1 syscall } macro syscall3 number, arg1, arg2, arg3 { mov rax, number mov rdi, arg1 mov rsi, arg2 mov rdx, arg3 syscall } macro syscall5 number, arg1, arg2, arg3, arg4, arg5 { mov rax, number mov rdi, arg1 mov rsi, arg2 mov rdx, arg3 mov r10, arg4 mov r8, arg5 syscall } segment readable executable entry main main: ; Write the "# " thing syscall3 SYS_write, STDOUT, write_text, 2 cmp rax, 0 jl .write_error ; Get the command syscall3 SYS_read, STDIN, command, 255 cmp rax, 0 je .read_error ; Replace the newline add rax, command sub rax, 1 mov byte [rax], 0 ; Fork it syscall0 SYS_fork cmp rax, 0 je .execute ; Wait for it to finish syscall5 SYS_waitid, P_ALL, 0, siginfo, WEXITED, 0 jmp main .execute: ; Run the command syscall3 SYS_execve, command, 0, 0 cmp rax, 0 jl .execve_error syscall1 SYS_exit, 0 .write_error: syscall1 SYS_exit, 1 .read_error: syscall1 SYS_exit, 2 .execve_error: syscall1 SYS_exit, 3 segment readable writable write_text: db "# " ; don't care about the content, but it's 128 bytes siginfo: rb 128 command: rb 255
Very interesting video, but there seems to be an error. I've tried on Debian and Arch and both segfault at main() in the 2nd shell created around 20:30. I've downloaded the code directly from your git repo. Turning on -Wall there is a warning that argument 2 of execve should not be null, as specified in unistd.h. Tested with GCC 14.2.1 on Arch, 13.3.0 on Debian. I'm not sure if this is the source of the error.
I love your contents! BTW, you could pass -Os or -O3 to gcc to save some space; also why not `strip' the executable? Also, it would be much more interesting if your userland could create new executable files (which Lua cannot). Thank you!
Yep. I built the C version of his shell with musl libc and -Os and it turned out to 4.7k, so beating his ASM version by not even trying. Also built Lua with the same config and it's 905K or 311K stripped.
@__kvik I am getting 26K with musl libc, so still more than twice as big compared to my ASM version with GCC. I ran "musl-gcc -static -Os shell.c", what command have you used to build?
Probably something to do with it being all done by hand showing (or mentioning) every keystroke. And to do with linux being quite a large monolithic kernel, quite famously, by design. (So, actually about the worst choice if you're aiming for 'smallest possible', at least so far as kernels go). Actually there are ways to make a system become a node of a supercomputer with not a lot more than this: mostly just building in support for a network card, and providing some way to spin-off tasks to it. IIRC There used to be a way to do this such that you could open a specific port and bake-in an IP address, then the kernel would run essentially as a co-processor for another network-connected machine, despite being paniced at not being able to run INIT. This was an early workaround for crosscompiling stupid C++ dependencies early on with arm on linux, because C++ libraries like gtkmm incorrectly assumed that the configure script would of course always be able to build and run local executables to automatically determine what were essentially makefile settings, at configure time. Unix-style cross compilation is quite interesting, and this 'trick' could be used so that the crosscompiling system could have a way to 'farm out' executables matching the target architecture to actual hardware. Ie, with the very minimal kernel crashed on it without INIT, but with the patch needed for it to be usable this way, and with the dev system's root fs nfs mounted via kernel option setting. It would of course crash because it would find the host's x86 INIT executable, and not an ARM one. But fortunately it would indeed find and execute any little 'test' executable the overly smartelec configure script would run, and even write the output where expected as well. The library would thus end up crosscompiling successfully anyway, not having any idea it was doing so. In actual fact, having a complete system such as LUA running like this is probably much more useful than MSDOS, considering that it supports current hardware in 64bit mode, and can immediately be used as a 'LUA computation node' as part of a larger system, potentially dealing with quite large datasets. And he didn't really scratch the surface of what Linux can do if you build in additional options, just really focused quite nicely on the absolute minimum in what I thought was quite a well thought out demonstration / tutorial guide of basic bare-metal Linux setup. If you really want extreme small, then you'd probably not choose linux: Yet running 'bare metal' Linux like this is a very valid choice: particularly where it already has kernel support for the hardware you are targetting. Saves you from 'reinventing the wheel' in implementing drivers and many other OS level services, whilst still giving you the freedom to just include *really* only what you want. A few MB is more than small enough, given how big EEPROM chips are nowadays. He does mention that he could have got it smaller, obviously by performing much the same minimal assembly to remove the C library depedancy as he already demonstrated. (or by using a substitute C libary specifically aimed at such, which he does mention is a good option). This approach demonstrated could be applied to any self-contained system, eg, SQLite would work really well, and potentially be very useful here. You would run this as one node of a cluster system, using something like a FreeBSD cluster elsewhere running a big ZFS array for the NAS. So, actually useful as compared to MSDOS.
@ It introduces pointless idiosyncrasies that make code hard to read, pointlessly muddying things in ways that can easily be mistaken, such as swapping from operation destination source to operation source destination. It introduces extra confusing syntax for little to no benefit over its Intel counterpart
Nice bait for newcomers, even if it doesn't explain much, it shows that the basic system is quite simple. Of course, step 0 would be stating that there is no abstract smallest distribution, the limit only makes sense if you have a defined set of tasks to perform. If you have no requirements, distribution of zero size suits you. If your task is to run an up-to-date web browser (which is almost an operating system in side your operating system today), you need quite a lot of stuff. That's the place for TinyCore and other “construction kit” distributions. You can choose smaller size and software fallback for everything (including graphical server), or bigger package with extra dependencies for intended use of modern hardware. Given that you may need a gigabyte of memory just to open a random web page in a browser (not to mention UA-cam), the former option makes little sense. The difference simply doesn't matter for most use cases. Even if you want to make a horrible extra cheap mobile device, you can stuff it with enough flash memory for a full-featured system. Also, you displayed system call numbers for 32-bit x86 kernel. For simplicity, legacy set matches in 64-bit systems.
@__kvik Pointed out that even smaller executable sizes can be achieved by using Zig cc with musl. With what I have tested, sizes with Zig are 7.8K on the only-C version and 6.4K with the C & Asm mixed version. Command to build with Zig: "zig cc -Os -target x86_64-linux-musl -static sys.S shell.c", I'll update this comment if I get the size of the shell any lower (and also welcome to offer suggestions in the comments)
@@nirlichtman how about luajit compiled with musl since it has support for direct bindings to C would seem fitting :D
zig might be doing this already but I think link time optimization would probably help reduce size without having to go without libc? Alternatively with the assembly version, I wonder if you can get it to be smaller by using a non standard executable format (like a.out?), but I won't bother to check since setting up the tooling is quite a hassle
would it be posible to boot directly into lua instead of into the shell?
Entirely pointless to use Zig for this. Zig is just using LLVM, which is a drop-in replacement for GCC in most cases, which you already use. You just don't pass the flags necessary to create an even smaller executable. Highly recommend reading Chris Wellons' blog, where he mostly focuses on Windows. The smallest possible Windows executable to print something without an import table and anything fancy beyond ASM entrypoint + normal C code for example is 1KB. You can get similar results on Linux as well.
@@terraria9934 Yeah, no reason why not. I guess loading a default script would be a bit trickier, but you could write a shell in lua and have it available by running require "shell"
"For this we're going to write a little bit of assembly.." this is what taking the red pill must feels like, seeing how deep the rabbit hole goes.
As someone who daily drives Linux From Scratch, I have to say this is one of the clearest and most to-the-point videos I've seen on low-level Linux concepts-beyond interesting.
@@PerfectT4ste 'Daily drives'? 🤯
@@vaibhavmishra5179 LFS is a hugely fun project that takes shockingly little time to complete, considering what you get. I remember the first time I spun up Xorg I had to first reread the docs a few times since I had a hard time believing it was that straightforward. It's amazing how modular Linux is.
@@vaibhavmishra5179I believe @PerfectT2ste, I've done daily driving LFS also, many times since about LFS 6. I make myself do it again every few years, then the novelty wears off and I go back to the normal stuff (i3 on arch). I learn an insane amount of new stuff each time, there's no better way to catch up on what's going on in this space. Your reaction is warranted, it's not for the faint of heart. Dependency hell is real. :)
Bro compiles his OS himself! Take that arch users.
What System do you have? On my system compiling chromium alone takes 9h.
I love the small little mistake at 20:30
It's always nice when small errors are kept like that because it shows no one is immune to making mistakes
This man explained his process better than any other kind of tutorial that I have ever seen, even though this was just created for fun
This is fascinating. How about a minimum distro that will boot straight into DOOM.
Hm, how about making it as init process? Will it work?
@@filip_sedlak This not the same thing, but I used to run Doom 3 as my xinit process when I wanted to play because it would significantly improve the framerate over launching from Gnome.
UEFI Doom?
@@DimDima09 i made a small iso that boots doom made using busybox and also a version with buildroot. You can patch the init file to run fbdoom but in my Experience i cant run it as the main init process
@@309electronics5 dude we need a GitHub link.
Really cool. You can still remove debug symbols with strip for extra ~0.5kB on shell and ~100kB on lua.
let's get this man the highest retention so the algorithm pushes him to good places
Really cool video. Very interesting to see the different steps and it helps you understand how the kernel is built and how the whole system works in general.
Makes me appreciate things like ATARI rom that have basic built-in, with some other OS utilities, in just a handful of KB, not MB
"in just a handful of KB, not MB" -- it's called BASIC for a reason ;3
Then again Fabrice Bellard (same madlad who created ffmpeg and JSLinux) also created QuickJS, which supports the ES2023 standard in ~210 KB of code, which is smaller than the Lua interpreter, a notoriously bare-bones language. 👀
You have a fantastic style of presentation, always showing the synopsis and allowing time to read before writing more code.
Bro no way, just about a week back I was learning how to compile Linux kernel from scratch for RISC-V and now I see this vid. This vid is so nice as a minimal example for playing with a kernel. I learned a lot, thanks!
Did you get it working? Learning riscv is one of my goals.
I hope no one gets inspired to release yet another distro that no one needs from this informative and educational video.
Yeah, It all seems totally pointless.
Mini Linux is coming out soon lol
jukes on you i'm releasing bussinMiniOS
"wahh too many distros wahhh"
Who tf cares if there's a million distro's to choose from? This is the lamest mindset in the Linux community ong.
Mad props for going right down to assembly and detailing really every step, and not glossing over minor mistakes either, but rather showing it as it is.
I especially enjoyed it being mostly one-terminal with vim and tmux
A suggestion: Please look into keypressOSD: I would love to see work done like this with a ticker-feed of actual keystrokes.
No need to go slow if framestepping through the video is enough to figure out what the keyboard was getting, and I think that would add a lot to the pedagogal value of content like this.
Enthralling. I didn’t follow absolutely every step, but I never felt completely lost and learned a lot. Amazing journey, thank you.
Holy shit! This is the most effective linux build tutorials I've ever laid my eyes on! I needed this kind of starter tutorial to start playing around instead of elaborate detailing on all the bells and whistles of linux build tooling or some other thing! Thank you OP!
This is one of the best tutorials I saw in a very long time. Thank you so much. This was really fun to watch!
The quality of your tutorial is insane. The thing is better than this is to show how documentations can be followed when you don't know.
This Video is great! I have never seen such a simple tutorial on kernal building. No overhead and straight to the point. Keep going!
Minor note: you could change the flags in make from the command line without changing the Makefile but maybe you are right by not making it more complex.
We didnt just get a video on tiny linux, we got a video on creating a shell, learning to mitigate the C stdlib AND how to build lua for said distro that you also build to iso....A fucking banger, well done.
super interesting! I adored Linux from Scratch but couldn't get all the commands to work 100%. This is way simpler
Great content!
I have deployed some super minimal edge computers in a manufacturing setting to run single programs. (communications back to ERP)
Don't have screen output, take in character input through a barcode scanner, play sounds for user feedback, and use the gpio pins to have some status lights. Used the lepotato platform, but could easily do it on a raspberry pi or any other SBC provided there is networking support.
Deployments like this aren't my specialty, so I just used Debian stripped down to the basics and it works great. But now I can see another level deeper would be I could write a custom init process for my application and use a custom kernel. I would have to handle DHCP with init as well, but otherwise would be very direct.
Application was developed in Rust (yea, I know crucify me), so already statically linked.
Debian was nice for initial roll out for debugging in the field. I could ssh into the endpoints, read log files, upload code changes, etc. But now that the solution is in a stable state these functions are vestigial.
Though the setup is a bit more involved, I could see the advantage of this being a resultant simpler execution environment and a stronger default security stance (less running means less attack surface). While my deployment scale doesn't call for this level of effort, none-the-less its a stimulating thought experiment in deployment engineering.
Thanks for the great video!
Was I under a rock? Why would you be crucified for Rust?
I thought everybody loved it
Also, the system you built sounds very cool
@@mipsters A bit tongue in cheek, but there has been spicy controversy around Rust especially in the Linux Kernel developer world with attempts to integrate Rust into Kernel development. Mostly Rust gets pitted against C and a lot of the arguments thrown around are charactures or badly informed on both sides. There are some vocal people against Rust. You can lookup some videos from the recent Linux Plumbers conference if you are interested.
Anywho, thanks for the boost. It was a fun project really helped with executing business operations. I have found Rust pretty great for application development. The compile time guarantees, static typing, concurrency model, and trait system in my experience are top notch.I built a couple hyper stable and fast applications quickly with it.
Wow! This is great to learn. I thought tiny core linux at 10mb was the smallest.
This is great. The explanation makes it so much simpler than I thought it would be!
This video has given me many ideas for potential single purpose images I could run to deploy little containerised apps. Super cool!
"stop typing so loud" lol -the lovely people I work with
Anyone else started their Sunday with coffee and this video ?
No ? Just me ? You don't know what you're missing. 🎉
I am always fascinated by your content and leaving learning something new, thank you.
Interesting video, but i have three things to say.
1. For your minimal shell, I think it would be better to write the entire thing in assembler. You're already writing assembler to create C bindings for the syscalls. You also get more control over eliminating suboptimal instruction encodings and useless instructions. I reimplemented your shell in assembler and was able to get it down to 760 bytes, versus your 9.1KB.
2. If you're going to compile libc and link it statically with lua, what was the point of eliminating libc from your shell? You could just link dynamically across both binaries and save space through the shared code. I'd recommend Musl libc for a tiny libc.
3. If the only program you're able to run from your shell is the lua interpreter, why not just make that your init process? You could eliminate the shell entirely, then static linking of libc would make sense since it's the only program on the initramfs.
dynamically linking shared libc probably a good idea although you then need to include ld.so as well, bur I guess it pays off if you need to add more programs
I think it's to show the size of the minimal install without static linking libc, before adding it and Lua.
I agree on most of this.
This seems like a wasted opportunity to give Musl more exposure. I can't use it because I use many glibc-dependent programs, but in this case, it would be the superior choice.
On the other hand, the issue with making regular programs the init process is that init needs to have the ability to reap dead processes. If, for some reason, the application-as-init ever ends up spawning subprocesses, I suspect the OS won't have a very long runtime.
You are a genius man. Learning so much from your videos!
The entire time watching this, all I could think was "I am not worthy! 😢"
Amazing stuff.
Excelolent explanation.
Thank you for this, you made the description very clear,
woww this is an amazing video. really interesting to see how everything plays into each other!
Awesome video, it's always been hard figuring out how to do some of this stuff, but this explains everything pretty clearly. If you're ever in the mood, I have been struggling to find clear explanation how to boot something "bare-metal," with no kernel. Like using qemu to run a program that writes to the serial address of an esp32 to print something.
I love this. Is bash that big? Maybe just use lua with a run/exec command?
Add a few drivers, some basic service support (a dir of shell scripts that run on startup?), busybox, something like mingetty for a basic login, a basic wayland window manager (to not worry about a separate x server) networkmanager and flatpak and then you should be able to run stuff like firefox and steam. That's like 5 core packages + kernel that you'd recompile when you want a system update, that would surely make for a fairly minimal but also fully featured distro to rival tinycore.
Yes Bash is big compared to shells in Busybox. Flatpak is humongous in comparison to downloading shared objects from the right git repo to the right /lib directory, or taking configuration from your server's /etc to your system's /etc. Having a server system to compile the software any way you like is way more efficient, but would take effort.
@@tiffcz3863 Yeah, I can see it defeating the point if size on disk was the one goal, but I'm kind of just looking at going from zero to fully featured in a minimum number of moving parts if that makes any sense.
Holy shit, I didn't know an ISO could be so small
Why not?
You don't need initramfs support if all your modules are built into the kernel.
initramfs support is required since I use it to load up the init cpio which holds the userspace. it is true that initramfs is useful for dynamically loading drivers before the main root fs is loaded from disk, but here I use it for the whole system (and there is no root fs that comes from disk)
Best I've seen is ~2kb with GeckOS, although that's more another Unix-like targeting the 6502. IIRC, older Linux versions could handle as low as 4MB of RAM too. Maybe update the title as "Modern Linux" to avoid confusion! :D I kid, of course. Great vid, I'm saving for later.
The fact that this could fit on a floppy is amazing
Thank you for sharing your genius! Amazing video! Bravo!
Wow, thank you!
Very interesting and nicely filmed!
wow, really impressive stuff. Although most of the stuff on this video won't help me directly, I still learned so many things.
really nice... I have not done that on linux... done similar stuff on BSD, where it is much more common to compile your own kernel.
1) surprised you didn't start with adding some gcc flags to make it smaller, and jumped straight to assembly, like just add -Oz -ffunction-sections -fdata-sections -Wl,--gc-sections and maybe also -flto and you'll see it can get a LOT smaller
2) zig is irrelevant here, zig's using LLVM behind the scenes, so it's same as clang basically, all the magic is in using musl, you can use "mussel" project to easily make a gcc toolchain that uses musl (gcc also makes sliiiightly smaller binaries), there's also even smaller libc variants, but with those it gets farther and farther from "just write code" and more into "write code for this specific libc"
there's a loot of fiddling that can be done for "smallest" stuff, including the fact you can use UPX to "pack" most elf executables smaller, with a tradeoff that you basically end up 2x the size of the binary, in ram (it has to unpack it and execute it, but the physical file is also in ram), which is neglegible in this case
One of the best video and explanation related to distro :)
Wow!!! Your videos are the best!!! Awesome!!!! By the way, you could use strip utility to get a smaller lua executable. Thank you and keep making this kind of work!!!!
thanks, good point!
Thanks for that amazing video
❤
I feel like most of the video is about writing a minimalist shell in asm from scratch...
still entertaining to watch
because obviously kernel is given so need shell and lang.
Using windows wsl to write a linux distro in vim is crazy work. My kinda crazy.
can you cross compile to arm microcontrollers? or riscv?
Somehow i wouldn't have expected you running Windows ...
he's probably trying to make linux apps as .exe by making each it's own WSL distro
Watching this makes me long for bloated distros. Fun video!
That’s great! Especially for embedded HW with limited resources/running a single task. I need to dust off the Milk-V Duo board for this… (not x64, but at least some challenge in figuring out the assembly)
What a legend he didnt make a linux distro bro this is a whole os that is just based on the linux kernel and its from scratch
And its small too! Minimal bloat
Always high quality content
Great information, great work!
I learned a lot. Thank you.
Great info thanks for sharing
I didn’t even watch the video yet, but I’m saying it’s one of the best ever made!
Damn looks cool. Thank you!
It's too bad gcc doesn't have a "-minimal" flag or similar to complement the -static flag which would build using only using the necessary functions and symbols as opposed to embedding the included libraries whole.
Finally new vid 🎉🎉🎉
This was a great watch!!
Thanks a lot, nice video
Okay, ignorant question here but: doesnt the kernel have a minimal libc nowadays? Could you link against that, or is it just not usable for something like this?
You can do this:
gcc -o shell shell.c sys.S --entry main -z noexecstack -static
(gcc can compile C, and GNU Assembly, and then link it together.)
you can also use -Os to optimize for size.
Also, to remove all libc, you can use the -nostdlib argument
You can also remove unnecessary information using the strip command.
Also, here is an implementation of your basic shell in only 435 bytes (I wrote it in fasm as I am more familiar with that one.)
format ELF64 executable
; I/O
STDOUT = 1
STDIN = 0
; Wait
P_ALL = 0
WEXITED = 4
; Syscalls
SYS_read = 0
SYS_write = 1
SYS_fork = 57
SYS_execve = 59
SYS_exit = 60
SYS_waitid = 247
; Syscall helpers
macro syscall0 number {
mov rax, number
syscall
}
macro syscall1 number, arg1 {
mov rax, number
mov rdi, arg1
syscall
}
macro syscall3 number, arg1, arg2, arg3 {
mov rax, number
mov rdi, arg1
mov rsi, arg2
mov rdx, arg3
syscall
}
macro syscall5 number, arg1, arg2, arg3, arg4, arg5 {
mov rax, number
mov rdi, arg1
mov rsi, arg2
mov rdx, arg3
mov r10, arg4
mov r8, arg5
syscall
}
segment readable executable
entry main
main:
; Write the "# " thing
syscall3 SYS_write, STDOUT, write_text, 2
cmp rax, 0
jl .write_error
; Get the command
syscall3 SYS_read, STDIN, command, 255
cmp rax, 0
je .read_error
; Replace the newline
add rax, command
sub rax, 1
mov byte [rax], 0
; Fork it
syscall0 SYS_fork
cmp rax, 0
je .execute
; Wait for it to finish
syscall5 SYS_waitid, P_ALL, 0, siginfo, WEXITED, 0
jmp main
.execute:
; Run the command
syscall3 SYS_execve, command, 0, 0
cmp rax, 0
jl .execve_error
syscall1 SYS_exit, 0
.write_error:
syscall1 SYS_exit, 1
.read_error:
syscall1 SYS_exit, 2
.execve_error:
syscall1 SYS_exit, 3
segment readable writable
write_text: db "# "
; don't care about the content, but it's 128 bytes
siginfo: rb 128
command: rb 255
I did this; guess the final size with -Os! 767KB!
This was SO good!
Very interesting video, but there seems to be an error. I've tried on Debian and Arch and both segfault at main() in the 2nd shell created around 20:30. I've downloaded the code directly from your git repo. Turning on -Wall there is a warning that argument 2 of execve should not be null, as specified in unistd.h. Tested with GCC 14.2.1 on Arch, 13.3.0 on Debian. I'm not sure if this is the source of the error.
It works perfectly with the zig command in the pinned comment.
you did your kernel compiling inside wsl thats pretty cool, was it ws1 or ws2?
WSL2
11:37 or just use musl and call it a day?
i was expecting a busybox based thing not an entirely custom shell
He already has such a video: ua-cam.com/video/QlzoegSuIzg/v-deo.html
Makes me appreciate how small is a dos boot disk. 90KB or so, if I remember correctly
You could have used inline asm with gcc btw
Question wouldn't using compiler options like -Os on both the kernel and shell/init files result in smaller binary files?
💖💖💖💖
Next create minimal text file creator and include luac to be able to improve the distro from the distro itself
Do you have any interest in doing things like this for ARM64, especially some of the corresponding projects listed in your GitHub hello page?
this almost makes me want to try and learn C
Why do you need an initramfs/initrd? Wouldn't it be simpler and smaller to put init and lua directly on the ISO?
when i try to install any linux distro it give me Kernal Panic similer to the error that you got in this Video
can you Helo me With This
What's the boot time after minification?
wow nice. So what about a 32b version?
why do you use wsl?
thats very very cool
lets gooo, also lua from scratch btw
I love your contents! BTW, you could pass -Os or -O3 to gcc to save some space; also why not `strip' the executable? Also, it would be much more interesting if your userland could create new executable files (which Lua cannot). Thank you!
Yep. I built the C version of his shell with musl libc and -Os and it turned out to 4.7k, so beating his ASM version by not even trying. Also built Lua with the same config and it's 905K or 311K stripped.
@@__kvik Thanks for confirmation!
@__kvik Interesting, I'll try that out with musl and verify this, if so I will add a pinned comment with this information, thanks for bringing it up
@__kvik I am getting 26K with musl libc, so still more than twice as big compared to my ASM version with GCC. I ran "musl-gcc -static -Os shell.c", what command have you used to build?
@nirlichtman Is the code available somewhere, so I can try it as well? Thanks!
why do you need initrd? just boot from uefi
How small this is.
Reminiscing to the Amiga with a whole OS including a GUI allowing games/etc to run in under half a MB.
It's called Linux From Scratch™
by Gerard Beekmans.
You're welcome.
"64-bits Intel" is Itanium, not really supported already. X86-64 or AMD64 :-)
👍Yes!
Shame it won't fit on a floppy
it can stay on 1 floppy disk, rename it 1 floppy disk linux!
WOW!!!
i was waiting for thiss
Fascinating. I understood almost nothing.
Tiniest Linux possible, yet orders of magnitude larger executable than MSDOS, while appearing to have even less functionality. How come?
Probably something to do with it being all done by hand showing (or mentioning) every keystroke.
And to do with linux being quite a large monolithic kernel, quite famously, by design. (So, actually about the worst choice if you're aiming for 'smallest possible', at least so far as kernels go).
Actually there are ways to make a system become a node of a supercomputer with not a lot more than this: mostly just building in support for a network card, and providing some way to spin-off tasks to it.
IIRC There used to be a way to do this such that you could open a specific port and bake-in an IP address, then the kernel would run essentially as a co-processor for another network-connected machine, despite being paniced at not being able to run INIT. This was an early workaround for crosscompiling stupid C++ dependencies early on with arm on linux, because C++ libraries like gtkmm incorrectly assumed that the configure script would of course always be able to build and run local executables to automatically determine what were essentially makefile settings, at configure time. Unix-style cross compilation is quite interesting, and this 'trick' could be used so that the crosscompiling system could have a way to 'farm out' executables matching the target architecture to actual hardware. Ie, with the very minimal kernel crashed on it without INIT, but with the patch needed for it to be usable this way, and with the dev system's root fs nfs mounted via kernel option setting. It would of course crash because it would find the host's x86 INIT executable, and not an ARM one. But fortunately it would indeed find and execute any little 'test' executable the overly smartelec configure script would run, and even write the output where expected as well. The library would thus end up crosscompiling successfully anyway, not having any idea it was doing so.
In actual fact, having a complete system such as LUA running like this is probably much more useful than MSDOS, considering that it supports current hardware in 64bit mode, and can immediately be used as a 'LUA computation node' as part of a larger system, potentially dealing with quite large datasets.
And he didn't really scratch the surface of what Linux can do if you build in additional options, just really focused quite nicely on the absolute minimum in what I thought was quite a well thought out demonstration / tutorial guide of basic bare-metal Linux setup.
If you really want extreme small, then you'd probably not choose linux: Yet running 'bare metal' Linux like this is a very valid choice: particularly where it already has kernel support for the hardware you are targetting. Saves you from 'reinventing the wheel' in implementing drivers and many other OS level services, whilst still giving you the freedom to just include *really* only what you want. A few MB is more than small enough, given how big EEPROM chips are nowadays.
He does mention that he could have got it smaller, obviously by performing much the same minimal assembly to remove the C library depedancy as he already demonstrated. (or by using a substitute C libary specifically aimed at such, which he does mention is a good option).
This approach demonstrated could be applied to any self-contained system, eg, SQLite would work really well, and potentially be very useful here. You would run this as one node of a cluster system, using something like a FreeBSD cluster elsewhere running a big ZFS array for the NAS.
So, actually useful as compared to MSDOS.
I don't know why *anyone* would use the AT&T syntax. It has no reason to exist and it's a pain.
What's wrong with it?
@ It introduces pointless idiosyncrasies that make code hard to read, pointlessly muddying things in ways that can easily be mistaken, such as swapping from operation destination source to operation source destination. It introduces extra confusing syntax for little to no benefit over its Intel counterpart
Nice bait for newcomers, even if it doesn't explain much, it shows that the basic system is quite simple.
Of course, step 0 would be stating that there is no abstract smallest distribution, the limit only makes sense if you have a defined set of tasks to perform. If you have no requirements, distribution of zero size suits you.
If your task is to run an up-to-date web browser (which is almost an operating system in side your operating system today), you need quite a lot of stuff. That's the place for TinyCore and other “construction kit” distributions. You can choose smaller size and software fallback for everything (including graphical server), or bigger package with extra dependencies for intended use of modern hardware. Given that you may need a gigabyte of memory just to open a random web page in a browser (not to mention UA-cam), the former option makes little sense. The difference simply doesn't matter for most use cases. Even if you want to make a horrible extra cheap mobile device, you can stuff it with enough flash memory for a full-featured system.
Also, you displayed system call numbers for 32-bit x86 kernel. For simplicity, legacy set matches in 64-bit systems.
nope, those are x86 64-bit syscall numbers, notice the file is syscalls_64.h
@@nirlichtman My bad, brain fogged.
Cool
liked before watching