Making Simple Linux Distro from Scratch
Вставка
- Опубліковано 16 жов 2023
- In this video I will demonstrate how you can create a small and simple Linux distro from scratch, together with the kernel I will use BusyBox for the user mode utils and Syslinux for the bootloader.
Installations: apt get install bzip2 git vim make gcc libncurses-dev flex bison bc cpio libelf-dev libssl-dev syslinux dosfstools
Linux Git URL: github.com/torvalds/linux.git
BusyBox Git URL: git.busybox.net/busybox - Наука та технологія
![[柴犬ASMR]曼玉Manyu&小白Bai 毛发护理Spa asmr](http://i.ytimg.com/vi/0TsXQ7z2Dh4/mqdefault.jpg)
My guy really using Ubuntu on Windows Terminal with DWM to build a distro from scratch
i legit had to go to the bathroom bc of that
Your "Guy" mabye use WSL (Windows Subsystem For Linux)
@@skybloxstars'your "guy"' most definitely did not use WSL lmao, he literally spun up a docker container step 1
@@huhulili9021 Sorry
@@huhulili9021 docker desktop (windows) depends on wsl but it exposes the docker command to use in cmd/psh
This ~12 minute video explained it better than the entire "Linux From Scratch" book.
yeh
The difference is that as the video said, this setup isn't usable or stable for production. Linux From Scratch, if done properly, can be
I have run Linux since kernel 0.99 (Slackware on floppies) and never really knew/thought about the individual bits and pieces that make up a Linux system. This is super educational. I would love to see an expanded version of this that allows for a basic Linux distro (that includes everything that you mention it would need for production) but without any of the bloat that normally comes with distros. A tiny server Linux if you will. Also, just in general, thank you for making these videos - they're awesome! I appreciate the time and effort you put into them.
Thanks! That's a cool idea :) maybe also in the direction of making a minimalist distro for raspberry pi
@@nirlichtman - Yes - as is said "you stole words from my mouth", raspberry pi. now i use diet pi but I still want to see the development from start for educational means. and 1 more thing now my eyes will be looking for the files shown in your video as Linux boots up on my laptop 😆
@@nirlichtman Yes, that would be awesome!
@@nirlichtman, thank you for the videos, that you make. A minimalist distro for raspberry pi sounds nice. Please do it, when you've got time. Waiting for it.
agreed, it would be very helpful!
Wow, so this is the way to a truly unbloated distro bliss!
Good stuff!
real unbloated implies kernel only has necessary options to run on hardware, busybox without the extra cruft...
WinXP Desktop Pure genius How long ago?
@@nonenothingnull Bro! How deep goes the rabbit hole?!?
@@alexviralata1356 very, tcc instead of gcc, musl, etc..
@@nonenothingnull who cares about busybox, it's only needed for a second at the startup. The real pain is choosing only the needed kernel options, while also maximizing the performance.
Instead of
dd if=/dev/zero of=boot bs=1M count=50
you can also use
truncate -s 50M boot
truncate changes the size of a file. If the file does not yet exist, truncate creates it. If the size is shorter than it used to be, the file is in fact just "truncated", hence the name. Yet if the size is bigger, zeros are added at the end until the file reaches the desired new size.
Not only is the command shorter, it also has another advantage: truncate creates a sparse file on systems that support that. That may not matter in this example but it would matter if the file was 100 GB in size. In a sparse file, only blocks of that file that contain actual data also use real disk space. All blocks that are still zero filled are virtual, they don't use any disk space and they are also not actually created when you created the file, so file creation is also ultra fast, as all that really happens is that a directory entry is created for that file. Yet when the file is accessed, it will always behave to any file operation as if the file really was 100 GB in size and all filled with zeros. If you jump to the middle of the file and write some data there, only blocks covered by that data will materialize and actually consume disk space, the rest of the file stays virtual.
Nice, didn't know about truncate, will be useful for my upcoming graphical Linux from scratch video, thanks!
@@nirlichtman just seen this one, can’t wait :)
First time knowing about sparse files, I love it!
Thanks for the great comment ⚘️
Very instructive thanks!
@@nirlichtman or use something like "fallocate -l 50M image" to do it in yet another way
I think this is the best educated Linux tutorial I've ever seen
Are you lying?
I have been studying how to do this for thirty years and this is the most coherent explanation/demonstration of how to make a bootable system that I have seen. This is a magnificent work. Thank you.
Even though I have no interest in doing something like this myself, I found the whole process rather interesting. 🙂
Thank you this video! I have been using Linux for years, but I am interviewing with a company that builds their own kernel for their product and I wanted to know something about the process. This helped me so much!
Really cool to see this, thank you for the simple and easy to follow tutorial. I look forward to diving deeper into how this works.
my dream of creating my very own linux distro one day, is a step closer. thanks!
i tried LFS a long time ago, but gave up half way through did something wrong didn't know what.
but yeah a full distro from scratch how cool is that. for educational purposes too.
I mean, lfs is more for developers. People who already know their way around on debugging the worst and shittiest situation possible
@@Mempler feels like this is for a certain kind of user too. Using docker and not volume mounting /boot-files, so that you have to docker cp later...
omitting the configuration of syslinux
using vi, instead of cat or echo to create files.
It's nonetheless great to have many people with different ways of achieving what they need.
@@LewisCowles agreed, doing lfs helped me to learn linux in depth. funnily enough, i never read the book.
I literally just googled a shit ton and reverse engineered arch linux lol
I see LFS as something that everyone serious about Linux should install once, and then never actually use it. (If you really _want_ to use it, you can, but it probably won't be very enjoyable.)
Oh wow, my genius brain clicked on this video thinking you were about to build a basic operating system written in Scratch. And now I want to see someone actually try to do that.
I mean, there is linux on scratch
yeah but that's just using the linux kernel making a disto, not a new OS itself@@commander3494
@@commander3494 Is that a word play on Linux from scratch?
@@ghosthunter0950 im not sure if ifs intentional or not
@@ghosthunter0950no but it's just running a compact RISC-V emulator to run a VERY RUDIMENTARY Linux implementation that does not include most of the basic system utilities, and even text editing is limited to "ed" only, not even "vi", and forget about "nano"
This video is gold, thanks man. Ever thought about making a video on your very custom distro using the knowledge you showed us in this video?
Funnily enough this is also exactly how Linus Torvalds created the first version of Linux back in the days. He just fired up docker and booted some Ubuntu image
I laughed so hard, just opened the video and scrolled to the comments and saw that
That's so cool :O
I did LFS like three times and i always wondered how could i simplify the process to the maximum
Very nice video! I would enjoy more like these!
Its hilarious how BusyBox is basically a legitimate add-on to docker in that both are containers but BusyBox works as a "from scratch" image for Docker, more so than the docker official scratch image itself
You did this in 12 minutes?! That's crazy! 🤯
That's linux, once you go black you don;t come back.
This is the absolute best tutorial on this topic. Please do a tutorial on compiling more packages for the custom operating system. Packages like GCC, sudo, nano and whatnot. And please also do videos on setting up ttys that work hand in hand with key configurations, I've set up my ttys but they don't work with key configurations. A lot of demands here but please also do a video on setting up the root user, sudo (root access) users and normal users, as well as, how to compile and set up wayland, xorg, sddm and possibly even KDE Plasma.
Hello! I might be doing something wrong, but I tried to boot this on an older laptop without EFI support.
Seemingly everything seems to be fine (booting it with GRUB, actually), but after I get a few messages from the kernel printed to the screen,
everything goes black, my pendrive (the boot device) blinks a few times and I get stuck there.
What could possibly be the issue?
May God bless you bro😂 I needed this tutorial. I tried this at first but since i knew nothing about where to place the kernel, or about how to create and use a bootloader, the best i could do was only to make a chroot folder (and i became really good at it lol i even forged my own "terminal" and got it to work in chroot) but now im gonna be practicing this new knowledge everytime i get a chance.
Thanks a Lot. For now all i can do is add you a new sub. 🎉
Thanks! :)
@@nirlichtman pleasure bro. One day you'll get as many views as you deserve
@@nirlichtman also, have you tried to make a syslinux boot disk with a drive size of say 8mb/8 counts? It doesn't work. Do you possibly know any reason why?
You mean you are trying to write syslinux on a physical disk of 8MB?
@@nirlichtman I mean a disk file (dd if=/dev/zero of=eight bs=1M count=8) kind of thing.
Another issue I have seen is that it loads the kernel and the initramfs but it kinda gets stuck there , even after I followed the along with your tutorial.
Great tutorial Nir! I have a few followup questions.
1) You used the default kernel config file that came with the repo, instead of replacing it with your Ubuntu machine's config file. And in the default config file, the only change you made was enabling 64 bit. Would that cause any issues if you were to use this kernel in production, since it wasn't "handcrafted" for your machine?
2) Why don't you have to run 'make modules' after you make the kernel? Does the default config file just set all modules to be compiled into the kernel?
3) If you set some modules in the config file to be compiled external from the kernel, where would we copy those .ko module binaries so the kernel could use them? Would they need to be copied into the initramfs cpio archive? Or would they need to be placed on a real disk in a root filesystem the kernel would switch to after it goes through initramfs?
4) Since you copied BusyBox directly into the initramfs archive, the kernel didn't need to mount any real root file system on a real disk. However, suppose we did want to use a real root fs on a real disk, and install BusyBox there instead. How would we need to change the init program and initramfs directory structure and content to make the kernel mount the real disk and change the root directory to the one on the disk?
Very cool and well communicated. Thanks!
this make me recall the time we were trying with LFS distro - just doing it without understand anything :)
Amazing and very useful for studying Linux!
what do you recommend i look into/do if i want to go further in making it useable?
Following this tutorial was easier than installing arch Linux
super clear and interesting! Subscribed.
I know a lot of people have asked the same question, but how would I write this to a USB drive to boot it on a real machine? I've tried to use tools like dd or genisoimage, but even after a few hours, I still can't get it to work. Usually, it just says there's no bootable device, other times, it says the device doesn't support UEFI booting.
BRAVO, BRAVO! That was great!
simply amazing...
Looks like someone else also watched Rob Landley's video on a minimalist Linux. Although this is even tighter than his, even if it doesn't exit properly.
good and precise to the point
Hello. If I understood correclty the ramfs is indeed for syslinux. The kernel will mount the cpio archive pointed out by syslinux to the kernel through the command linux. I remember It is also possible to embbed the cpio inside the bzImage itself (option in menuconfig).
How to create an iso from the img, maybe with grub to automatically load kernel +initramfs
The step I'm not following fully is at 9:25 when you mount boot into m. What happens when you mount a file to a directory then copy those files into that mounted directory?
The file I am mounting on the directory is a disk image I initialized as a FAT file system, after I copy those files into the mounted directory it adds those files to the FAT file system on the disk image so the Syslinux bootloader that I installed on the disk image can find the kernel and initramfs
Hi i keep getting an error when mounting boot as m (mount: m: failed to setup loop device for /boot-files/boot.)
Try it with sudo
me too, idk why and in docker im root
cool! combined with something like sinit and sbase and voila - extremely small desktop linux :)
This is super cool!!
Happy to see this!!😃
Subscribed right away
So busy box is a user space? What are alternatives, primarily interested into using apt, if applicable a basic GUI (I have my mind of what Icon set I want to use). Those options are and or conditions. Just want to know some user space alternatives and its names.
ofc I meant user space interface
You could also use gnu coreutils or uutils (rust rewrite of coreutils)
These are the minimal set of Unix tools to have a functional cli
You'll also need a proper init system like runit,initrc,systemd if you want to build a modern functional os
@@Person1873 Thank you very much. I just started to stick my head into this topic. This is very useful information, as before I wasn't able to find much about it without knowing what to look for.
How long did it take to compile??
mine is going for about 30mins
If I want to use the default init from the busybox instead of the simple bash you provide, what should I prepare?
LFS will guide you :)
Just compile busybox with it and "ln -s sbin/init init" so the kernel can use it
I have a custom built distro without boot configuration. So based on this tutorial, i did convert my whole root directory into init.cpio (init script also added). And did the same boot configuration(based on distro size, i have created the boot file with count=4000 (4GB)). But when i want to run the kernel via qemu, "init.cpio...failed, no such file or directory" error occures. Does anyone have a idea? The cpio file exist i had double check.
windows -> wsl -> ubuntu -> dwm -> distro from scratch
Hello NIr,
thank you very much for all your "in-depth" and short, very well explained videos! Great!
Do you like to do a "real" minimalistic linux system with a graphical interface with that approach? What do you think? Or is it too much to think of and better take arch linux and do a minimalistic linux with dwm, vim and so on from there?
Thanks, shalom and best regards
Daniel
Shalom Daniel, Thanks :) I haven't tried running Xorg on this distro so I am not sure if it would work, I do skip a few things that may be necessary to make the graphical interface work (for example running a proper init and setting up the tty devices), but this is a great idea for an additional video about actually setting up a minimalist graphical linux distro from scratch
@@nirlichtman any update on this?
Excellent
how would i go about implementing apt or opkg?
Is this minimalistic environment sufficient to bootstrap Linux From Scratch?
How did this come to be? I mean... I already knew what a bootloader, kernel, busybox and initramfs did, but I never knew (or even got close to thinking for that matter) that you could put them together just like this. Did you first come up with the idea to do the bare minimum to get a shell working as init process, then read how the initramfs worked in detail just to get the system running and finally looked for a minimalist bootloader? Just how did this pop in your mind?
Yes u made my day. 🎉😊
Anyone know why I'm getting this on the "mount boot m" step? Tried ubuntu and Debian containers.
Never mind. Turns out it was a Docker issue. Once I updated Docker, it works.
Thanks for guide.
I wonder , how extendable this system in your opinion ? I have been through Lfs book and I can't get rid of intrusive thoughts to build a (mostly) gnuless system from a linux installation you shown. I completely understand that it doesn't make any sense, pure enthusiasm and curiosity.
It is as extendable as any other distro except that you need to manually bring over the package binaries for every tool you use (or manage to get some package manager running). To make it actually usable for day to day you would probably also want to setup proper tty devices, init (and not just start the shell right away like i do in the vid - busybox also includes init built in but I didn't use it in this video) and a persistent file system to start after the initramfs
@@nirlichtman nah just write every program you need in vi like a real hacker
@@nirlichtmando you have some documentation or recommendations on the persistent storage thing?
@@itsoctotvYou could create a 1GB zeroes file, format it with EXT4 and mount it at boot. The first part is the same as the FAT one here, then you have to pass it to QEMU and also write an fstab that mounts it, etc.
@@nirlichtmantheoretically it maybe supports a subset of Debian packages since a minimal dpkg is present on it by default
Have you done one about buildroot? To make custom os for customs apps light weight
Why are you using Windows though?
thank you
Can you explain how you got DWM on what I assume to be windows?
Explanation in the welcome link on my channel description
@@nirlichtman OK thank you
- Artimis, bird of war.
how can i use grub instead of syslinux??
and how to make that thing where it says [ OK ] and something?????
You can install grub on the disk using the grub-installer package
About the second question, are you referring to systemd?
@@nirlichtman1. thanks 2. yeah
I have few things to request if you don't mind creating a video.
1. Can you create a distro using a base debian or ubuntu image?
2. Illustration of how to use systemd as a init system.
3. And use Grub2 as the bootloader.
good. thx
I just finished the build process, but I wanna convert the boot file to an ISO or a .img file. However, I'm afraid that when I change the file name and type, it'll erase everything, causing me to start over. Can I still change the file type within File Explorer?
Yes, there is no problem to do that, the boot file is a disk image so the .img file extension fits (it is just a matter of adding the .img extension to the end of the name).
@@nirlichtman Hey man. I have a little trouble. How can I fix it?
root@f7ad6d2c8317:/distro# mount boot m
mount: m: failed to setup loop device for /distro/boot.
@@rudykfox have you run the docker in privileged mode?
@@nirlichtman yes. I did it in privilege mode. But I little fixed this by doing it on main machine. But I have next some little troubles:
1. 'can't open /dev/tty2: No such file or directory' ,'can't open /dev/tty3: No such file or directory' ,'can't open /dev/tty4: No such file or directory' . How to fix it from starting system?
2. Data doesn't saves after reboot. How to make It save?
3. After putting some compiled program files in /bin/ we have kernel panic. How can I fix It?
Thanks in advance for your answers. I'm just new to systems programming, let alone the complete system building process
@@rudykfox check out my video about making a graphical linux distro, over there the file system is persistent, here it volatile. You can skip my steps over there related to the graphical stuff
Hey I have a project as a part of my OS Course called "Linux Kernel Development". Can i make this as my project. Or kernel development is far more beyond what you did in the video?
Depends on the project criterions, on this video I did not make any modifications in the kernel code but mostly worked with it as a whole component, I have a video about adding a simple system call to the kernel, that may be more relevant. Another option is my video about making a simple kernel module.
I wanted to customize the kernel for development boards (embedded systems)
and this video being what i realy expects by building the ninux kernel from scratch making this video as a series that show the other options for production standards
anyone else failing with "mount: m: failed to setup loop device for /boot-files/boot." on the first mount command?
Make sure you started the docker with the privileged flag
How can we use GNU instead of Busybox?
I had to do somth9ing very similar before to be able to boot systemd system when using a btrfs file system. I however did not use busybox as I did not need a full enviroment to work in. The system basically just needed to load up enough system to be able to read the file system and start systemd, then pivot root into the real FS. Funny enough what you did is pretty much what dracut and other programs are actually doing when they create the initramfs image.
What version of ubuntu are you using to make this build?
In this video I used the default latest docker tag of ubuntu
When doing "mount boot m" I get an error: "mount: m: mount failed: Operation not permitted." I've searched online but nothing seems to work, please help
Have you run the docker in privileged mode?
@@nirlichtmanwhen i try to do -privileged when running the container is says that -privileged doesn’t exist: "unknown flag: --privileged"
nvm i fixed it, i ran a new container and --privileged worked this time
Can you make a follow up video like configuring syslinux to auto boot the os and adding persistence or burning it to a iso/img
Yes, that is a good idea! BTW the image in the end that I boot with qemu is a disk image you can burn also to a physical disk using dd for example
technically with EFI stub compiled into the kernel, you can directly boot it from uefi, no bootloader needed.
"mount: m: failed to setup loop device for /boot-files/boot." ???
problem solved... did "apt upgrade" in addition to "apt update"...
Nice video!
I wanted also to implement the switching from the initramfs to a normal ext4 partition. It took me the last 6 hours because I had to use a different bootloader (systemd boot on aarch64) and because the fucking UEFI implementation for QEMU (asahi linux) did not had a SATA driver built in, I was not able to boot from any SATA disk, which means I have to use a virtio disk to boot and the Linux kernel did not want to detect a virtio disk but an SATA disk which means, I am currently having two disks with the same img file just to be able to boot from it and mount it in Linux. Arm is nice but the next time I am doing it on my Desktop with x86 xD
I too managed to do this but after a few days of reading and reverse engineering the archlinux iso
mine is the same thing as in the video up until the init file, which instead of calling /bin/sh, mounts the hdd, chroots into it and asks for login without the kernel dying if you press Ctrl+D
its very simple and not very operational so far but I think I can easily add systemd, git and gcc, and eventually make an actual distro out of that you can add your own packages
such amazing video. Can U publish similar but to create minimal system to boot on old raspberry 1/2 ? Regards!
Slackware installer is my ideal distro 😂
splendid
I'm getting an error with mount:
`mount: m: failed to setup loop device for /boot-files/boot.`
EDIT - FIXED: If you get the same error, just create a loop device and try mounting the file. If mount doesn't work, use losetup (search “how to mount file using losetup”). If losetup doesn't work *after* you've tried mount, delete the loop device by rebooting and try again, but just try losetup and not mount.
How to create a loop device:
`$ sudo mknod -m 660 /dev/loop0 b 7 0`
Interesting, do you have loop devices in the /dev directory (for example /dev/loop1)?
mount uses a loop device when mounting a file
Also, what distro and architecture are you using and do you get anything more with mount -v?
@@nirlichtman Oh, I don’t have any loop devices. I did `lsblk` and didn’t see any loop devices. I was using the ubuntu:latest image on Docker with x86_64 if that helps. I can’t run the command right now, unfortunately.
@@nirlichtman I fixed it! It was just as simple as creating a loop device myself.
@@SystemTomcat Nice :)
hey i wanna make my own ubuntu based distro with some custom tools amd softwars which i can then install on real computer too, with custom boot logo and all, can you guide in some main points like what to learn and from where to start, waiting for reply please must guide me
Nice.
is there any way to install a package manager onto this?
Yes, you can bring for example the package manager that comes with Alpine that is called apk-tools, I may make a video about making a distro around that package manager.
Hey man nice job on the tutorial! Only thing im stuck on is mounting the boot file. It says that it keeps on saying that vfat is unknown. Im not sure what I did wrong.
Perhaps you forgot to install the dosfstools ?
@@nirlichtman weird thing is that I have it installed. I'll reinstall it again and check it
@@nirlichtman yeah it still gives me that error for some reason
Also, do you think you could please make a tutorial video on how to compile the custom distro into an ISO file?
No need to create an ISO to get this running on a real computer, since all you have to do is write the boot image to a physical disk using something like dd for example (I should have added .img to the boot image name in the video to avoid confusion)
is there a way to run the boot file on vbox?
Shouldn't be a problem, you can also boot this on a real computer, it is a bootable disk image
I love your video, can you make one on how to make http request for windows os?
Yes, I plan on covering some winsock related stuff as well :)
why was this recommended to me at 12:53 AM and why am I watching this
Great job, thanks!
what a mad lad
Instead of running the shell with the shell, you could just use a symlink I think?
Yes, that is a good point :)
How?
@@abrahimzaman360 ln -s
Good point if you're not going to extend script further. Which I doubt if you would like to play with your distro and extend it.
How would you go about installing a tty?
On my next video about making a GUI Linux Distro I will also setup the tty properly.
hmm... simple panic fix would be while [ 1 ]; do /bin/sh; done? xD
At any rate fun stuff, I used to do LFS for fun when I was a teen, makes me miss those days.
how to install gcc there?
I am gonna try once i get some time to spare. Really interesting. Can we make a distro with AI integration ?
Great question
I can’t remember the name but in the pre Linux kernel 1.00 days there used to be a single boot floppy that would ask a bunch of questions, download and compile from sources. In those days I’d boot it and answer everything in the morning then let it down load, configure, compile, and install till the next morning. Lol. Ah the good ole days
Since you've built the distro using Ubuntu, doesn't that mean the built distro is Ubuntu-based? Cause I know that some commands implement some of the components used in the currently running distro, but I wonder if your build has any components from your Ubuntu container.
Nope, it is not based on Ubuntu, the user mode only contains busybox
@@nirlichtman So it's the Linux kernel with no distro sources right? Cause I have an idea for a distro, but I don't want it to be distro-based. I just want the kernel to be bootable with no sources from other distros, so I can use the kernel as a foundation for my future project. The reason why I just want the kernel to be bootable, is so I can add my own files and tools to it, all through the CLI. It sounds strange that I just want the kernel to be bootable without it being distro-based.
@@MasterH2005 Yah it's official kernel source with no additional distro stuff (and in user mode its just busybox) in this video I grab the latest kernel source from the Git repo master branch, but it may be a better idea to use a more stable release which you can find on kernel.org
Hello, i like this video, Is it possible to install an interface like GNOME or Xfce? If so, can you show how to do it? I would also like to know how to take this to production.
Thanks! Yes, I am working on research for a future video about making a GUI distro, stay tuned :)
@@nirlichtmanhey i wanna make my own ubuntu based distro with some custom tools amd softwars which i can then install on real computer too, with custom boot logo and all, can you guide in some main points like what to learn and from where to start, waiting for reply please must guide me
Why not GNU Coreutils?
6:15
Ahh, it confuses me, much like the chicken and egg situation 😅
Are you talking about the fact that I used the shell to start the shell? I am just running the shell and the first command the shell is going to run is the shell :)
Could you theoretically install a desktop envirement and make an iso out of it. thx, btw. good video
sweet
Can I do it on a real (Debian) machine instead of in a docker?
Or do it on a VM?
Yes :)