I've made a discovery not so long ago, that the SDL library used for game development has actually built-in support for static linking. Their official releases section only has the dynamic versions precompiled and every tutorial I watched uses them, even though the static version is so much easier. You only have to turn on and off two flags during the configure phase and everything else is done automatically by the build system.
The two biggest ones for me are... - Binaries are built for 20 year old cpus and don't account for modern cpu instructions. - Binaries have too much feature bloat or is missing features I want.
The software where modern instructions actually matter and make a huge difference like blender,handbrake,ffmpeg etc.. already has multiple compute paths and chooses the right instructions at runtime so is that really that important? I even asked multiple people who have used gentoo for 15 or more years whether compiling everything in my system for my cpu makes a reasonable difference compared to prebuilt binary and they said in 95% of cases it doesn't make a difference and where it does you can just compile only that software specifically with optimizations.
@@Vitis-n2vright? I just don’t get the reasoning of using gentoo. I’m not saying it’s bad, it doesn’t provide enough benefits to justify the amount of time you need to invest in it. I guess that’s why it’s only used by hardcore enthusiasts who do it just for the sake of doing it
Been a stable Debian user for life. I only get my software from two ways. Repositories or compile from source. Been doing this way for life. Plus compiling from source is a great skill to have anyway. Many reason why and you pointed them all out.
A reason not expected and not talked about by (I think), anybody, is if you are a programmer, and want to help develop the software, whether that's forking it, further creating it, or patching it, or customizing it in a complete way.
Another reason to compile is related to latest and greatest and missing features. That being experimental flag features. An example is when JXL (new image format) was first introduced to ffmpeg and you had to compile with experimental flags to use it. I even saw a post about how to possibly hack ffmpeg to use JXL as the video image instead of a regular format thanks to extra flags to compile it with which seemed pretty neat though it's not something I feel like trying.
This is tied in with the "latest and greatest" section, but if there's a bug in a software in your distro's repo its always a good idea to compile the software from the most recent source and ensure that it has not been addressed already, alongside the other things you would do before submitting an issue like checking to see if an issue report already exists.
In the good old days I used Gentoo. There were two good reasons for this. Gentoo is like LFS, but without the pain of having to maintain all the dependencies on your own. Also, back then many distributions created their binary packages for i386 while we already had an i586 under our desktops. And while videos from the dark depths of the Internet ran smoothly under Windows, an i386 Linux had to surrender even on an i586. Today I don't have the time or the desire to deal with Gentoo. The one thing I really miss is the blessing of use flags.
With the AUR a package that will be compiled will have the "-git" at the end of the package name while the "-bin" package is a binary. I can't remember about packages with no suffix such as "paru" the AUR helper.
You see I have tried to get into the idea of installing gentoo and compiling everything from source but every single time without fail while installing gentoo I straight up get bored and wipe it. Literally every time I get to around the kernel hacking stuff I just go “fuck this I can’t be bothered” As a side note the gentoo manual is one of the best guides I have ever used for anything ever..
you can use sys-kernel/gentoo-kernel or sys-kernel/gentoo-kernel-bin to skip kernel hacking & get safe defaults. i did that when i first started before eventually coming back to compile my own kernel, till now i still have gentoo-kernel-bin as a fallback in case my own own custom kernel effs up
You nailed it! First time I see an youtuber actually understand why we use Gentoo: It's not about performance optimizations, but control and extreme personalization. I can build my own packages instead of using it the way the distro package maintainer decided it's the best way to do it.
My first real long time daily driver distro was Gentoo, back on a single core Athlon 64. Contrary to popular belief I did not in fact spend the majority of the time compiling code. You very quickly learn how to set USE flags in order to avoid pulling in unnecessary cruft and to make informed choices about what software you use from the perspective of how difficult it is to build. Back then it was also way less common to spam out new versions on a conveyor belt, so my former blackbox + rxvt + firefox life didn't really involve that many updates at all.
This is my experience also. After a bit I got a sense of which packages would take a while to compile and prepare to update when I have time. I do household chores or feed the four hungry cats while compiling. Or I run the update overnight it's a big one. I don't sit at my terminal for hours eyeballing the compilation. If I need a new app ASAP or just want to try something new, I install a flatpak or Distrobox export. My Gentoo installs a very stable and easy to maintain. I am a simple minded little lady with no IT background and any distro more complex than Gentoo confuses me. Also no matter what keep qtwebengine from ever being installed.
I’m new to Linux (just left windows this past weekend). And I went through about 5 different distros trying to find the one the seem like would be easy to start with. But after using fedora and arch I noticed both of those use a really bad version of OBS so I ended up going with Debian because that version felt like the windows version. I didn’t know about installing from source. I never knew what that source was for. But it’s nice to know about this for later down the road because I would like to use Arch but I wanna learn about how to set it up the way I want it to look.
One corollary for me with regards to adding features is that sometimes I have to write the feature myself. There have been a few emulators I've used throughout the years that were open source and lacked support for various controllers I use and I've had to add code to support them, along with in certain cases adding code to give me more control options in games. Back when I used pcsx_rearmed for all of my PS1 emulation needs I added support for the analog sticks so I could play Doom on the PS1 using Halo-style controls.
I compile from source for the 4th reason. Sunshine is available as a Flatpak, but doesn’t work as well. So I compile from source if I am on a distro that doesn’t have a direct binary.
Compiling from source doesn’t mean not using package manager. You mentioned Gentoo and Arch’s AUR (via helpers), and those are both examples of using package managers that simply build from source before installing the package. Most distros have `src` repos that you can use to do the same. Gentoo, via emerge, is the most powerful of them all, especially because of those USE flags you mentioned! ;)
Saying as a new Linux user, compiling is good BUT the problem is I don't know how to compile. And either I'm not good at researching that subject or there is no good tutorial on compiling in complicated ways. Such as changing the installation location, as you mentioned. I sometimes magically be able to compile and run things and sometimes they work sometimes they don't.
I have compiled from source, but the final boss compiling from source is to compile Linux itself from scratch lol. I don't think I will do that anytime soon, but definitely one day. I always fancied making my own opinionated distro. That said, if If I can't be lazy and use the package manager or AUR, well, gimme that tarball and I will do it myself!
I thought the 6th reason was for security; to actually personally check/inspect the source code. You have to watch out for exploits or backdoors that could be injected. We can only hope the project author is good natured and that the community reports very fast if something funny is going on. Thanks for explaining about Gentoo and the other reasons people sometimes compile themselves.
i looked through the repository for the application i want.. its not in the repository.... snaps and flatpack is too fat... i wont use them... compile is easy...
You have nothing better to do with your PC? You want to be an elitist dork? You want an excuse to rewatch the entire director's cut of LotR while your desktop is built? So many possibilities.
In my particular case, I have to test a Python package on multiple versions of cpython, and Debian only provides 3.11. I have also compiled some other packages, such as nano. I'm not a big fan, but I don't mind having to do that either. I don't know how I would feel if I had to compile every single package on my machine.
I'm an IT guy, but not a programmer, vaguely remember compiling linux kernels back in the 90's for our helpdesk box but I really just followed someone else's instructions, didn't fully understand what those 'make' & 'configure' cmds did. Fast fwd to 2010 I migrated to linux as my daily driver. But even with my now 14 years daily (and semi-advanced) experience I still tend to stay away from compiling. I only tried to compile a couple of minor things and had trouble each time. One piece of software required installing dependencies first, some of which conflicted with my existing system and was tricky to navigate. I think I broke my APT system? I vaguely remember giving up and learning what statically linked binaries were (from some PPA), which got me over my immediate need. Then I had to figure out repairing my APT system. Maybe I had a particularly unlucky experience or maybe the instructions given were just poor, but it really put me off until I learn more about what 'make' and 'configure' and all those other commands actually do. And...how to deal with different library versions if needed. But truth is I figure I probably won't learn them until I learn to code in C. Which probably won't happen.
I think you forgot one more reason: Nerd cred I'm one of those people who has never compiled from source! I wouldn't even know how, and most github repos assume you actually do know what you're doing with that. If I can't install it from my distro's repository, or something like flathub, I"ll just move on. Maybe someday I should give it a shot.
For the most part, if it's C source code; 'make && make install' at the terminal is like... 85% of what you need to know. Otherwise, you read the documentation, you follow along. It's really not that scary.
NixOS has lots of capabilities for building from source. Most things are just stored in the binary cache, but if you come across a package that isn’t or you need to make some tweaks to its installation, you’ll end up installing from source but from the safety of having nix do it for you :)
Another benefit of Gentoo being source-based is that is gives better way of managing rolling release distro. You can freeze any package you want, you can mix between stable and unstable "branches" with minimal risk of breaking the system. If updated program or library has some annoying bug you can just roll it back because ebuild for previous version is almost always still there and emerge will rebuild everything that gets broken from this rollback. On binary rolling distros you usually either roll back whole update if you have snappy configured or suffer.
it's also interesting that you can use weird things like clang instead of gcc and musl instead of glibc. Although one time I tried to install gentoo with both of these I could get base system working but I couldn't successfully build a proper desktop
I don't have the time or patience to compile from source. I prefer ready to use apps from the repository. Sometimes building it yourself can be a headache that isn't worth it. Sometimes the build instructions make no sense, or they didn't take something into account, which creates errors in the terminal. The only time I compile is when I have no other choice.
this is why i love portage; it makes compiling really simple because you just use ebuilds combined with useflags, and you can automatically compile multiple packages, basically eliminating all the actual effort necessary to compile from source. if you time the emerge command, time also isn't an issue. (for example, i ran `emerge -avuND @world; shutdown -h now` as root before leaving my house and updated around 200 packages effortlessly and without losing any time since i wasn't home anyway.)
Also downloading with the official package manager of the distro has the advantage that it's mostly tested, and it will very rarely break. Ie i use fedora, and never had problems with installing software with dnf, as it never once failed and the packages always work well enough. I remember when i tried nix os and arch, that many times when i installed programs by compiling them, it would break in the middle of the installation (and not so rarely) That's why i just stuck with fedora dnf: stability and up to date packages (which must be crazy hard to mantain, and for that i am grateful to mantainers)
This was my experience. 30 years I been around IT, but I'm not a coder (at least not for compiled languages) and felt a bit 'lost' trying to follow instructions which spewed errors about missing libraries, or had me install them which subsequently left me with conflicts / dependency issues. I'm game for staying up late trying to make something work, but for me compiling will remain a last resort. Maybe I had a particularly bad experience. I think it would be different if I understood more about libraries & how to manage conflicting versions. I don't want broken APT system again.
For points 2 and 5, there's also the reasons of licensing in that some distros don't want to support certain "non-free" licenses like things like media codecs or are prohibited by law from shipping binaries that are built with certain libraries. Example of this was when media players couldn't be distributed with the code to play movie DVDs. As for Gentoo, there's also the concept of overlays where packages can be updated quicker than the main Gentoo repo and "live" ebuilds that pull directly from Git (or SVN, etc) repos to get the source code. Also, most ebuilds can be copied to your own local ebuild repo and renamed to latest package version to get latest version of a package, sometimes you might need to update the ebuild to add/remove a patch or enable/disable a new build feature if that's what's going on.
Compiling is interesting, exciting, and fun. Been doing it since 1958 or so. I only have gentoo as a vm. Couldn't use it to do real stuff as you can take 50% of your time working on the system. Never saw anyone say they saw measurable differences by installing a custom kernel.
Why? Because you can't compile from binary. Optimization for your hardware? The reason why big software companies never will port their stuff for linux. Keep "enjoying" gimp and other crap.
Every week I release a Patron only podcast. I just posted this week's episode. patreon.com/thelinuxcast Available to all tiers.
I've made a discovery not so long ago, that the SDL library used for game development has actually built-in support for static linking. Their official releases section only has the dynamic versions precompiled and every tutorial I watched uses them, even though the static version is so much easier. You only have to turn on and off two flags during the configure phase and everything else is done automatically by the build system.
And GCC flags. When I build a package for my raspberry pi, I optimize for size (-Os) but when I compile for my server I used hardened flags
The two biggest ones for me are...
- Binaries are built for 20 year old cpus and don't account for modern cpu instructions.
- Binaries have too much feature bloat or is missing features I want.
or the other way around - binaries dont work on older hardware
The software where modern instructions actually matter and make a huge difference like blender,handbrake,ffmpeg etc.. already has multiple compute paths and chooses the right instructions at runtime so is that really that important? I even asked multiple people who have used gentoo for 15 or more years whether compiling everything in my system for my cpu makes a reasonable difference compared to prebuilt binary and they said in 95% of cases it doesn't make a difference and where it does you can just compile only that software specifically with optimizations.
@@Vitis-n2vright? I just don’t get the reasoning of using gentoo. I’m not saying it’s bad, it doesn’t provide enough benefits to justify the amount of time you need to invest in it. I guess that’s why it’s only used by hardcore enthusiasts who do it just for the sake of doing it
Cause it looks cool
This
Look mom, I'm a hacker!
Been a stable Debian user for life. I only get my software from two ways. Repositories or compile from source. Been doing this way for life. Plus compiling from source is a great skill to have anyway. Many reason why and you pointed them all out.
A reason not expected and not talked about by (I think), anybody, is if you are a programmer, and want to help develop the software, whether that's forking it, further creating it, or patching it, or customizing it in a complete way.
Another reason to compile is related to latest and greatest and missing features. That being experimental flag features. An example is when JXL (new image format) was first introduced to ffmpeg and you had to compile with experimental flags to use it. I even saw a post about how to possibly hack ffmpeg to use JXL as the video image instead of a regular format thanks to extra flags to compile it with which seemed pretty neat though it's not something I feel like trying.
This is tied in with the "latest and greatest" section, but if there's a bug in a software in your distro's repo its always a good idea to compile the software from the most recent source and ensure that it has not been addressed already, alongside the other things you would do before submitting an issue like checking to see if an issue report already exists.
In the good old days I used Gentoo. There were two good reasons for this. Gentoo is like LFS, but without the pain of having to maintain all the dependencies on your own. Also, back then many distributions created their binary packages for i386 while we already had an i586 under our desktops. And while videos from the dark depths of the Internet ran smoothly under Windows, an i386 Linux had to surrender even on an i586. Today I don't have the time or the desire to deal with Gentoo. The one thing I really miss is the blessing of use flags.
With the AUR a package that will be compiled will have the "-git" at the end of the package name while the "-bin" package is a binary. I can't remember about packages with no suffix such as "paru" the AUR helper.
The ones without -git are the same as with -git but just building the latest release.
You see I have tried to get into the idea of installing gentoo and compiling everything from source but every single time without fail while installing gentoo I straight up get bored and wipe it. Literally every time I get to around the kernel hacking stuff I just go “fuck this I can’t be bothered”
As a side note the gentoo manual is one of the best guides I have ever used for anything ever..
you can use sys-kernel/gentoo-kernel or sys-kernel/gentoo-kernel-bin to skip kernel hacking & get safe defaults. i did that when i first started before eventually coming back to compile my own kernel, till now i still have gentoo-kernel-bin as a fallback in case my own own custom kernel effs up
You nailed it! First time I see an youtuber actually understand why we use Gentoo: It's not about performance optimizations, but control and extreme personalization.
I can build my own packages instead of using it the way the distro package maintainer decided it's the best way to do it.
I think the big thing is the flags to optimize the software to suit your needs.
Or for building with a hardened toolchain and things like PaX type protections.
My first real long time daily driver distro was Gentoo, back on a single core Athlon 64. Contrary to popular belief I did not in fact spend the majority of the time compiling code. You very quickly learn how to set USE flags in order to avoid pulling in unnecessary cruft and to make informed choices about what software you use from the perspective of how difficult it is to build. Back then it was also way less common to spam out new versions on a conveyor belt, so my former blackbox + rxvt + firefox life didn't really involve that many updates at all.
This is my experience also. After a bit I got a sense of which packages would take a while to compile and prepare to update when I have time. I do household chores or feed the four hungry cats while compiling. Or I run the update overnight it's a big one. I don't sit at my terminal for hours eyeballing the compilation. If I need a new app ASAP or just want to try something new, I install a flatpak or Distrobox export. My Gentoo installs a very stable and easy to maintain. I am a simple minded little lady with no IT background and any distro more complex than Gentoo confuses me. Also no matter what keep qtwebengine from ever being installed.
I’m new to Linux (just left windows this past weekend). And I went through about 5 different distros trying to find the one the seem like would be easy to start with. But after using fedora and arch I noticed both of those use a really bad version of OBS so I ended up going with Debian because that version felt like the windows version. I didn’t know about installing from source. I never knew what that source was for. But it’s nice to know about this for later down the road because I would like to use Arch but I wanna learn about how to set it up the way I want it to look.
One corollary for me with regards to adding features is that sometimes I have to write the feature myself. There have been a few emulators I've used throughout the years that were open source and lacked support for various controllers I use and I've had to add code to support them, along with in certain cases adding code to give me more control options in games. Back when I used pcsx_rearmed for all of my PS1 emulation needs I added support for the analog sticks so I could play Doom on the PS1 using Halo-style controls.
I compile from source for the 4th reason. Sunshine is available as a Flatpak, but doesn’t work as well. So I compile from source if I am on a distro that doesn’t have a direct binary.
compiling from source really helps with some applications. Ardour was a mess for me until I compiled from source.
Compiling from source doesn’t mean not using package manager. You mentioned Gentoo and Arch’s AUR (via helpers), and those are both examples of using package managers that simply build from source before installing the package. Most distros have `src` repos that you can use to do the same. Gentoo, via emerge, is the most powerful of them all, especially because of those USE flags you mentioned! ;)
Saying as a new Linux user, compiling is good BUT the problem is I don't know how to compile. And either I'm not good at researching that subject or there is no good tutorial on compiling in complicated ways. Such as changing the installation location, as you mentioned. I sometimes magically be able to compile and run things and sometimes they work sometimes they don't.
My reason is the opposite of "latest and greatest", compiling things that were already deprecated from the repos.
That's filed under "no other choice".
Small programs i will,
But most oh hell no, to quote a meme " ain't nobody got time for that"
I have compiled from source, but the final boss compiling from source is to compile Linux itself from scratch lol. I don't think I will do that anytime soon, but definitely one day. I always fancied making my own opinionated distro.
That said, if If I can't be lazy and use the package manager or AUR, well, gimme that tarball and I will do it myself!
I thought the 6th reason was for security; to actually personally check/inspect the source code. You have to watch out for exploits or backdoors that could be injected. We can only hope the project author is good natured and that the community reports very fast if something funny is going on.
Thanks for explaining about Gentoo and the other reasons people sometimes compile themselves.
i looked through the repository for the application i want.. its not in the repository.... snaps and flatpack is too fat... i wont use them... compile is easy...
You have nothing better to do with your PC? You want to be an elitist dork? You want an excuse to rewatch the entire director's cut of LotR while your desktop is built? So many possibilities.
In my particular case, I have to test a Python package on multiple versions of cpython, and Debian only provides 3.11. I have also compiled some other packages, such as nano. I'm not a big fan, but I don't mind having to do that either. I don't know how I would feel if I had to compile every single package on my machine.
add and remove features from the program in the source... though freebsd ports does a MUCH better job.
I'm an IT guy, but not a programmer, vaguely remember compiling linux kernels back in the 90's for our helpdesk box but I really just followed someone else's instructions, didn't fully understand what those 'make' & 'configure' cmds did. Fast fwd to 2010 I migrated to linux as my daily driver. But even with my now 14 years daily (and semi-advanced) experience I still tend to stay away from compiling. I only tried to compile a couple of minor things and had trouble each time. One piece of software required installing dependencies first, some of which conflicted with my existing system and was tricky to navigate. I think I broke my APT system? I vaguely remember giving up and learning what statically linked binaries were (from some PPA), which got me over my immediate need. Then I had to figure out repairing my APT system. Maybe I had a particularly unlucky experience or maybe the instructions given were just poor, but it really put me off until I learn more about what 'make' and 'configure' and all those other commands actually do. And...how to deal with different library versions if needed. But truth is I figure I probably won't learn them until I learn to code in C. Which probably won't happen.
I compile from source cause I don't want to go anywhere near flatpak or snap
I think you forgot one more reason: Nerd cred
I'm one of those people who has never compiled from source! I wouldn't even know how, and most github repos assume you actually do know what you're doing with that. If I can't install it from my distro's repository, or something like flathub, I"ll just move on. Maybe someday I should give it a shot.
For the most part, if it's C source code; 'make && make install' at the terminal is like... 85% of what you need to know.
Otherwise, you read the documentation, you follow along. It's really not that scary.
NixOS has lots of capabilities for building from source. Most things are just stored in the binary cache, but if you come across a package that isn’t or you need to make some tweaks to its installation, you’ll end up installing from source but from the safety of having nix do it for you :)
OpenSUSE Leap 15.6 was released, yei!
Another benefit of Gentoo being source-based is that is gives better way of managing rolling release distro. You can freeze any package you want, you can mix between stable and unstable "branches" with minimal risk of breaking the system. If updated program or library has some annoying bug you can just roll it back because ebuild for previous version is almost always still there and emerge will rebuild everything that gets broken from this rollback. On binary rolling distros you usually either roll back whole update if you have snappy configured or suffer.
Why not bother to compile from source, the compiler or libraries might be compromised anyway.
it's also interesting that you can use weird things like clang instead of gcc and musl instead of glibc. Although one time I tried to install gentoo with both of these I could get base system working but I couldn't successfully build a proper desktop
I don't have the time or patience to compile from source. I prefer ready to use apps from the repository. Sometimes building it yourself can be a headache that isn't worth it. Sometimes the build instructions make no sense, or they didn't take something into account, which creates errors in the terminal. The only time I compile is when I have no other choice.
this is why i love portage; it makes compiling really simple because you just use ebuilds combined with useflags, and you can automatically compile multiple packages, basically eliminating all the actual effort necessary to compile from source. if you time the emerge command, time also isn't an issue. (for example, i ran `emerge -avuND @world; shutdown -h now` as root before leaving my house and updated around 200 packages effortlessly and without losing any time since i wasn't home anyway.)
Also downloading with the official package manager of the distro has the advantage that it's mostly tested, and it will very rarely break.
Ie i use fedora, and never had problems with installing software with dnf, as it never once failed and the packages always work well enough.
I remember when i tried nix os and arch, that many times when i installed programs by compiling them, it would break in the middle of the installation (and not so rarely)
That's why i just stuck with fedora dnf: stability and up to date packages (which must be crazy hard to mantain, and for that i am grateful to mantainers)
This was my experience. 30 years I been around IT, but I'm not a coder (at least not for compiled languages) and felt a bit 'lost' trying to follow instructions which spewed errors about missing libraries, or had me install them which subsequently left me with conflicts / dependency issues. I'm game for staying up late trying to make something work, but for me compiling will remain a last resort. Maybe I had a particularly bad experience. I think it would be different if I understood more about libraries & how to manage conflicting versions. I don't want broken APT system again.
No thanks. I value my time and low energy bills. Plus an auto-optimised kernel can be worse than a generic one.
For points 2 and 5, there's also the reasons of licensing in that some distros don't want to support certain "non-free" licenses like things like media codecs or are prohibited by law from shipping binaries that are built with certain libraries. Example of this was when media players couldn't be distributed with the code to play movie DVDs.
As for Gentoo, there's also the concept of overlays where packages can be updated quicker than the main Gentoo repo and "live" ebuilds that pull directly from Git (or SVN, etc) repos to get the source code. Also, most ebuilds can be copied to your own local ebuild repo and renamed to latest package version to get latest version of a package, sometimes you might need to update the ebuild to add/remove a patch or enable/disable a new build feature if that's what's going on.
To have 1% more performance
XBPS-SRC
Love void but have dependancy issue with source packages mix
Compiling qt apps from source 💀
Can you review bedrock linux?
Compiling is interesting, exciting, and fun. Been doing it since 1958 or so. I only have gentoo as a vm. Couldn't use it to do real stuff as you can take 50% of your time working on the system. Never saw anyone say they saw measurable differences by installing a custom kernel.
I've been using Gentoo for over 20 years, if you want to update once a month, that's maybe a day per month tops... 🤔
1958?!
Why? Because you can't compile from binary.
Optimization for your hardware? The reason why big software companies never will port their stuff for linux. Keep "enjoying" gimp and other crap.
I compile from source because debian is so damn out of date - its impossible to use in certain situations
This doesnt apply to my arch systems though lol, this is primarily for my home lab and servers that run on debian
"debian is so damn out of date - its impossible to use in certain situations"
What are some examples?
Being out of date is the whole point of Debian though?
To be fair, you could have used Debian Testing or Unstable if you wanted the latest versions of software without the hassle of Gentoo, though...
With parallel computing, compilation could go brrrrr
Distcc has been around for years.
Cause elitist nerds.^^
In my cases, i compile pi kernels, realtak wifi drivers,aircrack-ng at live-cd environment.
Everything about linux seems to be related to combatting it's inadequacies