I sure do love it when a random yt channel with under 1k subs and a celeste speedrun posts an in-depth technical video explaining a complicated subject intuitively!
Flattered that you spent your time on my vid :D h-m has been a game changer for me. It’s like NixOS with having to give up your normal OS. The weirdest corners for me so far have been to not be able to quickly edit my config files and having to do a special dance for nvm, rustup and similar tools.
@@dassurma if you don't use configure those directly with nix, but simply use home-manager to place them in the appropriate location, you can use home-manager to symlink them instead, allowing for rapid iteration. You can then turn that on or off with a custom option.
I loved the content, the tempo and the story: the end result is a consequence of logic steps, not something you take for granted. I just found that the music was a bit too loud.
Most videos start where you ended. I felt like there had to be something simpler underlying all of this, and there is! Thank you so much for this. You actually explained Nix. By going through the whole math, only to then, afterwards!, show the shortcut.
This is such a great video! I've never really understood the reasoning behind Nix until now, let alone how it works. Saving this one to my "most important videos" playlist
I like how you presented this. Building up from the basics without getting too bogged down in the details. I do have a few notes, though: 8:49 This is actually (unfortunately) not the case. Antiquotes of path types do expand to a nix store path, but builtins.toString actually returns the non-store path as a string. I consider this broken behavior, and have argued that it should be changed, but it is the current behavior. As a rule, just never use toString on a path-type. If you think you need to, then you're probably mixing eval time and run time in a way that will come back to bite you at some point. Also, it isn't exactly when a path-type is evaluated that it gets copied to the nix store (in particular, if you import a path type, it needn't be copied, and isn't), but I'm pretty much fine with glossing over that. 13:07 Yes, you did a bad thing, but perhaps more importantly, on linux, this build would have completely failed because there, builds are run inside a container without access to the host filesystem at all. Might have been good to mention. 21:12 You did a subtle bad thing here, by overriding phases without calling the pre and post hooks. This keeps the hooks from firing, and can lead to hard-to-debug behaviors as a result. This is one of the biggest causes of frustrating confusion I see for people writing their own derivations. Some construct that depends on hooks is supposed to work, but doesn't, and they can't figure out why. It's cause they overrode a phase and didn't use runHook to run the pre and post phase hooks.
Oh cheers! Really appreciate the thorough review. I’ll add those to the video description. The remark about the hooks is interesting. That was something I definitely misunderstood and I suspect did cause a lot of headaches lol. Wrt containerization: I actually had that on my TODO list to check if you can have your build run in a container, and I never got around to it. Can derivations opt out of the containerization? Or how does bootstrapping work on Linux?
@@dassurma Containerization is a nix option, controlled in nix.conf or on the command line with --sandbox or --no-sandbox. It's off by default on darwin, I think because the sandboxing tech available on darwin causes too many problems, but on linux it's on by default. The sandbox only has the closures of the store paths mentioned in the .drv available in it. Nothing else. As far as I know, there's no way for a derivation to skip containerization; if your build can't work in a container, then it really isn't a proper nix build in the first place, so it isn't really viewed as sensible to give that option, especially considering the security implications of building other people's (not necessarily trusted) derivations. Bootstrapping stdenv in nixpkgs (in linux or darwain, afaik) is done without reference to the host system, because it instead starts with a precompiled set of tools, downloaded much like source code is. Those precompiled tools are very rarely updated, and extra care is taken to ensure provenance when they are. I believe several separate groups build them independently and check that they get bit-for-bit identical results, among other things. They don't really need to be updated until you reach a point where the latest version of the compiler can no longer be built with them, anyway.
This is really well done and useful, been trying to wrap my head around nix for a few months or a year at this point. Still very far down the rabbit hole to explore but this was crazy useful!!!
Kudos on an incredibly well made video. This is absolutely the foundational knowledge I needed that fits between "Why to use" and "How to use" which are well represented elsewhere.
Honestly, I love the explanation on the topic, and I would like to request more on the topic. Mind blown by how well it is presented. I myself trying out Nix and want to get started. The problem was finding where to start. Thanks a lot for the video!
One can start wars over whether "with;" is encouraged or not. In my opinion, sometimes it's just the most elegant way to express yourself without repetition.
Thank you for this very informative, nicely balanced video. The documentation situation for Nix is indeed not ideal, so I appreciate your work of collecting various distributed pieces of pertinent information and combining them into a "digestible" format. 👍
Thanks! That is an amazing learning experience! Pleasant motioncanvas graphics, well pased explanation, relaxing music and colors. I'm happy that UA-cam got me here. Subscribed and patiently waiting for more =]
Cool video! exactly what I needed :)) Can you please also do a follow-up on replacing docker? The way I currently see it it's more of a one-time temporary thing (e.g. nix build, nix develop). I'd be very interested to see how a production deployment would look like
I would love to have more videos on this series (this is now a series, right?). Personally I’d love to know more about home manager, but I’m sure there is a lot more to talk about on the nix ecosystem
Thanks for this video. You have touched on the very essential things that is confusing me so far. One more thing, I think I am missing something here, when building clang-s2, you've used in the derivation of glibc the clang-s1 derivation, which depends on system libraries. Doesnt that mean that eventuall clang-s2 also depend on system libraries?
I'm curious about how it replaces docker on your servers? I understand that you would run some bit of nix software that is sort of pinned to a specific version - like a docker image. But docker is more than that. Does nix also provide filesystem isolation and a networking layer?
Personally I liked both Nix and containers. The isolation of files systems and network of docker is a pure joy and the reproducibility and sandbox of building of Nix is perfect. That’s why I used nix to build docker Image that I can just run with Docker. Nix is not a Docker killer. Nix is a Dockerfile killer
Nix doesn't provide the security benefits of docker, no. But honestly, that's not the main reason people use docker most of the time. Also, nixos and similar systems can certainly set up containers with those security benefits, if you want, and the nature of the nix store tends to make it rather easy to create such containers.
It's possible to use a nix derivation to build a docker container (so there are no problem is someone used :latest and the docker file fails). I don't remember where the docs for this are
I'm assuming your a sysadmin who uses docker like I do. There's a thing called nix she'll or something that makes a temp env for a package. That package could be your software. When you exit, the shell and envelope are gone I think and it's basically ephemeral like a container. Also if you think about it, that docker networking layer is only there because it has to be. Even if you are good at it like I am it's still an extra thing to remember and EVERYONE gets but by it the first time they try to send something to local host right? We'll if it's just on your system then you suddenly get to use local host again. That's nice right?
I am an Arch user and I am not really looking to distro hop anymore, however I am very tempted by NixOS. It has a very mature community and I think the way it works is interesting. I would have to set off some time to get used to the userland of it, but man I am tempted just to do it. Should I? Before you say this, yes I have looked into all my usage cases and NixOS can handle all of them just fine. I would just have to get used to all the various things NixOS is good at to optimize my system and you know... as Linux users of this area, that is the _fun_ of a distro, isn't it? Agh, I am very tempted! 😄
Also I am not looking to ever dual boot any distro or any operating system, so I wouldn't want to keep my Arch install around if I decided to do this. All in or not.
Honestly, my recommendation is to go for arch with nix rather than go full NixOS. NixOS is attractive from a purist perspective, but I have found it impractical for a workstation (I do like it conceptually for servers tho, but haven't taken that leap yet)
@@dassurmawhat makes it impractical for you? wanted to switch back to a different distro since i can't use nix at work, but to be honest i can't ever go back to something else. just being able to for example have a stable zfs release but have the newest kde release is just to good. also love the fact that my system is inherently documented, though i haven't used home manager yet
It's a great video and an angle, I haven't seen before. Thanks for that. The only thing I didn't understand is the following: If clang has a dependency onto libc, then the hash of clang is dependent on the hash of libc. And if libc is build with clang then its hash is dependent onto clangs hash. Isn't that bad? Or why is it not?
You talked about replacing docker. How did you do that and how do you manage your containers? I did not fully understand that jump to connecting nix with docker replacement
Yooo, that Dutch pronunciation of the research paper's title was _tight_! That was so clean! I see you live in the UK, but did you live over here for a bit or what?
at 5:49 the error message in not related to the code: /* f = */ {a, b, c}: a + b + c f {a = 1; b = 2;} # error: function called # without requirement argument 'b' It's not 'b' but 'c' (i hope :p)
1:15 software is usually distributed as binaries since a binary has a smaller file size, less dependencies, and you can directly run it rather than compiling it (which in the case of something like Chromium would take several weeks of 100% CPU usage if you were to do it on an old office PC from 2010s, if it would even finish and not run out of RAM, which it probably would) 1:20 Firefox is Open Source, under the Mozilla Public License which is a weak copyleft license. 2:15 no, it doesn't, the source code gets compiled on a compiler farm of each Linux distro, uploaded to its repositories, and the user downloads the resulting binaries. Unless you run Gentoo.
I love how you animated the code throughout the video. Is there a tool that performs these transitions automatically, or did you create them manually when editing the video?
I think we were sorely lacking effective presentation and communication techniques for highly technical information and concepts, and I'm continually impressed by how everyone utilises Motion Canvas.
Nix is truly the best thing that happened to the linux ecosystem. Been using it for 1.5 years now and loving it. But it has a pretty steep learning curve I must admit, harder than Rust or CUDA IMO.
I sure do love it when a random yt channel with under 1k subs and a celeste speedrun posts an in-depth technical video explaining a complicated subject intuitively!
I think this is surma from google! Hes one of the hosts of the http 404 podcast. Jake archibald is another. Looking forward to his videos
@@vikingthedude It's HTTP 203 but yeah
@@vikingthedude also they both work at Shopify nowadays
He's not a random UA-camr! Trust me
@@vikingthedudeqq
Always love a "I made the video I wish I'd had." Thanks for putting this together, it's very well done.
This is very helpful, Nix needs more material that helps us understand the "Why", more stuff like this basically
Learned a ton from this video Surma. Thanks for summarising all of this!
Have been digging into nix/h-m over the last few weeks, awesome to have the algorithm send you to me pal!
Flattered that you spent your time on my vid :D
h-m has been a game changer for me. It’s like NixOS with having to give up your normal OS. The weirdest corners for me so far have been to not be able to quickly edit my config files and having to do a special dance for nvm, rustup and similar tools.
@@dassurma if you don't use configure those directly with nix, but simply use home-manager to place them in the appropriate location, you can use home-manager to symlink them instead, allowing for rapid iteration. You can then turn that on or off with a custom option.
@@TheSastif the files in the appropriate location are manager by h-m, wouldn't you need to rebuild with every change?
Antichamber, Celeste, and now Nix. This guy knows ball.
One of the best videos explaining how Nix works under the hood. Wish there was such a video when I first started using it.
A good balance between handwaving nitpick details and drilling down into how it all comes together!
I loved the content, the tempo and the story: the end result is a consequence of logic steps, not something you take for granted. I just found that the music was a bit too loud.
Thanks for the kind words. You are not the first person to point that out. I’ll keep that in mind for next time :D
If you could upload without the music I would use it as a default for explaining nix
Excellent video. I'll be sharing this one when people ask me to explain Nix.
Most videos start where you ended. I felt like there had to be something simpler underlying all of this, and there is! Thank you so much for this. You actually explained Nix. By going through the whole math, only to then, afterwards!, show the shortcut.
Thank you for the kind words. Means a lot.
This is such a great video! I've never really understood the reasoning behind Nix until now, let alone how it works. Saving this one to my "most important videos" playlist
Excellent as always! Good to hear your voice again.
I’ve been running nixos for a couple of years and contribute to nixpkgs.
I like how you presented this. Building up from the basics without getting too bogged down in the details. I do have a few notes, though:
8:49 This is actually (unfortunately) not the case. Antiquotes of path types do expand to a nix store path, but builtins.toString actually returns the non-store path as a string. I consider this broken behavior, and have argued that it should be changed, but it is the current behavior. As a rule, just never use toString on a path-type. If you think you need to, then you're probably mixing eval time and run time in a way that will come back to bite you at some point. Also, it isn't exactly when a path-type is evaluated that it gets copied to the nix store (in particular, if you import a path type, it needn't be copied, and isn't), but I'm pretty much fine with glossing over that.
13:07 Yes, you did a bad thing, but perhaps more importantly, on linux, this build would have completely failed because there, builds are run inside a container without access to the host filesystem at all. Might have been good to mention.
21:12 You did a subtle bad thing here, by overriding phases without calling the pre and post hooks. This keeps the hooks from firing, and can lead to hard-to-debug behaviors as a result. This is one of the biggest causes of frustrating confusion I see for people writing their own derivations. Some construct that depends on hooks is supposed to work, but doesn't, and they can't figure out why. It's cause they overrode a phase and didn't use runHook to run the pre and post phase hooks.
Oh cheers! Really appreciate the thorough review. I’ll add those to the video description. The remark about the hooks is interesting. That was something I definitely misunderstood and I suspect did cause a lot of headaches lol.
Wrt containerization: I actually had that on my TODO list to check if you can have your build run in a container, and I never got around to it. Can derivations opt out of the containerization? Or how does bootstrapping work on Linux?
@@dassurma Containerization is a nix option, controlled in nix.conf or on the command line with --sandbox or --no-sandbox. It's off by default on darwin, I think because the sandboxing tech available on darwin causes too many problems, but on linux it's on by default. The sandbox only has the closures of the store paths mentioned in the .drv available in it. Nothing else. As far as I know, there's no way for a derivation to skip containerization; if your build can't work in a container, then it really isn't a proper nix build in the first place, so it isn't really viewed as sensible to give that option, especially considering the security implications of building other people's (not necessarily trusted) derivations.
Bootstrapping stdenv in nixpkgs (in linux or darwain, afaik) is done without reference to the host system, because it instead starts with a precompiled set of tools, downloaded much like source code is. Those precompiled tools are very rarely updated, and extra care is taken to ensure provenance when they are. I believe several separate groups build them independently and check that they get bit-for-bit identical results, among other things. They don't really need to be updated until you reach a point where the latest version of the compiler can no longer be built with them, anyway.
Please, make more videos about Nix! Amazing and astonishing content!
This is really well done and useful, been trying to wrap my head around nix for a few months or a year at this point. Still very far down the rabbit hole to explore but this was crazy useful!!!
Honestly, the best nix video I have seen so far! Thanks, Surma!
What a great and helpful video for anyone trying to get a first, or a second deeper grasp on what Nix is and how it works. Thank you!
This is superb! I think this is the best tutorial I've seen yet.
Kudos on an incredibly well made video. This is absolutely the foundational knowledge I needed that fits between "Why to use" and "How to use" which are well represented elsewhere.
BANGER. thank you sm for this cool video. i’ll rewatch this once i try nix
Make more. With content of this quality you deserve more subs!
Honestly, I love the explanation on the topic, and I would like to request more on the topic. Mind blown by how well it is presented. I myself trying out Nix and want to get started. The problem was finding where to start. Thanks a lot for the video!
This is the documentation I was looking for! Thank you so much!
Great video, was very helpful! Interested in nix for a long time and this helps a lot to understand nix and all of its meanings better.
this is just a great explanation of nix that i ever heard. please make more videos about nix. Thanks.
The video we needed, not the video we deserved! Amazing Job!
I just gave my first talk that explained similar concepts… but dang! Your explanation is amazing! This is just great!
Extremely well done! You probably don't use this, but if you ever do a video on NixOS, I'm here for it!
I needed this a year ago! But I’m happy that it’s here now. :)
One can start wars over whether "with;" is encouraged or not. In my opinion, sometimes it's just the most elegant way to express yourself without repetition.
The editing and explaining of this video is insane
Great video. Just enough content that I wanted to watch it all and enough teasers to make me go research some more.
Thank you for this very informative, nicely balanced video. The documentation situation for Nix is indeed not ideal, so I appreciate your work of collecting various distributed pieces of pertinent information and combining them into a "digestible" format. 👍
Thank you soo much! This was the Tutorial new users need, and I felt was kind of missing by the official documentation.
Best video i saw this year, just awesome, THANK you!
I've been hearing a lot about nix recently but haven't bothered to check it out. Maybe I will take a look now. Thanks for the video
Thank you Surma. excellent video
Thanks! That is an amazing learning experience! Pleasant motioncanvas graphics, well pased explanation, relaxing music and colors. I'm happy that UA-cam got me here. Subscribed and patiently waiting for more =]
That is such an incredible video. Thatns for your hard work
Awesome video !!! Wish i had it when i just started learning, but you still managed to fill in some knowledge holes for me.
Thank you for making this super clear explanation!
This is great! Do flakes or home-manager next!!!
This is a really good video with a lot of care, thanks!
thank you so much for this video! such a great explanation, you're awesome!
Great video. I loved your OTMT podcast.
best description about nix - great video
Loved It. Keep the videos coming Surma.
Cool video! exactly what I needed :))
Can you please also do a follow-up on replacing docker?
The way I currently see it it's more of a one-time temporary thing (e.g. nix build, nix develop). I'd be very interested to see how a production deployment would look like
This was very well done good sir.
Cool, loved the visualizations!
Great video! look forward to the next one.
Great video, thank you!
God what a great explanation
Elegant explanation. Much tanks for making this video. Could you also make a video explaining the machinations of NixOS too?
Do you plan on making more Nix content? Thank you very much for such a polished and informative video.
Great video! Can you elaborate on how it compares (pros and cons) with docker?
I would love to have more videos on this series (this is now a series, right?). Personally I’d love to know more about home manager, but I’m sure there is a lot more to talk about on the nix ecosystem
Someone please ask for this to be added to the wiki and awesome-nix repo. Awesome video!
Dude sick video!
Mathematically a function takes 1 input and produces 1 output. Multiple arguments can be passed (and returned) as a tuple or by currying.
Great content as always, Surma! I’m also genuinely curious to how you find it work as a replacement of docker on your servers.
22:56 Now that we have the Nix Trinity, can we expect a full systematic theology?
Thanks for this video. You have touched on the very essential things that is confusing me so far.
One more thing, I think I am missing something here, when building clang-s2, you've used in the derivation of glibc the clang-s1 derivation, which depends on system libraries. Doesnt that mean that eventuall clang-s2 also depend on system libraries?
THANK YOU!!
Love this
Hey! Awesome video !
beautiful video 🙏 keep it up bro🫶
This video is so well made, I'm glad I watched it!
great pacing 👍
More videos please. ❤
Thank you!
Top tier vid
I'm curious about how it replaces docker on your servers? I understand that you would run some bit of nix software that is sort of pinned to a specific version - like a docker image. But docker is more than that. Does nix also provide filesystem isolation and a networking layer?
Thanks for the video, BTW. I appreciate you.
Personally I liked both Nix and containers. The isolation of files systems and network of docker is a pure joy and the reproducibility and sandbox of building of Nix is perfect.
That’s why I used nix to build docker Image that I can just run with Docker.
Nix is not a Docker killer. Nix is a Dockerfile killer
Nix doesn't provide the security benefits of docker, no. But honestly, that's not the main reason people use docker most of the time. Also, nixos and similar systems can certainly set up containers with those security benefits, if you want, and the nature of the nix store tends to make it rather easy to create such containers.
It's possible to use a nix derivation to build a docker container (so there are no problem is someone used :latest and the docker file fails). I don't remember where the docs for this are
I'm assuming your a sysadmin who uses docker like I do. There's a thing called nix she'll or something that makes a temp env for a package. That package could be your software. When you exit, the shell and envelope are gone I think and it's basically ephemeral like a container. Also if you think about it, that docker networking layer is only there because it has to be. Even if you are good at it like I am it's still an extra thing to remember and EVERYONE gets but by it the first time they try to send something to local host right? We'll if it's just on your system then you suddenly get to use local host again. That's nice right?
I am an Arch user and I am not really looking to distro hop anymore, however I am very tempted by NixOS. It has a very mature community and I think the way it works is interesting. I would have to set off some time to get used to the userland of it, but man I am tempted just to do it. Should I? Before you say this, yes I have looked into all my usage cases and NixOS can handle all of them just fine. I would just have to get used to all the various things NixOS is good at to optimize my system and you know... as Linux users of this area, that is the _fun_ of a distro, isn't it? Agh, I am very tempted! 😄
Also I am not looking to ever dual boot any distro or any operating system, so I wouldn't want to keep my Arch install around if I decided to do this. All in or not.
Honestly, my recommendation is to go for arch with nix rather than go full NixOS. NixOS is attractive from a purist perspective, but I have found it impractical for a workstation (I do like it conceptually for servers tho, but haven't taken that leap yet)
@@dassurmawhat makes it impractical for you? wanted to switch back to a different distro since i can't use nix at work, but to be honest i can't ever go back to something else. just being able to for example have a stable zfs release but have the newest kde release is just to good. also love the fact that my system is inherently documented, though i haven't used home manager yet
It's a great video and an angle, I haven't seen before. Thanks for that.
The only thing I didn't understand is the following: If clang has a dependency onto libc, then the hash of clang is dependent on the hash of libc. And if libc is build with clang then its hash is dependent onto clangs hash. Isn't that bad? Or why is it not?
Great video!
You talked about replacing docker. How did you do that and how do you manage your containers? I did not fully understand that jump to connecting nix with docker replacement
Yooo, that Dutch pronunciation of the research paper's title was _tight_! That was so clean! I see you live in the UK, but did you live over here for a bit or what?
at 5:49 the error message in not related to the code:
/* f = */
{a, b, c}: a + b + c
f {a = 1; b = 2;}
# error: function called
# without requirement argument 'b'
It's not 'b' but 'c' (i hope :p)
great ! Thanks !
1:15 software is usually distributed as binaries since a binary has a smaller file size, less dependencies, and you can directly run it rather than compiling it (which in the case of something like Chromium would take several weeks of 100% CPU usage if you were to do it on an old office PC from 2010s, if it would even finish and not run out of RAM, which it probably would)
1:20 Firefox is Open Source, under the Mozilla Public License which is a weak copyleft license.
2:15 no, it doesn't, the source code gets compiled on a compiler farm of each Linux distro, uploaded to its repositories, and the user downloads the resulting binaries. Unless you run Gentoo.
its a great video. Had a doubt on how you create these videos? what is the software used?
Brilliant!!!!
Great video, how do you create the animations and the voice overs? Will you share the source code for this video?
If this is actually able to compile the Qt frontend, it would be incredible, as Qt is very difficult to compile replicably, even with Docker.
I really liked the video but can you tell me what did you used to create such a good quality video.
That’s the wonderful Motion Canvas :)
better title: explaining nixos in forza motorsport terms
Very good video, the music is very disturbing though. Any change you can re-upload without such loud music?
That's a bold claim indeed. Can it install Crisis on my Tamagochi?
I love how you animated the code throughout the video. Is there a tool that performs these transitions automatically, or did you create them manually when editing the video?
Motion Canvas
@@cinderwolf32 Thanks!
I think we were sorely lacking effective presentation and communication techniques for highly technical information and concepts, and I'm continually impressed by how everyone utilises Motion Canvas.
It’s simple: I see Surma, I subscribe.
Amazing, quick question regarding "placed docker on my servers". Do you do cgroups by hand? how do you handle sandboxing?
I love you Surma
I am watching this video just for motion canvas
Let me guess, nix making new users is the equivalent of me compiling and testing a program on a VM for OS that I don't have.
Nix is truly the best thing that happened to the linux ecosystem. Been using it for 1.5 years now and loving it. But it has a pretty steep learning curve I must admit, harder than Rust or CUDA IMO.
Last time I installed Nix on Mac there was no need for an extra partition for Nix.
Is it mandatory that I dye my hair blue before installing nixos?
No, you can also go the programmer socks and skater skirt route.
9/10
-1 for placing the nix language syntax in the beginning part.
Felt overwhelming and weird!