Installing Arch Linux w/4x NVME RAID! LVM & LUKS!
Вставка
- Опубліковано 13 вер 2024
- Arch Wiki Links:
+ Encryption: wiki.archlinux...
+ LVM RAID: wiki.archlinux...
+ forum.level1te...
**********************************
Thanks for watching our videos! If you want more, check us out online at the following places:
+ Website: level1techs.com/
+ Forums: forum.level1tec...
+ Store: store.level1tec...
+ Patreon: / level1
+ L1 Twitter: / level1techs
+ L1 Facebook: / level1techs
+ Wendell Twitter: / tekwendell
+ Ryan Twitter: / pgpryan
+ Krista Twitter: / kreestuh
+ Business Inquiries/Brand Integrations: Queries@level1techs.com
IMPORTANT Any email lacking “level1techs.com” should be ignored and immediately reported to Queries@level1techs.com.
-----------------------------------------------------------------------------------------------------------
Intro and Outro Music By: Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 3.0 License
creativecommons... - Наука та технологія
"What could you possibly offer, that the archwiki can't offer"
me_irl
redundancy ;P
@@tinygriffy haha, like your name.
nico nico wha? No redundancy? I would consider having your ass saved by good documentation redundant :P
Finally some Arch talk, good job.
I'd like a zfs vs btrfs comparison from you
Seconded. Also, a ZFS setup video with root on ZFS and encryption would be cool (I know I can look all that stuff up, but still...)
Easy enough.
Btrfs
Benefits
- built natively into the Linux kernel
Negatives
- will randomly wipe your data
ZFS
Benefits
- won't randomly wipe your data
Negatives
- if you don't want to use Ubuntu, you'll need to install a dkms module, and be careful with kernel updates (or build your own custom kernel)
@@Treversaccount ... Or use kernel from Proxmox :D. I solved that way my problems with OpenMediaVault in the past :P
@@Treversaccount maybe, but zfs is very slow, specially with encryption
@@luigitech3169 not really, I'm running nvme drives, and it's able to get the full 3GB/s
They did have an issue with encryption speeds a while ago, because they weren't able to use hw acceleration due to a change in the kernel. But that's all fixed now, performance is good, especially on 0.8.4.
Now he can say " I use arch btw"😆😆
I use Arch btw ^^ Love it so much i will never change. I´ve configured my first raid 1 today with software raid mdadm.
You can have your EFI partition also be your boot partition. It works perfectly fine, really. (if you use systemd-boot or syslinux you actually have to do it that way)
The audio at the start of the video for the AsRock Z490 Motherboard sounds rough, you may want to check that. Starts at 0:00 and lasts for about 10 seconds.
@Level1Linux few things: *) swap is only striped if priorities of devices are the same (and older kernels only striped the first two devices even if more were available the rest were being used as JBOD) which can be checked with swapon -s. *) If you're using RAID and not AID (RAID0 == AID) you also want to have redundancy for swap. Learned this the hard way years ago, when a failed disk meant that data was still available but processes were acting funny because the kernel could not swap in memory pages from a device that went missing. Which is basically the same as having a portion of RAM going bad. *) Personally I have three partitions per device (EFI, /boot). The third one is LUKS encrypted and is then used as a PV in LVM.
Saw the title came to comment. YOU'RE A MADMAN!
You legitimately almost killed me with your trout live in trees comment...holy cow that was funny and caught me off guard.
Wouldn't it be less of a headache to have /boot and efi partition on a separate fifth device and use those 4 nvme drives as / ?
Was thinking the same.
sure, if you want? I alluded to this on a USB stick as well, which would be my pref for a luks setup anyway.
I think it actually makes more sense to have your root + boot on a separate physical drive and then just use your raid setup for your home drive. This way you can easily reinstall your OS with less risk of stomping on the data you care about.
At least for a desktop PC. For servers you care about your configuration files.
Very nice overview! Just what I was looking for to conceptualize all the text that's been floating round from rtfm'ing
Swap is not only for RAM extend. It is for hibernation. Users like to shutdown their computers and restore the previous session state need it as well. However, in this case an ordinary mechanical hard disk should carry the swap partition to safe the lifetime of the SSD.
Very much enjoyed this, a topic I've been putting off for a while now. Thanks for the video and looking forward to more like it!
lvm uses the md code for RAID. So it's irrelevant if you use md or lvm for the RAID. It's easier to manage directly in LVM though because there's no additional device layer in comparison to using md directly.
Technically true, but in my experience totally untrue from a tools/administration standpoint. Historically, LVM tools for managing bad situation were quite lacking. RedHat has really put a lot of work into making that not so anymore, but I wouldn't exactly describe the differences as irrelevant. mdadm still has some special sauce for raid10 functionality which is not exposed by lvmtools/lvextend/lvconvert etc
Agh, I've been postponing this. Time to encrypt my swap.
Very cool, I don't think I will ever do this but it's very cool. I use Arch btw.
Finally a video worth watching. Now this is some great tech content
Sidenote: grub can be used from a coreboot BIOS payload, so a separate boot partition is not needed in order for you to decrypt.
Wait, I thought that LVM and mdadm had basically both been smushed into the linux device mapper subsystem, and that essentially LVM raid was dm raid?
Hahaha that moment you had starting at 6:00... This is some kind of side effect the Arch Wiki sometimes has on humans
The only reason you need more than a few gigabytes of swap is for hibernation. Though encrypted swap and hibernation is likely to be trouble.
In the past encryption, swap and hibernation really did not work reliable enough but nowadays this should work without issues.
Of course, if your intent is (as per the video) to improve speed - then encrypting swap is a rather counter productive idea. Though you could argue that speed and swap itself is already not on the same roadway - you want to make use of swap as little as possible, else you know your speed has already deteriorated.
If your intent was security, then sure - encrypt the entire drive or all the partitions / volumes, including any swap space. Just take note that you're giving up some performance for the encryption - which in turn sort of negates the discussion about RAID (at least in this context).
@@benriful I just bought the smallest xpoint drive, and use that as swapspace. And try and limit swapping but it will happen a bit anyway, and so it is encrypted.
Swap is useless on modern machines.
Personally I stopped using LVM and md a while ago. They're is good if you need raid 5 or 6 but my personal setup at this point is luks encrypted physical partitions with a linear btrfs volume on top. I could do raid 0 but the performance wouldn't be worth the resilver time when I add disks. Maybe when I max out the PCI-E lanes in my system after adding all the NVME my CPU can handle I'll resilver it lol.
I pretty much do this in a similar way but using the manjaro installer,
/boot/efi as FAT32
/boot as ext4
/ as luks
/home as luks
by having /boot unencrypted helps a lot with boot up time and since the keys are stored in /etc/luks-keys/ managing additional drives is fairly easy to do with gnome-disks.
I was interested in booting from UEFI to my zfs pool. Maybe you could do a video on that? bypassing grub altogether
The EFI/Boot partition part in this video was a little confusing. The EFI partition *is* the boot partition on my system. Eg: I have a partition (100MB) that is set to 'EFI System' as the 'type', but it is mounted at /boot (making it my boot partition). I have a directory in my "/boot partition" called EFI (/boot/EFI). So the way I would personally describe this is that I have an EFI partition, mounted at /boot. I wouldn't say I have an EFI partition AND a boot partition, because it's just one partition.
this can work, but I would suggest it is not a best-practice.
5:58 - Swap can be a file on a mounted filesystem which is then mounted as a loop block device. Doesn't perform the best, but if you're desperate or just need some extra space for hibernate, it works okay. ,
Can you make one about BTRFS? I know ZFS is THE filesystem, but still.
Thanks Wendell - I am digesting similar issue for my next build. Currently on fully encryted LVM group. Proablly need to homelab some ZFS and learn it
Ok, I admit it, I only watched this today because the video preview thumbnail was spot on the "math thinking meme".
I have LVM1 (I think) on LUKS with a detached header that lives inside my initramdisk on my FAT32 /boot, which is also my EFI and lies on external storage.
I need to enter my pw only once and sd-encrypt sd-lvm2 hooks handle everything.
It's non-standard but elegant imho, no partition/key shenanigans described here.
Though I plan to move to a SSD raid so it may be a bit of a headache.
If Arch Wiki is not good enough, good news, you can contribute to it to make it better.
If your tight on ram and your drive is slow. I would highly recommend setting up compressed memory swap!. Its swap thats in ram, that is compressed on the fly. Helps prevent the write/read penalty by trading some CPU cycles.
wiki.archlinux.org/index.php/Zswap
ooh that would have been handy on my ultrabook. It had an i5 but only 4 gigs of ram. Would have been nice. Im on the thinkpad meme now.
I particularly use a Zram device as swap, to avoid 100% using the drive. I found the compression ratio of zram (lz4) to be typically 2.5/1, with a fast enough CPU you won't even feel it paging (just so you know, I'm using an FX-8350 on DDR3 1866) with a zram device the same size as RAM (16GB). Zswap I found to be slower on extreme cases, when it starts writing some pages to disk. Zram keeps up way better in my machine, plus I don't degrade my SSD with swapping.
LVM on LUKS ! Just 1 big LUKS container and everything LVM inside.
I use Arch btw.
I personally just make a 128M-1G /boot partition, that is fat32 formatted, with the EFI directory in and call it a day.
I think it would be interesting to hear you guys talk about immutable operating systems e.g. fedora silverblue
LVM also gives you snapshot capability - I think that is potentially a key factor to consider when designing your system.
Wait, do I really need to usa SWAP as a partition?
I am using swap as a file, kinda like Windows pagefile.sys.
Swapfile 8GB
I'm also using a 250GB SSD with a 4GB recovery partition, 100GB of root and 135GB as a bcache cache partition registered along side a 1TB HDD for /home.
so I got a superfast OS with 1TB of Hard Drive that loads my 135GB of most used files super fast as well.
Swap files are the newer version, but some people still like to have a separate partition, so that they have a fixed amount of swap that doesn't interfere with their "usable system space" if you will.
I'm so glad I'm not the only one who finds the Arch wiki borderline unusable, I had to resort to the Gentoo wiki to understand how to setup mbr and boot partitions when I installed Arch
Swap needs to be redundant otherwise when a drive goes away part of your swapped out memory disappears leading to crash of whatever has been swapped out on it
Good stuff, would've liked to see some benchmarks
It would be great to see some benchmarks here. What method is the most efficient? Do you know the cloudflare dm crypt performance patches?
Finally Linux video! Maybe you could make some video about fedora silverblue and development in containers?
Wow that's great 👍
Very interesting video, but the person who asked for LVM and LUKS on Arch didn't RTFM. I don't want to be rude, but he should have looked for more documentation, not just on LVM and LUKS, but on Linux partitioning. It's cool that we have Wendell explain it though and if Wendell or anyone else was ok with explaining it to him, well, I don't mind, good for him. I would probably lead him to some documentation and tutorials and maybe gave him some tips on how I do things. The way I encrypt drives on laptops is with unencrypted boot and efi partitions and with LVM on LUKS (asking for the encryption key at boot), in which I put the whole root partition and a swap partition.
On servers, I don't usually deal with LVM on raid arrays, I usually just make a md array and use LUKS on md directly. But that is because when I make a storage, I fill it with drives (I never went above 12 drives and most of the time I just go with multiple 4 disks arrays on smaller servers). If I were to go with fewer disks today and add more disks when I need to, I would definitely go with LVM on md, because from what I have read, the LVM tools are using md behind anyway and some features like lvs or lvreduce don't work when you use the LVM raid tool. Unless your software wasn't purposely built to deal with disks, raid, volumes and file systems from the get go (ZFS), I trust the Unix way of doing one thing and do it well. LVM works well, md works well, they complement each other well. For the love of KISS, don't try to use LVM tools for RAID, just use LVM on md (in this case, LUKS on LVM on md).
These days, I just put my EFI partition and encrypted /boot on a usb key with a ring lanyard on it so I can yank it out and take it with me. The /boot has the keys to decrypt root partition with a passcode, so 2FA booting.
Why not more than 4-8GB for swap? What about hibernation?
Hibernation? You mean on a laptop? I'm guessing this vid is about RAID for performance purposes, with that expansion card for 4oiff M2 drives it definitely isn't for a laptop. In which case I can't see why anyone would hibernate some desktop / server machine, rather just shut it down or run it indefinitely on a UPS.
If you mean something like a low power consuming NAS ... uhmm ... really? Hibernate and then WOL? That's going to feel extra unresponsive! I'd rather try to suspend parts of it instead of turning the whole thing off.
What happened to the "gaming on Linux" videos?...
Thank you
I'm sad, I was expecting some pornographic performance numbers to justify bothering with all this xD
Dude, it's a wiki, if it's wrong, you can edit it
I like the concept but would worry about the maintenance down the road. It seems to be more practical to establish a normal working installation of Linux on a typical form of media and then add a self-contained version of this concept that could be started as a VM or in another way.
The downtime due to a malfunction or configuration change would bother me in that it would happen in six months or a year after the system was setup and the ability to resolve the problem quickly would be problematic.
Wtf I literally JUST googled this
Arch revs my, OH MY cpu !!!
Please do a video comparing various strategies for system backup and restore. Ideally, how to maintain configurations for the entire system and applications in a file, so that it may be kept in version control.
Similar to a `.vimrc` or `.emacs` file for the entire system, if possible.
What archwiki theme is that?
Wouldnt be a tpm chip a much better solution than a usb stick, which can and will fail. You can buy them for nearly every mainboard as a plugin-board
Don't question the sacred scrolls!
I haven't made swap since I had 16 gig ram like 7 years ago, is swap still necessary?
Depends on your computer and your workload. I have 32GB and I can max that out pretty easy, but as average user with 16GB and cleans up their browser tabs probably can't. Either way it is good to have a swapfile at least in case you ever need it. It doesn't do any harm sitting there
For hibernation
It's important to have a little bit of swap, as otherwise if you run out of ram your system will start to act weird. if you have a bit of swap, it will be much more tolerant to having no free ram.
Depends. If you never hibernate and you never load up programs / data close to the total 16GB of your RAM, then you don't "really" need any swap.
If you ever run something which just "touches" that 16GB limit - you crash immediately. The swap would at least allow you to close something, even thogh the computer would run slowly. No swap would be an immediate crash without any means to recover - reboot and loose all unsaved data.
i remember i first upgraded to 16 gig ram and than ssd, and my os was on hard drive. then every time I was using swap I could feel it, the whole thing went so slowly. but these are good suggestions thanks. i make sure to partition some space for it next time.
I havent seen it yet (lack of time) but one question: does he mention how to do snapshots of encrypted volumes efficiently?
No.
LVM can do RAID5 functionality by itself
why is your audio sometimes active the noise cancellation and not?
I use Arch, btw
so what your saying I can quit unraid by using LVM and simply add more disks without too much worry.
Are these real megabytes/gigabytes or powers of ten? It's so confusing, they should have picked new names and contractions.
Could you use optane memory for swap?? Maybe as a RAM extension but not quite...
8:10 And there is me who is 3.5GB into swap because laptop has only 1 RAM slot and there is no way to upgrade RAM.
Note to future self: get a computer with enough RAM to not ever need swap again.
Question: why not use zswap? If I understood correctly, zswap is dynamically allocated swap manager. It creates 64MB swap files as need for swap increase. I believe zswap would be merciful solution to SSD devices.
I would really like to here from you about zswap.
Nice guide.
If you boot off a USB and lose the USB, are you SOL?
Does he have a video how it is done or not?
It's time for 4K editing.
MADNESS!! :)
While you're giving tutorials, I'd love to see you guys set up encrypted zfs on nvme on Gentoo on Threadripper... Because reasons
Minus the threadripper, that's how I have my laptop set up, although it's only a single nvme drive. At least with dracut, the one major difference between ZFS and anything else encrypted is that you have to be careful how you lukeOpen your encrypted volume during your install (or really, whenever you build a new initramfs). The zfs dracut modules appear to assume that whatever your encrypted volume was named when it was created is what it's going to be called during boot time.
I couldn't get it to boot on my encrypted volume until I loaded it with rd.debug which is where I saw it was looking for /dev/mapper/gentoo_lvm and then going into basically an infinite loop. When I did my initial luksOpen, I named the volume gentoo_lvm, so for whatever reason, that's what it decided it was looking for. I booted back into my live environment (ubuntu 19.10) and then did a luksOpen with the volume name as luks-uuid### and then recreated my dracut initramfs.. After I rebooted, it magically started working.
My laptop hasn't been set up that way for very long, but it's been working pretty well so far.
@@vacant2012 that's pretty dope. Is there any reason you went LUKS and not ZFS native encryption?
Any actual test figures you can show? E.g. a HW-RAID vs Motherboard RAId vs LVM RAID vs single NVMe disk? Your post just lists "Fast. We’re going to go fast." And then shows a kernel compile test, while that's all good and dandy - there's nothing to compare this to. I.e. all I now know is that your setup compiled the kernel in so many seconds using various thread counts. It doesn't say how much the RAID improved it over a normal (single or JBOD) or other sort of RAID setup.
Things like boot time, as well as starting programs and loading / saving data files would be ideal as a practical test, at least for most people. But even just a disk benchmarking could be informative. And a kernel compile is equally useful, no real need to show different threading options for the compiler (stay with say 64 threads for all tests) - the point would be to test read/write performance of the drive(s), not how your CPUs caching performance deters gains from extra threads.
Meanwhile Fedora33 does this out of the box with a more hardened kernel and security protocol.
I have a 16 phase ASRock motherboard from 2009 that can power a 300w CPU without vrm fans, why does that one have so many?
NVME LUKS YUP
I would like to know why the USB 3.0 ports do not work for me in ArchLinux
Try sudo modprobe usb_xchi
Intel platform? Enough PCIe lanes?
if trap live in trees it will not rain? What?
What NVMe/PCIe card is he using?
The Liqid HHHL flash drive
I would like to see a video on building a PC (the CPU to use, memory, storage, etc...) for software development (system or user land) or machine learning ... both requiring visualizations. (I know that you have already done this here => ua-cam.com/video/JvuDrrFHrhQ/v-deo.html but I guess that I wanting to see something that would be everyday developers too (?)).
It helps to set swappiness to 1
I’d like to see some distrotalk without SystemD-fect
Can someone please enlighten me as to how to enable dark mode on the arch wiki, or in case of some custom theme, where I can get it?
Less talk, more commands.
ZFS would be the better choice since....... Oh, wait.
Installing Arch.. yes, ok, cool... but why?
I use arch btw without any swap
hi
Dont do that - use ZFS !
Don't use ZFS on Linux
Don't use ZFS on Linux
Don't use ZFS on Linux
Don't use ZFS on Linux
Don't use ZFS on Linux
Do not use ZFS on Linux
use Oracle, use Solaris, use FreeBSD
don't use ZFS on Linux