Easy as 1-2-3: Organize your storage with LVM
Вставка
- Опубліковано 9 тра 2024
- There are some easy ways to manage your storage and there is the hard way. For years I mamaged my file systems with physical disk devices and partitions. But there is a better way, let's look at one of them called: Logical Volume Management or LVM.
AI Thumbnail: My Office (just kidding, mine is worse)
Support me on Patreon: / djware
Follow me:
Twitter @djware55
Facebook: / don.ware.7758
Gitlab: gitlab.com/djware27 - Наука та технологія
I keep meaning to switch my home desktop to LVM, but I'm used to being paid to work with LVM.
The overhead is trying how to remember to do stuff you haven't used for six months, and having all those lovely backups done by other people when you blow things up.
You pointed out the benefits very well. Of course it's better than raw partitioning, and if I fix lazy, I'll do it for free 🙂
I really appreciate this particular episode.
I've been using LVM for several years and come to rely on it heavily.
I give logical volumes to all of my guests (virtual machines).
When possible, those virtual disks are unpartitioned, so I can mount the LV on the host and fix a broken guest. VERY HANDY.
I usually feed entire physical volumes (unpartitioned) to LVM as PVs. Where boot tools like GRUB have heartburn over LVM, I put /boot on other media, like a USB flash drive. Works.
Thanks for the tips! I’ve used LVM over the years, but didn’t know all the advantages. Excellent!
Just can't help subscribing, love the content and the delivery. Thanks!
Welcome aboard!
Thank you, i was just thinking to try LVM myself.
Interesting. I’ve had home linux machines for decades and have avoided lvm because a. It’s new :) and b. gparted and mdm have met my needs. I usually use 3 partitions on my system drive (boot, swap and everything else). Data drives are raid 10 using mdm. For a home server this is all i have needed. I’ve been thinking about re-doing my home system though as I’ve acquired a few 1 liter pc’s with the idea of making a home lab along with greatly expanding storage for video production. Subscribed!
Awesome Thank you for Sharing! 💯✴
Being able to grow the pseudo-device in the hypervisor, and the grow the root file system in the VM, without needing to take the VM offline, is a nice side effect of using LVM.
We have gone the other direction with volume management. Our clients are all internal and never want to constrain their data. So to simplify management we just put almost everything in / and put the file system on the disk. Use the hypervisor to enlarge the disk then expand the file system. Cutting out the time needed for dealing with disk partitions and lvm. Just a different approach to meet a different use case.
Supposing I have a media archive consisting of 4 physical hard drives and I want to manage these as one large volume group. One of those drives fails. Does this not take the whole volume group down with it, essentially making data on the 3 remaining drives inaccessible? I am tempted to use LVM, but this prospect disturbs me, unless I've got it wrong.
Unless you are using a RAID level that provides redundancy, you are right and it's all gone. Look into the various RAID levels to figure out what works for you but RAID5 is where I'd be looking as parity is spread across the disks and you can lose one without loss of data. You'll only get three drives worth of storage space (given that they are all the same size) but you will not lose everything with one dead drive. And RAID is not backup: If you value the data back it up as well.
This is a bit confusing for me too!
As i understand , whe i make lvm volume i should configure raid after i made LVM. And logical volume management will configure itself that it will have one space drive.
Tho i could create raid 5 before setting up LVM.
Don't understand what the difference does it makes.
As i understood when one is setting up raid after i created created lvm... as i read when making brtfs or zfs raid 5 its checking data for bit rot while usual raid 5 is not resilient for bit rot.
@@Rom2Serge With LVM, you add physical drives into the LVM volume group which just makes the space on them available for you to set up how you like under LVM. You could set up RAID in hardware or via mdadmin and layer LVM on top of that but it makes a bit less sense to do it this way. With LVM, if you need more space, you can add another physical volume (disk drive) to the volume group and then extend the logical volume(s) to use some or all of that new available space. This is the flexibility of LVM and why you'd want to use it to manage your disks rather than setting RAID up at the lower level.
RAID5 spreads the parity out across all the disks in the pool so that it doesn't matter which single disk dies; the data on the dead disk is still preserved by the stripe + parity on the remaining disks.
As I understand it, ZFS (and perhaps BtrFS) will identify any 'bit rot' type of data loss *and automatically heal it* while a filesystem on LVM may be resilient to bit rot - i.e. you can still read the data because you have the parity - but it doesn't actually heal it.
Just one, but IMO important, bit. There is no problem with the /boot partition being on a logical volume. I've been using this with Ubuntu for a decade or so. Although probably it's a Debian thing. It's Fedora that cannot do it. Each time I've tried to install Fedore I've been put off by its subpar disk layout support. I always use LVM. Much easier to use and much more flexible than raw partitions. Plus, it gives stable device filenames so I don't need to resort to disk uuids to prevent problems with reshuffling /dev/sd* names. Even with ZFS I provide logical volumes as the drives. Contrary to popular belief, ZFS is not a volume manager. Or, at least, it's a very poor one.
BTW, one piece of advice for future LVM users. Put all your physical disks in a single volume group. When you create logical volumes you can specify the phyical disks that you want the volume to be placed on. But when you want multi-disk volumes, e.g. for RAID, they need to be in the same volume group.
2:26 Notice how 'ACCESS' has animated highlighting!
LVM become handy when you dealing with let say some e-mail servers, when your storage overgrows 2-4 Tb especially, if you've got only 200-300G disks.
It's possible to bump your storage performance with LVM in this case, and LVM-SSD-cache is also interesting thing, I used to do it before ZFS era.
And, I used to do LVM thin volumes and export it via iSCSI
Sure, but when you're talking about email servers then you are talking about enterprise-class applications... and whilst I credit DJ with an extensive amount of knowledge and many years experience in computing, his audience here is going to be made up mostly of people building Linux homelabs or just tinkering at home with something new. For those people, I don't see a case for using LVM in the first place - unless you really can't decide how to construct your arrays in the first place, in which case you should probably let someone who does know how to do it.
I thought this was going to help me declutter the house. In all fairness the video's thumbnail...
My god, it's David Letterman.
I use lvm on my server and have done for many years but it does make things more complex when you are trying to recover disks and it gets into a mess. I had a nonbooting install recently that turned out just to be that lvm was getting confused and not mounting something which just caused the boot to fail and restart the vm in a loop.
@DJ, Have you had premature failures with any particular SSD manufacturer?
why would you put btrfs on lvm? i dont think it has any benefit in that scenario?
Yeah, I agree with you. I see no benefit of layering BtrFS on LVM. BtrFS is meant to be more like ZFS which is a vertically integrated storage management system which handles everything from the raw disks up to the filesystems and provides snapshots and other higher level management tools.
I'm trying to learn LVM and have a few questions about the video:
1. Would it make any sense to use LVM for only a single physical volume?
2. I don't understand the /tmp step at 12:20. Doesn't tmpfs use RAM, not storage? What's the point of LVM managing /tmp and adding it to the fedora_fedora volume group?
3. What was the rationale for removing zram as swap at 15:34 ?
Just my guess:
1. LVM on one disk is useful if you want to manage multiple file systems possibly expanding or reducing them over time. You'd probably care more about this on a server rather than on a workstation. Personally, I think all in one partition for a workstation is more efficient as you don't need to fiddle with the logical volume (filesystem) sizing and you will probably notice if the disk fills up with log files and be able to address that promptly.
2. The /tmp logical partition is probably to prevent the system from grinding to a halt if temporary files fill up all available space. Again a significant safety measure for a server but on many workstations, /tmp is flushed every time you reboot the system which is usually more frequently than happens with a server.
3. I wondered about removing zram too. zram doesn't actually use disk space for virtual memory management but uses a portion of memory and compresses everything it swaps into it. It's therefore faster than a disk swap partition but still uses the memory; just a bit less of it. Opting for a swap volume just ensures that a bit more memory is available for processes. Possibly the swap volume is a better option for a server while zram makes a workstation a bit snappier under heavy load.
Again, just my guess. :)
@@dingokidneys Thanks, all you said makes sense to me.
Regarding point #2, the LVM partition for /tmp is just some sort of extra swap partition exclusively used by tmpfs? (not including the actual system-wide swap partition he made)
@@Yxcell It's not extra swap. Swap space is virtual memory where unused memory blocks get hived off to when the memory space is needed for something else.
The /tmp directory/filesystem is used for processes to save temporary files. These can be for any purpose and they are managed by the process creating them rather than the Virtual Memory system of the kernel. Often users can drop files into the /tmp area as well and so fill it up with downloaded cat pictures.
Ideally, the process that creates them should delete them when not needed but some processes are better at this than others. So the /tmp directory can fill up with old, no longer useful temporary files and swamp available space on the filesystem that they reside on. If /tmp is it's own partition, you have given it a hard limit on how much space it can occupy. Filling up might cause the next process that wants to use a temporary file to hang or die but it shouldn't bring your system down. If /tmp is part of the root file system, old useless temporary files can fill all available space and bring the whole system to a screeching halt which is less good.
@@dingokidneys Hmm, I guess I need to learn more about LVM and tmpfs, haha. 😆
So you said that "If /tmp is its own partition..."
Where does that partition live, exactly? Is it a partition of a RAMdisk? (since tmpfs obviously wouldn't be on a persistent storage device)
At 12:35 in the video we can see that he made a 2GiB logical volume in a QEMU HARDISK (/dev/sda) using LVM under the fedora_fedora volume group. I find this confusing.
I understand the idea behind tmpfs and the ability/need to set quotas for it (as that's the advantage it has over ramfs). I'm just missing the piece of the puzzle for how LVM and logical volumes fit into the picture.
Thanks for patience with me. :)
@@Yxcell Maybe I'm playing fast and loose with terminology here. The /tmp directory, just off the / root directory, is a place to store temporary files and doesn't have anything to do with tmpfs. If you run 'mount' on your system you will likely see a few 'tmpfs' filesystems such as '/run', '/run/lock', '/run/user/1000', '/dev/shm' and these do actually exist in memory because access to them needs to be fast to keep the system snappy. They are usually small filesystems because you don't want them eating up RAM that you want to use for more interesting things and the files are small and numerous. On my system the /run/user/1000 filesystem contains over 980k files alone at the moment but the total memory occupied by the four tmpfs filesystems together is 54MiB. Trying to run this on an SSD let alone an HDD would cause the system to grind to a halt, I've no doubt.
The /tmp area, on the other hand, can take up considerable space with potentially large files written into it, which makes a 'tmpfs' filesystem very inappropriate to hold it. DJ put it in a logical volume because he was using LVM to break up the space available on his virtual hard disk, but on a machine without LVM, just MBR or GPT partitioning, you would create a partition to hold it and thereby limit the amount of space it could occupy, then mount that partition onto /tmp in the root filesystem with something like 'mount /dev/sda4 /mnt'. So your partitioning scheme (without LVM) might be, as a simplified example:
/dev/sda1 -> /boot
/dev/sda2 -> swap
/dev/sda3 -> /
/dev/sda4 -> /tmp
/dev/sda5 -> /home
So LVM logical volumes are equivalent to, but provide more flexibility than, simple partitions on a hard disk where no other disk management is used.
Try going back over the video and googling or checking out the man pages for things that don't make sense to you. Virtualising disk storage can be confusing until you get a good handle on the terminology and the concepts behind it.
It’s not related to LVM but do you still use GlusterFS?
I took my servers down about a year ago and put up another zfs pool to replace them, so I am not using glusterfs anymore.
@@CyberGizmo oh ok. Is it because there is no support from Red Hat to the project?
@@lapin972 no the amount of storage I needed became impossible using the ARM computers, since each one could only handle one HD. So went over to ZFS where I could use as many HDs on the same machine as I had SATA ports for. If I had gone x86 to begin with I would still be using GLusterFS
@@CyberGizmo thank you for your answer. I was looking for a GlusterFS distributed and I rewatched your video posted 2 years ago.
I prefer ZFS(latest version). I also prefer debian or freebsd btw😊
I use ZFS for my two nas servers, and I also use debian and freebsd. FreeBSD as you probably already know doesn't support LVM
Having used Linux since 1996, I installed LVM once on a software RAID installation, decided it was completely pointless and just added a layer of additional management that I didn't need, removed it and have just used ext4 ever since - whether on RAID or single disks.
I'm not running an enterprise-level solution at home, or on friends' and family's networks when I build and support those also, the KISS principle applies.
If you plan storage, disk sizes, partitions and backups before you deploy them (i.e. you "engineer" stuff properly) and follow proper logical hierarchies for directories and sub-directories, LVM is of no use.
If one is not organised or logical in how one stores data, then maybe LVM puts a band-aid over that administrative failing that one has.
mdadm is good, but lvm can make it too. So it's doubling functionality.
@@eugenesmirnov252 And I need "doubling functionality" because...? By my definition, as a computer engineer, that is just "over-engineering".
Just to point out, if you do use physical partitions, the only one you can resize easily is the last on on the disk, if you need to resize the ones above you can use gparted and reorganize each one in turn so you have free space between then...what a waste of time.
@@CyberGizmo But I don't resize, I stick with what I initially create, generally I just have one partition on data storage disks anyway - there's only me and the wife in our home environment, plus a few friends that have access through VPN (as do I when I am away from home). Linux file permissions are granular enough for me to put on the access controls that I need to for that small number of people.
By the time I start running out of space, the disks are 3 or 4 years old anyway and probably ready to be replaced with bigger drives - so replace them, build the new RAID array, stick ext4 on them, restore from the old array and I'm done.
Again, in an enterprise environment with maybe hundreds of users with their own access permissions and storage arrays, I get why LVM would be important - but it's not necessary for small homelabs with just a handful of users. Plus it's just one more application layer to go wrong anyway if you don't need it.
I guess if you are working with small devices like a bunch of old SSDs or small disks LVM might be useful, but with huge multi-terabyte drives so cheap, I'm not sure I see much benefit. To me a stripe/raid manager would be a lot more useful. I could be missing some of your concepts on LVM.
LVM provides software RAID, but so does mdadmin, BtrFS and ZFS. LVM has been around longer on Linux, is very reliable and may be better understood by many people.
@@dingokidneys
I want to do two RAIDs, or maybe one RAID of a MIRROR and a STRIPE, mirroring the elements of the stripe - but on a Mac.
I have a bunch of 10TB USB3 desktop drives that I think could be made faster and more secure by this this with "n" elements, "n" being 2, 4, 6 or 8 10TB drives linked together, but I am not sure if Apple iMacs even support that any more. I am saving up to buy a new iMac maybe next year.
I used to play around with Linux in the late 80s and early 90s and it was fun, but today I just want off-the-shelf stuff that just works. However Apple with all their trickiness is starting to work my last nerve as they say
@@justgivemethetruth I know nothing about Apple gear but I wouldn't try setting up RAID (other than *perhaps* a 2 disk mirror) on USB connected devices. Their identifiers can change from boot to boot which would break the RAID and hose your data.
@@dingokidneys
You know nothing more about a lot more than Apple gear it seems,