Roman, PLEASE create more Linux content . You just went through some pretty complex technologies spending mere minutes on each topic. Not only that but also some very helpful tips which us average Linux users usually only discover by accident. Really good content and very well delivered. More of the same please, much more.
Just an idea! When running the "badblocks" test open 4x SSH terminals and run one in each = 4days will become 1day, good job today is a bank holiday :)
Really good video! I was searching for zfs on Linux and this was one of the results - and am I glad it was! You have a very nice easy to understand and follow way of describing each part of the process. No need to make things complicated when you can do it like this. Super great guide! And you got a new subscriber :)
It's also advisable to add a spare drive to a zpool. This way in case of a disk failure, spare vdev starts resilvering (taking place of a degraded vdev).
"I Claim The Golden Goof Spotter Price" - 8:14 - ok very trivial, but have when you get a chance to have a go at your teacher :) Very detailed editing .... for those that cant see a clue between the highlighted "sdd" and the "sudo zpool c" - Many thanks for the guides - invaluable.
Nice video but what about extra options during zpool creation like zpool create -f -o ashift=12 -o autoexpand=on -O atime=off -O compression=lz4 $zp \ mirror /dev/disk/by-id/disk1 /dev/disk/by-id/disk2 \ mirror /dev/disk/by-id/disk3 /dev/disk/by-id/disk4 and explain the arguments as well. i t would be more thorough
I dont get any info to use with /dev/disk/by-id/ I have to use by-path but he can change also because it changed on me after I update the raid controller firmware and bios. only thing that was safe was the raid5 all other single disk lost there by-path ID.
Great tutorial but even though i created a zfs pool it was only ever visible in terminal, wasn't visible in dolphin or files, converted it back to btrfs raid as that seems to work more seamlessly with Ubuntu GUI
Will you do an update video on the full release of Cold War? or are your findings from the beta still relatively the same. Cus I've seen some new options in the game.
Hello, great video and I was trying to do what you did but my speed is nowhere near what your getting. I got everything but not the same! only getting 178MB/scopying from my hard drive or NVME threw the new work. I did a raidz1 with 7 drives . testing the speed over the network and with the hard drives and I cant get 10GB threw my NIC. Any help would be great or a video on steps we can do to get higher threw speed.
How does the 900MBps transfer speed of this Raid10 configuration add up? Assuming your drives are 200MBps transfer and two of them are mirrored shouldn't that equate to 400MBps? Or if ZFS does some caching magic to utilize all 4 drives at once, still at most 800MBps best case? Real world wouldn't that number be less, like 650-750MBps tops? I could understand 800MBps in a RAID5 array of 5 drives of this speed but this isn't making sense to me. Could you explain how this all adds up?
Hello Tech Guides, excellent tutorial, I have a quick question, I thought that on ZFS after creating a mirror and the ZFS mount happens automatically after each reboot? however, when I run `df -h` or `mount` I don't see the name of the ZFS pool mounted :( is there a follow up tutorial about this or do you have any suggestions on what might be happening? Thank you in advance.
Great video, thank you for taking the time sharing your knowledge with us mere mortals. Can I ask you what is the command you have launch to show the network data transfer speed? How is it being mesured to show your 8.03GB on your enp13s0? Thanks :)
Hello. Is it possible to add an SSD (1xNVME) cache to this type of ZRAID as your described (4 drives, zfs mirror vdevs)?. If SSD cache is not recommended, how much RAM should I have for this setup, assuming the hard drives are 10TB each. I will be using Linux Mint 22.
You mean clone the boot drive ..? I think dd would do that, but if not, at minimum ddrescue will 100%; that's exactly what it's for. Cloning drives. It used to be free which is why I suggest you check if dd by itself will, as ddrescue is oriented to failing drives but might be using the dd command in addition to other features.
I'm afraid yes this could very likely be a pain to deal with. If you have frequent power cuts than I would highly recommend that you get a USP uninterruptible power supply (UPS) which can than safely shut down your server if you intend on using ZFS.
I followed your previous tutorial about settings these up as drives in my cloud, now I am stuck when trying to check for badblocks. Did I mis a step, or do I need to undo the previous drive mounting from the first tutorial?
Yes badblocks does not work when the drives are mounted. Also don't do it like I showed if you already have any data on the drives!! Badblocks in the mode I showed will wipe all data from your drives (there is also a non-destructive test but it will take forever)
Absolutely, although I'm already exceeding my 10GE network with this solution (unless copying terabytes of data) so I'm not really trying to get anything faster working :)
I am using a very old office server without ECC memory to create my first home server as a test run. Considering I dont have ECC memory, should I proceed with this guide or use the 2019 videos?
I would definitely recommend to use my updated guides, but you probably don't want to use ZFS on your server. The rest can be applied to basically any hardware :)
9:51 - Can I please ask *why*? What are the possible ramifications if I don't create a folder inside the zpool and just mount it directly? I am new to this and I haven't seen this advice anywhere else. Thank you!
Not entirely sure anymore but it had to do with creating snapshots I think. Just keeps things organized too (you don't have tens of folders in your root) but you can probably also just save everything in the root
Thank you for the response! I think I will make the folder. After all, it can be hidden from view anyway, by pointing the sambashare right to the created folder. I especially appreciate the "creating drive aliases" part, as I have omitted it during testing stuff and I will be moving the drives to another hardware in couple months. Should spare me a very probable headache.
I also think moving content in the root directory is more difficult ... certainly if you've defined it as a shared folder...as it'd break any shares you had.
Do you mean a RAID card or simply a PCIE SATA extension board? Because I'm not sure I would recommend a RAID card because of the added level of complexity and additional potential for catastrophic hardware failure
@@kithuto I am using a raid card and it is by far faster than the onboard sata controller. Yes there are problems with the raid card cause my /dev/disk/by-id doesnt show the ID and my raid card doesnt show the hard drive aID either. It also swaps the /sda around . but I did do a small test between onboard and raid card. the raid card copied files at around 200MB/s and the onboard did it a 70MB/s. Waiting for some new drives to come in at the computer store and I will create 4 drive mirror and test them with the onboard NVME.
If drive 03 failed, and you need to replace it how do you do that? if i put a new drive in, do i change the vdev aliases before doing a replace command?
You could, but I would generally just use something like mergerFS if the only point is to create one big combined drive. Or did you want to make use of some of ZFS' other functionalities?
Yes. Really how much safety/performance/space you get out of the drives is up to the user/setup of the drives. In this case half the drive space is used for redundancy (safety) But due to the stripe you can only lose 1 out of each of the mirrors. So you can lose up to 2 drives. 1 and 3 can be removed and everything is fine. But half your data is on drive 2 and the outer data is on drive 4. Your really in trouble but your data is still there. All speed improvements are gone etc etc. But if 1 and 2 die at the same time. (or 3 and 4) data is lost. So really only one drive can die without worry. 2 drives means you have lost data or no redundancy left in your drives. The hole point of this is more about gaining speed. And less about safety. You can have 2 of 3. Speed, Safety, Capacity. This setup is Safety and Speed. With the big thing being SPEED. Safety is really just that you can lose 1-2 drive out of 4. That is kinda shit but respectable. You can not expect much from 4 drives when it comes to how many you can lose. And you need the speed you can get out of slow HDD's.
@@TheDiner50 Thanks a lot for your reply. Is it possible to mirror 1 to 2 and latter to 5 to have 3 disks instead of two, only ? (Same for disk 3 to 4 and 6) Then 1 and 3 can fail, since 5 ist still present. For me it is important to gain size and savety.- Other question: Is it possible to add new drives to a working set if space gets out ?
I don't get it why can you reach that speed with hdd. Even with parallel write, I would expect upto 5gbps. How can each hdd gets 2+ gbps??? (I know sata cap at 6 or 10) Is it a special hdd or something?
@@TechGuides I tried using 6 drives on a mirror but I was still getting the same speed nowhere near 10GB, as for the ram it was not using anymore than normal. I am copying files to it now and it says 5% used 6GB. Would love to get my system running fast like that. Any ideads or help or new video on doing that.
RAID Z2 made RAID 10 obsolete With RAID Z2 you can lose any 2 drives and still have your data. With RAID 10, if you lose both drives in a mirror, you lose data.
There are plenty of reasons one might want to use one RAID type over another and redundancy is only one factor to consider. How fast does it need to be? If you lose a drive, can you afford a few days with decreased performance while the array resyncs? I run a database server at work on RAID 10 and a 144TB media server in RAID 6. Both have multiple offsite backups. The database is small and quick to restore from backup, so even if I lose the RAID, I can have it back in an hour or so. It's much more important for it to be fast than drive-failure tolerant. The media server is backed up onto slow LTO tape. Restoring it all would take weeks. Two drive redundancy was a priority. The motherboard on the 144TB server blew up last week after a power surge, even behind a capable UPS. Luckily the data was fine, but others without backups have not been so lucky... Never make the mistake of thinking drive redundancy is the only important consideration. Anything could fail, or even the building catch fire. RAID is not a backup solution.
Something doesn't add up here. You're saying your drives W: 450MB/s each? I've do data recovery professionally; I've literally NEVER HEARD of a 300 MB/s HD. Let alone, 450 - 500MB/s .. sustained, with perfect 2x performance in a mirror. And yes, I've used ZFS for over 5 years, and have set up about 10 units for customers during that time. Every single one of them with an SFP+ card. I don't mean to poop on your parade, it's just something I've never seen spinning drives do.
Huh sorry its been a while but when did I mention such individual wirte speeds? I think each drive tops out at about 220MB/s but the data is additionally striped and is thus written to two disk simultaneously with gives me write speeds of roughly 450MB/s. Also not the ZFS does some magic buffering with RAM so that I get the 1GB/s transfers. Did I mess it up when explaining?
Guess Linux/Ubuntu messed this up already, MISSING STEPS BELOW: 10:14 after "zfs create rpool/data" "cd .." to the root directory then you can "sudo chown -R user:user data" as "sudo chown -R user:user rpool/data" will fail. For SMB "rpool/data" "/rpool/data" "/rpool/data/" "/data" and "data" will all fail, don't even bother save yourself 6 hours of troubleshooting...
This server series is just a gold mine, thank you so much
Really appreciate it man! I have planned to release a few more vids in this series soon :)
Roman, PLEASE create more Linux content . You just went through some pretty complex technologies spending mere minutes on each topic. Not only that but also some very helpful tips which us average Linux users usually only discover by accident. Really good content and very well delivered. More of the same please, much more.
Really good stuff here. I love how you add tips, for instance, the alias naming of your drives for ease of reference. Keep up the great videos.
Glad you liked it, thanks!
Just an idea! When running the "badblocks" test open 4x SSH terminals and run one in each = 4days will become 1day, good job today is a bank holiday :)
Yes thats absolutely how it should be done haha
Really good video!
I was searching for zfs on Linux and this was one of the results - and am I glad it was!
You have a very nice easy to understand and follow way of describing each part of the process. No need to make things complicated when you can do it like this.
Super great guide!
And you got a new subscriber :)
Great tutorial i ran this through a test Virtual Machine and it worked a treat, next step physical hardware. Thank you for the great tutorial.
It's also advisable to add a spare drive to a zpool. This way in case of a disk failure, spare vdev starts resilvering (taking place of a degraded vdev).
Yes, that's definitely a good idea (if one has the money)!
"I Claim The Golden Goof Spotter Price" - 8:14 - ok very trivial, but have when you get a chance to have a go at your teacher :) Very detailed editing .... for those that cant see a clue between the highlighted "sdd" and the "sudo zpool c" - Many thanks for the guides - invaluable.
Thank you very much for the help! I'm looking forward to the next video on nextcloud!
Thanks for watching! Cool, I'm working on it :)
@@TechGuides rllly? please relase it!!!!!! I love your tutorials and I just finished up.
Great guide, Roman. QQ, how can I get the pool to show up in the Files GUI?
Found it, had to use the zfs to configure the mountpoint.
Thank you for that great tutorial!
Thanks!
Nice video but what about extra options during zpool creation like
zpool create -f -o ashift=12 -o autoexpand=on -O atime=off -O compression=lz4 $zp \
mirror /dev/disk/by-id/disk1 /dev/disk/by-id/disk2 \
mirror /dev/disk/by-id/disk3 /dev/disk/by-id/disk4
and explain the arguments as well. i t would be more thorough
I dont get any info to use with /dev/disk/by-id/ I have to use by-path but he can change also because it changed on me after I update the raid controller firmware and bios. only thing that was safe was the raid5 all other single disk lost there by-path ID.
Do disk by path. not id. By path will mark the actual hardware address of the port. So if a drive goes dead, it is way easier to change.
Great tutorial but even though i created a zfs pool it was only ever visible in terminal, wasn't visible in dolphin or files, converted it back to btrfs raid as that seems to work more seamlessly with Ubuntu GUI
Any indication of when the nextcloud part is releasing? Really looking forward to it!
Will you do an update video on the full release of Cold War? or are your findings from the beta still relatively the same. Cus I've seen some new options in the game.
I will eventually do an upgraded CW fps guide but I got so many other projects in the pipeline as well...
Hello, great video and I was trying to do what you did but my speed is nowhere near what your getting. I got everything but not the same!
only getting 178MB/scopying from my hard drive or NVME threw the new work. I did a raidz1 with 7 drives . testing the speed over the network and with the hard drives and I cant get 10GB threw my NIC. Any help would be great or a video on steps we can do to get higher threw speed.
How does the 900MBps transfer speed of this Raid10 configuration add up? Assuming your drives are 200MBps transfer and two of them are mirrored shouldn't that equate to 400MBps? Or if ZFS does some caching magic to utilize all 4 drives at once, still at most 800MBps best case? Real world wouldn't that number be less, like 650-750MBps tops?
I could understand 800MBps in a RAID5 array of 5 drives of this speed but this isn't making sense to me. Could you explain how this all adds up?
Nice video. @14:29, what command did you use to get the display looking like that? I’ve used htop and top but not seen one look like that.
glances :)
Have you ever used - ncdu ..? it actually looks kinda similar (and you can use brew to install it) ...
Hello Tech Guides, excellent tutorial, I have a quick question, I thought that on ZFS after creating a mirror and the ZFS mount happens automatically after each reboot? however, when I run `df -h` or `mount` I don't see the name of the ZFS pool mounted :( is there a follow up tutorial about this or do you have any suggestions on what might be happening? Thank you in advance.
Great video, thank you for taking the time sharing your knowledge with us mere mortals.
Can I ask you what is the command you have launch to show the network data transfer speed? How is it being mesured to show your 8.03GB on your enp13s0?
Thanks :)
You're welcome! Do you mean the application shown at 14:24? Its called "glances"
Amazing content. Thank you so much!
Glad you liked it!
I'm trying to follow along here. I'm stuck at 8:06 when I enter "cd by-vdev/" I get "-bash: cd: by-vdev/: No such file or directory"
Hello. Is it possible to add an SSD (1xNVME) cache to this type of ZRAID as your described (4 drives, zfs mirror vdevs)?. If SSD cache is not recommended, how much RAM should I have for this setup, assuming the hard drives are 10TB each. I will be using Linux Mint 22.
Hi, whats the name off the Tool showing at 14:25?
Thx
its called "glances", cheers!
Please one more video about Obs setting for 10-25mbps internet speed
loved the video bro. though i have to ask, whens the next cloud part coming?
Thanks man! Actually the next part should release tomorrow :)
Any application use interface or gui easy more than termenail,?
Can you make a video on how to take a default 21.04 desktop zfs (wizard) install and mirror it to a second identical drive please?
You mean clone the boot drive ..? I think dd would do that, but if not, at minimum ddrescue will 100%; that's exactly what it's for. Cloning drives. It used to be free which is why I suggest you check if dd by itself will, as ddrescue is oriented to failing drives but might be using the dd command in addition to other features.
ZFS z2 is the best for home servers. It is recomended to have a low-level z2 rather doing it through ubuntu.
No 'smbpasswd -a ' ?
Hey thanks a lot, i was really struggling for info on how to set up a zfs Raid 10 on my new server as im new to linux :)
Glad I could help!
@@TechGuides One question though if you don't mind, I have quite frequent power cuts. Will zfs be a pain to keep recovering from crashes?
I'm afraid yes this could very likely be a pain to deal with. If you have frequent power cuts than I would highly recommend that you get a USP uninterruptible power supply (UPS) which can than safely shut down your server if you intend on using ZFS.
I followed your previous tutorial about settings these up as drives in my cloud, now I am stuck when trying to check for badblocks. Did I mis a step, or do I need to undo the previous drive mounting from the first tutorial?
Yes badblocks does not work when the drives are mounted. Also don't do it like I showed if you already have any data on the drives!! Badblocks in the mode I showed will wipe all data from your drives (there is also a non-destructive test but it will take forever)
What is the monitoring screen / utility you are showing at 14:26?
glances (sudo apt install glances)
can we setup iSCSI instead of smb? this will probably give much faster access over the network
Absolutely, although I'm already exceeding my 10GE network with this solution (unless copying terabytes of data) so I'm not really trying to get anything faster working :)
Interesting... So why would I need trueNAS OS if Ubuntu OS can do it? 🤔
Because its more work by the user ;)
I am using a very old office server without ECC memory to create my first home server as a test run. Considering I dont have ECC memory, should I proceed with this guide or use the 2019 videos?
I would definitely recommend to use my updated guides, but you probably don't want to use ZFS on your server. The rest can be applied to basically any hardware :)
How do you do this with 6 drives?
9:51 - Can I please ask *why*? What are the possible ramifications if I don't create a folder inside the zpool and just mount it directly? I am new to this and I haven't seen this advice anywhere else. Thank you!
Not entirely sure anymore but it had to do with creating snapshots I think. Just keeps things organized too (you don't have tens of folders in your root) but you can probably also just save everything in the root
Thank you for the response! I think I will make the folder. After all, it can be hidden from view anyway, by pointing the sambashare right to the created folder.
I especially appreciate the "creating drive aliases" part, as I have omitted it during testing stuff and I will be moving the drives to another hardware in couple months. Should spare me a very probable headache.
Exactly, the mounted folder will just be the one you've created :) Great!
I also think moving content in the root directory is more difficult ... certainly if you've defined it as a shared folder...as it'd break any shares you had.
Do you recomend RAID 5 with a dedicated PCIE card?
Do you mean a RAID card or simply a PCIE SATA extension board? Because I'm not sure I would recommend a RAID card because of the added level of complexity and additional potential for catastrophic hardware failure
@@TechGuides a raid card. Okey thx.
@@kithuto I am using a raid card and it is by far faster than the onboard sata controller. Yes there are problems with the raid card cause my /dev/disk/by-id doesnt show the ID and my raid card doesnt show the hard drive aID either. It also swaps the /sda around . but I did do a small test between onboard and raid card. the raid card copied files at around 200MB/s and the onboard did it a 70MB/s. Waiting for some new drives to come in at the computer store and I will create 4 drive mirror and test them with the onboard NVME.
If drive 03 failed, and you need to replace it how do you do that? if i put a new drive in, do i change the vdev aliases before doing a replace command?
Sorry I never had to do this so I don't know the exact procedure but I guess there are many good tutorials out there on exactly this topic, no?
Superb 👍
Please add link to mentined video in 2:40
almost 1 GB/sec with just 4 spinning drives? That is good!
Thanks to the magic of ZFS caching
Please guide me on how to improve (OBS)
Hi man, what exactly would you be interested when it comes to an updated OBS tutorial? Streaming? Recording?
@@TechGuides streaming
@@TechGuides my internet not so fast
Can I do ZFS without RAID, I only have 3 200GB ssds?
You could, but I would generally just use something like mergerFS if the only point is to create one big combined drive. Or did you want to make use of some of ZFS' other functionalities?
Question is, what will happen if drive 1 and 3 will be broken at the same time ? I see that we have four copies and nothing will happen. Am I right ?
Yes. Really how much safety/performance/space you get out of the drives is up to the user/setup of the drives.
In this case half the drive space is used for redundancy (safety) But due to the stripe you can only lose 1 out of each of the mirrors. So you can lose up to 2 drives. 1 and 3 can be removed and everything is fine. But half your data is on drive 2 and the outer data is on drive 4. Your really in trouble but your data is still there. All speed improvements are gone etc etc.
But if 1 and 2 die at the same time. (or 3 and 4) data is lost. So really only one drive can die without worry. 2 drives means you have lost data or no redundancy left in your drives. The hole point of this is more about gaining speed. And less about safety.
You can have 2 of 3. Speed, Safety, Capacity.
This setup is Safety and Speed. With the big thing being SPEED. Safety is really just that you can lose 1-2 drive out of 4. That is kinda shit but respectable. You can not expect much from 4 drives when it comes to how many you can lose. And you need the speed you can get out of slow HDD's.
@@TheDiner50 Thanks a lot for your reply. Is it possible to mirror 1 to 2 and latter to 5 to have 3 disks instead of two, only ? (Same for disk 3 to 4 and 6) Then 1 and 3 can fail, since 5 ist still present. For me it is important to gain size and savety.- Other question: Is it possible to add new drives to a working set if space gets out ?
hey I think they patched uploading with vpo9 because it's not working do you have a fix?
If you upload at 1440p or 4K, you will get VP9, at 1080p any video is now AVC even from people with 2million subs
Your last warzone fps settings is pretty outdated can you do a new one ?
Yes very soon!
@@TechGuides 🙏🏻🙏🏻🙏🏻
I don't get it why can you reach that speed with hdd.
Even with parallel write, I would expect upto 5gbps.
How can each hdd gets 2+ gbps???
(I know sata cap at 6 or 10)
Is it a special hdd or something?
Data is first written to RAM and only then to the HDD's. Its also split across 2 drives all the time, essentially doubling the write speed
@@TechGuides I tried using 6 drives on a mirror but I was still getting the same speed nowhere near 10GB, as for the ram it was not using anymore than normal. I am copying files to it now and it says 5% used 6GB. Would love to get my system running fast like that. Any ideads or help or new video on doing that.
RAID Z2 made RAID 10 obsolete
With RAID Z2 you can lose any 2 drives and still have your data.
With RAID 10, if you lose both drives in a mirror, you lose data.
There are plenty of reasons one might want to use one RAID type over another and redundancy is only one factor to consider. How fast does it need to be? If you lose a drive, can you afford a few days with decreased performance while the array resyncs?
I run a database server at work on RAID 10 and a 144TB media server in RAID 6. Both have multiple offsite backups. The database is small and quick to restore from backup, so even if I lose the RAID, I can have it back in an hour or so. It's much more important for it to be fast than drive-failure tolerant.
The media server is backed up onto slow LTO tape. Restoring it all would take weeks. Two drive redundancy was a priority. The motherboard on the 144TB server blew up last week after a power surge, even behind a capable UPS. Luckily the data was fine, but others without backups have not been so lucky...
Never make the mistake of thinking drive redundancy is the only important consideration. Anything could fail, or even the building catch fire. RAID is not a backup solution.
Something doesn't add up here. You're saying your drives W: 450MB/s each?
I've do data recovery professionally; I've literally NEVER HEARD of a 300 MB/s HD.
Let alone, 450 - 500MB/s .. sustained, with perfect 2x performance in a mirror.
And yes, I've used ZFS for over 5 years, and have set up about 10 units for customers during that time.
Every single one of them with an SFP+ card. I don't mean to poop on your parade, it's just something I've never seen spinning drives do.
Huh sorry its been a while but when did I mention such individual wirte speeds? I think each drive tops out at about 220MB/s but the data is additionally striped and is thus written to two disk simultaneously with gives me write speeds of roughly 450MB/s. Also not the ZFS does some magic buffering with RAM so that I get the 1GB/s transfers. Did I mess it up when explaining?
Next time trying to get hard drives with 520 bytes sectors
I was told it's a myth that you need ECC Ram.....
Definitely not. You can read up more on it here: louwrentius.com/please-use-zfs-with-ecc-memory.html or not, whatever you like
Guess Linux/Ubuntu messed this up already, MISSING STEPS BELOW:
10:14 after "zfs create rpool/data" "cd .." to the root directory then you can "sudo chown -R user:user data" as "sudo chown -R user:user rpool/data" will fail.
For SMB "rpool/data" "/rpool/data" "/rpool/data/" "/data" and "data" will all fail, don't even bother save yourself 6 hours of troubleshooting...
Did you just snap at me..