Why You Should NOT Use RAID 5 Storage ( But Use RAID 6! )
Вставка
- Опубліковано 22 вер 2015
- RAID 5 is not a good choice for redundancy these days, and likely won't protect you against a disk failure. Here's why you should instead use RAID 6 in your NAS or RAID array!
▼ Join the conversation! ▼
Twitter ▻ / thiojoe
Facebook ▻ / thiojoetv
Instagram ▻ / thiojoe
Website ▻ www.thiojoe.com
▼ More Videos ▼
How Long do Hard Drives Last?: • How Long Do Hard Drive...
How Are Old Movies in 1080p?: • How Can Old Movies be ...
▬▬▬▬My Channels▬▬▬▬
Comedy ▻ / thiojoe
Gaming ▻ / cachegaming
Extra ▻ / @thio
Get cool merchandise: thiojoe.spreadshirt.com/
Support me on Patreon: / thiojoe
--------------------------------------------------------- - Наука та технологія
A lot of people are having trouble understanding that yes, a single URE can indeed cause the entire array to fail to rebuild. Because of the nature of the parity system, it is NOT the same as having a URE on a standalone drive, where only that single file is lost. You need it ALL to rebuild it all. You might be able to send the drives to a data recovery service who could potentially rebuild the array and only lose the files directly using the lost parity data, but that would be very expensive and time consuming, and not necessarily a guarantee.
Not to mention that because array rebuilds take so long, and are very very intensive, it drastically increases the chances of having yet another URE in that time, which could completely ruin any chances of any recovery at all.
You can google something like "Raid 5 rebuild failure probability calculator" if you don't believe me on the odds. But using 10^14 URE rate drives is extremely risky and you actually are VERY likely to experience a rebuild failure even with small arrays.
In any case, when looking to buy drives for a NAS, my suggestion is ONLY buy a drive with a URE rate equal to or better than 1 per 10^15 bits. This can be found by googling for the 'Data Sheet' of the specific model drive you're looking for. It will be called something like "Non-recoverable read errors rate", "Error Rate (non-recoverable)", "Unrecoverable Bit Error Rate (UBER)", or something similar, probably listed along with the 'reliability' specs in the data sheet.
I haven't really been able to even find any HDDs with a URE rate better than 10^15. However, with SSDs you can usually find better URE rates. Data sheets for consumer desktop SSDs (Like samsung's EVO/PRO SSD line or Seagate's Barracuda SSD line) don't list URE rates. But Seagate's "Ironwolf" NAS SSD line actually have URE rates of 1 in 10^17, so 100 times better than any HDD I've seen. However those cost several times more than an HDD (right now it's $750 for just a 3.8TB drive).
Jeeeez rebuild issues where the whole thing dies is when old RAID cards would just give up. Software doesn't, could lose a few files around that failure (which isn't as likely as any of these calculators say. Rebuilds read from multiple disks too...
So how does a raid 6 help with ures?
I have a question. Given your example you have 14TB with 7 drives in RAID 5 parity is split evenly across every one. So that means that each drive has only 2TB/7 dedicated for parity. When rebuilding a failed drive does it mean you need to read remaining DATA plus Parity ( whole 12TB ) or not ( maybe a smaller amount is enough )? What i want to ask is basically how does rebuilding work?
@@dennisrkb Watch the video AGAIN!!
I understand that, but according to a thread:
"most hardware RAID will abort the reconstruction and some will also mark the array as failed, bringing it down. The rationale is that if an URE happens during a RAID5 rebuild it means some data are lost, so it is better to completely stop the array rather that risking silent data corruption. Note: some hardware RAID (mainly LSI based) will instead puncture the array, allowing the rebuild to proceed while marking the affected sector as unreadable (similar to how Linux software RAID behaves).
linux software RAID can be instructed to a) stop the array rebuild (the only behavior of "ancient" MDRAID/kernels builds) or b) continue with the rebuild process marking some LBA as bad/inaccessible. The rationale is that it is better to let the user do his choice: after all, a single URE can be on free space, not affecting data at all (or affecting only unimportant files);
ZRAID will show some file as corrupted, but it will continue with the rebuild process (see here for an example). Again, the rationale is that it is better to continue and report back to the user, enabling him to make an informed choice."
meaning that its not a big deal if I use Linux software RAID 5? right?
I would argue that the bigger lesson here isn't to get overly hung up over RAID5 vs RAID6, but to remember not to fully trust either of them and always have a proper backup. A good backup system turns a nightmare into an inconvenience.
100%. A Raid is not a substitute for a backup, no matter how you stripe or parity it. Even raid 60 could have an internal power surge or something and toast all those spinning drives at once... your data NEEDS to be elsewhere, off the raid, if it matters.
I know some nutjob who uses 8 SSDs in raid zero. That dude's crazy.
+Colton Blumhagen Well, I'm not familiar with the error rates of SSDs. If they have very low error rates (if at all), then it might be safe. This video kind of assumes the hard drives have a URE of 10^14, but if you have one with 10^15 or even 10^16, you could probably get away with RAID5.
Bombersnomore Haha. This guy gets it.
Yeah, this video goes hand in hand with the "how long do hard drives last" video. But that one doesn't go into the difference between sata, sas and ssd drives and what vendors make the higher quality drives. Unfortunately because of all the low quality production these days all of this makes a difference.
+Ebel Ray I might have a new video coming soon about different types of drives ;)
+ThioJoeTech i was going to suggest that as a video for you but you beat me to it. Hard drives can give you lots of directions for multiple videos too I think.
What a load of BC.
First of all, explanation for an average home user goes like this:
If you build a redundant array with:
2 drives - your only meaningful choice is RAID1
3 drives - your only choice is RAID5
4 drives - RAID5 for capacity vs RAID10 for redundancy.
Only starting with 5 drive array you should consider using RAID6.
And, of course, you don't lose ALL data with RAID5 disk loss and URE. You lose one file. It might be critical for an enterprise environment, IF they don't make backups. For an average homeuser Joe it means one corrupted photo, or movie. Not the end of the world. And even Joe should make backups of critical data.
Andris Orinskis and beside that, if the data is so important you need an offsite backup, so RAID 6 is not needed
Thanks for th einfo. If the controller restored a degraded raid array, will it report any URE's (and affected files?)?
Andris Orinskis what if you loose an important file that would go un noticed until necessory for use
If you have 3 drives and have to put them in an array, you buy another drive to do so. Who even needs to RAID 3 drives in which kind of environment for what purpose anyway?
Exactly what I thought also when watching the video. But nevertheless I'm thankful the guy made the video. It's not wrong, just not applicable for me.
2 drives use raid 1,
3-4 drives use Raid 5,
5+ drives use raid 6.
I use Raid 5 for 10 SAS drives
Raid 5 can have u to 16 drives
@@jabetajones the amount disks is limited by the nas or server your using
@@leexgx oh ok im kinda new to pcs
@@jabetajones the more disks you use the higher chance of a double fault can happen and loss of data (or all data if 2 disks fail) that's where raid6 should be used,
but size of the disks also matter as well over say 4TB you should use RAID6 as well (unless you don't mind restoring your data from backup if you have one) if you don't have a backup you must use RAID6 minimise risk of total data loss but you should have critical information backup up (like pictures, personal videos, and documents)
The problem with raid 5 it's not very forgiving when your rebuilding the array after a failed disk has been replaced because if another fault happens
1, because of a data error witch will result in Data loss (normally you can still access the remaining unaffected data)
2, a second disk fails and you lose all data with bi way of recovery
The statistical probability of 3 faults happening when you got RAID 6 setup is very unlikely (but not impossible just unlikely, but gets more likely as each disks get larger) but raid 6 good enough for quite a long time even with 16Tab disks
as 2 disk failures at the same time are rare, as raid controllers and nas boxes do a weekly or monthly raid patrol read and data consistently checks (depending on how they are configured) witch normally weed out problems disks before they suddenly fail
it doesn't matter if the raid fails because you should have a back up of the raid itself.
You'd be surprised/depressed how many people think RAID = backups.
Having an array fail is kind of a big deal, even with backups.
RAID1/10/5/6 is about availability of data not safety of data. I.e. minor failures not causing an interruption of service, not protection from loss of data from drive failure, server failure, building burning down floods threat explosions etc etc etc.
+Joseph Lucas: Only if the downtime is costing you a lot of money. Otherwise, it's not a big deal at all. A mild inconvenience at best.
I can confirm that what he is saying is correct. This has just happened to me on Raid 5. Good thing is that I still backup each week, but HE IS ABSOLUTELY CORRECT!!!
This is because you are using consumer devices
@@TheWayOfTheHeart NOPE! Using enterprise drives and Areca Chasis.
are you using raid card too? 🤔
i'm thinking to use some LSI MegaRAID for RAID 5 btw.
People have different storage needs. There isn't a one size fits all solution, and an absolute statement like "RAID 5 is bad, never use it" is just silly even today. RAID 6 _is_ safer for large arrays where more disks means more points of failure, but for smaller arrays RAID 5 is still fine. It may even be preferable because it has better storage efficiency.
Besides, you should be backing up data that's important anyway. RAID is only meant for minimizing down time when a drive dies. If you're ever using it by itself as a substitute for a true backup then you're an idiot.
Whats the "safety limit" of raid5? at which number of drives it makes more sense to got with raid6? honest question
Thank you for this vid. Now I backup my data 'cause of bad sector, tried raid 5 twice and it always have error, will try this raid setup :D
Good source of misinformation. If one drive goes you still have access to the logical disk even if another drive has a read error. Take a backup of the logical drive using Acronis or similar software, replace the defective drives, rebuild the RAID and restore your backup. You wouldn't lose everything as this video states unless you have two full drive failures, but then you should be doing regular backups anyways. RAID is about keeping your data available all the time , not protecting it from loss.
For small ammount of disks (like 4 or 6) usefull space / used space ratio is too small to consider RAID 6, you just need to stick with WD Red, Seagate (not Barracudas!) Ironwolf / Constellation and similar kind of disks and you'll be fine. For larger ammounts, you use ZFS anyway to avoid bit rot. The only really dangerous scenarios i see are some instant power failures and thus you need either RAID controller with battery or UPS.
Good and proper explanation, except i have to wonder about your reasoning why RAID6 is not a futureproof concept (or as drives get bigger).
Unless you get two UREs in the same exact sector in both parity segments, you will always be able to rebuild your lost drive by using the readable sector from the other parity segment and this problem does not scale with drive size.
I love your channel. Its just so awesome how u teach me things in an interessting way! And quick. It fits in every day and im Happy to have u subscribed! Greeting from germany!!!
For the record, raid 5 is still fine for small arrays like an executive workstation for your CEO. His premise here is that you're using a very large array like in a NAS, Backup array, or Data Center
NAS isn't "very large" ... certainly nothing like a data center. It's standard practice for people who do content creation and basic video projects. An 8TB NAS is hardly a huge amount of storage for a UA-camr/Vlogger.
But how do I download more raids?
+Clark Epic You download a copy of Windows and make as many (virtual) RAID's as you like.
+Clark Epic i think you can use ramdisk
you can download it from amd
Unfortunately you can't download extra raids like you can with more rams
+ThioJoeTech Wrong channel to reply from, should have been main channel (would have made more sense) :)
You need to write a letter to the president asking for a download link. Also works for free money.
I lost a primary domain controller once back in 2003. The raid 5 didn't help, neither did the secondary domain controller. It was a huge mess. I had to accept defeat and rebuild everything from scratch in two weeks with the help of a friend. Slept at the office for that time. So now I am a bit skeptical to all redundancy systems :)
I use raid 1 (plus cloud backup). Sure you don't get the storage size advantage of raid 5, but IMO anything worth keeping is worth keeping 3 copies of.
Great video. I'll be using RAID 6 on my new storage!
Many thanks for your effort,
cheers!
I've been using a 12 drive Linux Software RAID5 setup the past 10 years. Gradually growing and replacing drives (from 8x200GB to my current 12x2TB). Although I agree with you that RAID6 would be safer, I've never experienced the problems you're describing. I've had 1 drive failing periodically, and replaced maybe 4-5 drives in total (due to failed/dead drives) during the same time period.
This is data that I am comfortable loosing, although I would prefer not to.
The worst problem I've encountered during this time, is the failing of a controller, where symptoms indicate that multiple drives are lost. But thanks to Software RAID, I'm not bound to a specific hardware disk controller. Which makes it easy to rebuild or even move the entire array to different hardware. So if I encounter the bad sector issue I'll report back when that happens, although I haven't seen a bad sector since the mid 90s
I have yet to see a RAID controller that stopped rebuilding a disk because of ONE read error. So far I only managed to get a corrupted file as a result but the rest was recovered nicely. So the big question is: Is ONE defective file (which is possible, but not guaranteed to happen) worth an extra disk (and a bigger NAS system in case you need more than 4 drives)?
Hi from 2019 - what do you think is the new standard now Joe?
Normally if you create a raid set you also set a spare drive so it can start rebuilding the raid asap. Depending the raid you want/need it will be better one or the other. Also keeping in mind performance of course
Raid 6 doesn't sound like a good option if you're looking to build a four drive array like I am. I'm looking to build a four drive setup using 3, 4 or 5tb drives, but I don't want to loose the storage of two drives, so raid 5 is really the only option. If I was fine with only having two drives worth of space, may as well go raid 10.
Joe. Thanks for the Vid. Question: If you get an URE on a 4 disk RAID 5 during rebuild, is that disk array still readable, assuming a second disk hasn't crashed? i.e can you still copy off the data onto a backup?
this video is super helpful, thank you for making such a clean and straight video!
Tripple partiy will never come. With the increase in disk sizes we are running into the limits of what RAID can do with parity. Rebuilding huge arrays simply takes too long and is no longer worth it. If rebuilding takes over a week it simply takes too long. RAID5 is not a bad choice, depending on your use case. You've taken a very specific use case where the capacity of the array equals the URE to say that you are statistically certain that one of the drives will have a URE. While I did not study statistics in school, I do doubt you can simply state it like that. You have 6 remaining drives, each with 10^14 bits so it wouldn't surprise me that you can say in this case that the actual value would be 6*10^14. And what would you do with the new 14TB drives that got just announced. If they also have a URE of 10^14 you statistically will always have an URE on the drive. Not use it? (welcome to the wonderfull world of statistics).
And to add a few more things:
- The URE is the avg over the lifespan of a drive. It might be possible that you read 1000TB of a drive without a single URE and then all of a sudden you get a multitude of them resulting in the avg of once every 12TB
- The URE is a single bit in a sector. Every sector has a CRC that gets updated on every write (how else would you know about the error?) The URE can even be in the CRC data. Most bit errors happen on write anyway and the system will always read back written data to confirm it is written OK, further diminishing the chance of having bad data due to a bad bit.
- On modern drives with 4k sectors if you write small amounts of information the drive will pad the sector with the same information, as long as it is smaller than 2k, thus making a copy of the same file making it more likely that the information can be recovered.
And lastly, let's not forget: RAID is not a substitute for backups. If you do come across a URE, you should still be able to recover the unrecoverable data from your backups.
awesome video!, but I think you should of also include RAID 10 as well. because it gives you best of both RAID 0 & 1.
I have read that for a raid with multiple hard drives next to each other, you should only use NAS or Enterprise hard drives as they have vibration detection. Because cheap hard disks without vibration detection are actually only intended to be alone in the PC case or individually in an external case.
Could you change raid setting with data enclosed any Time you want, with out any data risk lost? Thanks
I've rebuilt a few dozen 20~30TB RAID 6 arrays without a single rebuild failure. RAID 6 is solid. This has always been with Adaptec controllers (51645, 6805, 71605 etc.)
This is a topic at work as we deal with a lot of data. So the answer is much more complicated. For some more to chew on:
1. When dealing with large amounts of data with many mechanical drives the answer typically comes down to ZFS RAID-Z level 2, which is somewhat similar to RAID except more check sums in more places, CoW, snapshots, etc. Plus flash based cache drives as RAID-Z is not the fastest thing in the world on mechanical drives.
2. For small arrays, a few big things are out there often favoring RAID 5. You can find reliability reports on many drives and avoid the known bad lines of drives. You can look at SMART data and pull a drive as soon as it starts going bad. As this is usually a slow process, most competent admins pull drives long before they actually fail and enterprise drive vendors usually take back drives even if they have one remapped sector. You can use hot swap as often the double fault happens when you power down a system. You can buy drives in a staggered fashion as sometimes sequential serial numbers in your array lead to drives failing close together in time, which is the last thing you want in a RAID. Flash drives if you look around will sometimes have RAIN, which is NAND level RAID. Flash also will often fail into read only mode, giving you a chance to copy over to a new drive, but not always. However flash in a RAID tends to get beat on more both because RAID controllers don't support TRIM and if you are using throwing money into a flash array, it is probably because you beat on the system a lot more than the average bear. However this wear concern can be addressed with higher endurance flash at a higher cost. For example my main home flash array with Crucial MX500 drives should burn out its flash in about 6 years at the rate I am currently grinding away at it, which is about how long my older flash drives on low use systems failed due to old age, however for twice the cost I could get SSDs with 10-100x the endurance and never come close to burning them out before they failed due to old age.
3. With any data storage system, you should not completely rely on the RAID to save you. Someone might delete important files and RAID does not stop file deletion. Someone might try updates that go horribly wrong. A cooling fan may fail and the whole array bakes to death. The computer with RAID array might get stolen or lost to a fire. Or like the other day with a software RAID 5 a power loss caused the array to scramble while the hardware RAID on the same system was fine. I have also seen arrays lost when the power supply in the box (often cheap NAS) fails and fries all of the electronics in the process. I have seen all of the hard drives seize at the same time after many months of running because the drive manufacture used the wrong lubricant, so once the drives stopped spinning, in this case routine maintenance, they never spun again.
100%. A Raid is not a substitute for a backup, no matter how you stripe or parity it. Even raid 60 could have an internal power surge or something and toast all those spinning drives at once... your data NEEDS to be elsewhere, off the raid, if it matters.
Hi. I want to build a Raid 10 with 4 HDDs. But now I ask myself what if my motherboard breaks down at some point ? Can I simply connect the 4 HDDs to a new and more modern MB that can also run Raid 10 and it will continue to run (plug & play) as before ? Or do I have to set up and format Raid 10 and then have to upload the data from a last backup ?
Thank you
Thanks for the education. If I'm using SSD drives and do very little writing (mostly just archives) on a 12TB array (2TB x 6) Should I go for RAID 6 or SHR2? or just go with SHR or RAID 5?
can you make a video showing how to download more cpu cores?
HAHA U GET REDDIT WOOSH
lol
As usual very interesting. I have an external drive with 2 X 2 GB drives in RAID 1 it meets my needs and budget. Perhaps in future you could talk about building an external RAID array.
I've got three 1tb wd blues I wanted to put into raid 5 for my less played stream games so data loss isn't a huge deal, but I don't want to go to raid 6. Should I just do raid 0 or stick with 5 since the likelihood of errors is generally less on smaller drives?
i am planning to setup a NAS for the first time. i know you talking about drive failures. what happen if the raid controller fails? lets just say if i got a qnas enclosure and setup a raid 6. if qnas dies will my whole raid 6's data lost?
I understand RAID 6 has a slower write speed than RAID 5. What about RAID 5 plus a hot-swap drive? That's what I'm doing right now. So if a drive goes bad the hot-swap kicks in and it rebuilding the array automatically. I replace the bad drive with a new one and it becomes my new hot-swap. You get part of the benefit of RAID 6 (although not the exact same) without the performance hits of 6 vs 5.
all that means is less time is required to discover that a drive has died. If you have any errors during a raid 5 rebuild, data is toast. Raid 6 allows the data to be protected during a raid 5 rebuild
I use RAID 5 for all my servers and never had a issue when replacing a dead HDD. Touch Wood.
Touch SSD
I rarely have dead drives to begin with. One or two in the last fifteen years. Just periodically replace them.
Virtualization has made the vast majority of my hardware super easy to replace.
Move guests to different box. Box can be turned off.
@@kevinsky86 How odd you rarely have any dead drives if you periodically replace em. *insert meme*
@@mesaber86 it sounds like periodically is being used to define 1/2 failures, not uncovering hidden secret failures.
Although I could be wrong, only Kevin de Bie really knows how many drives have failed.
"I use RAID 5 for all my servers and never had a issue when replacing a dead HDD."
Famous last words before cold sweats, tons of sleep deprivation, and data loss.... If one of your other drives fails during recovery you're SOL......smh
What do you guys recommend me for a torrent movie library? (Don't judge me, where I live Blu rays are not sold and streaming quality is not on par for OLED TVs) I want to have around 50TBs in RAID5 or 6. As far as I know HGST is the best brand. Is it still the same today? And what NAS Brand?
im about to buy a new NAS, it will have 8 drives that are 4TB each. is RAID 6 still effective enough to have a low chance of data loss if a drive should fail?
excellent content from the Michael Cera of Tech :D
I'm not sure this advice is particularly helpful for a SOHO user. His
implication is that that your RAID array somehow forms part of your
backup strategy (or perhaps even *is* your backup strategy) which in
itself is a poor message to convey. I really can't see too much issue
with RAID 5 + proper backups. How many SOHO environments need 2 levels
of redundancy unless in special situations?
Mark Browell what do you use besides raid to backup your data?
So for a 4bay NAS, would Raid 1+0 be a better option than raid 6?
Just subbed and Thank you for your informative videos.
Ok so I can use RAID 5. Thanks
Wait is this the non troll channel?
Yes
Adolf Hitler Okay, thanks Hitler. You always got my back.
I'm Not Chase I enjoyed this conversation immensely ^^^
Nope, but funny. Subject to Satire, being part of your sense of humour? No but it is like phoning tech support and speaking to two technicians, a day apart. Explain the Lion? Since you need the attention? :) Then go back to Facebook, please. :)
those diagrams were super useful! i had no idea how parity stripes worked even after reading extensively until i had a bit of a look at those pictures.
Fantastic video, very clear. Thank you for explaining.
ZFS is the future of drive arrays.
It's total overkill for desktop systems or small servers.
It-s not. My home build 4 disk NAS, primarily used for archiving is build on ZFS which has one big advantage over RAID5. It is protected against bit-rot.
how is it a big advantage? bit-rot is not undetectable and between smart and filesystem protection you'd have to be incredibly unlucky to end up with a corrupted file - you DO have backups right? and i have never heard of a raid array failing due to bit-rot. but go on, all you zfs ppl just keep saying it over and over, it's not going to make zfs any better you know.
I've had my share of corrupted files on large storage (2TB) devices.
th00ht Every harddisk has already some pretty good ECC algorithm against bitrot, i really think it's most likely a ZFS marketing strategy and not a real problem for the majority of people.
I had a major RAID 5 failure. I lost 15TB worth of data 5 disks. I've now got 3 NAS with a total of 85TB each running RAID 6 and being backed up in realtime to each other. When my RAID 5 did crash and I went to install the replacement drive that is when the shit got real and the whole RAID died. I had to use a RAID recovery software to retrieve my data most of it but not all. Enough to get me out of hot water.
There are two kinds of people on earth people who have lost data and learned from it and other who have yet to learn. Peace from Sydney Australia
+OnTarget100 Dude thanks, you just crystallized my decision between RAID 5 and RAID 1.
OnTarget100 More fool you for not having a backup. I use two RAID 5 arrays, one backs up to the other nightly. The first array also backs up to cloud storage as soon as I write to it. I do this because I don't want to loose data, not because I learned the hard way
If you follow the 3-2-1 rule you shouldn't ever lose data.
Must have been some valuable data lost to over compensate like that. I’ve lost data too, usually when upgrading OSS. By my own mistake of thinking I’ve completely copied data over , then resetting the disks for the new os.
I’ve never lost data due to a disk failure. I also never use cheap disks. Only HGST hard disks and Samsung SSDs. I’ve had hard drives fail but only WS and toshiba.
You’re smart and have the budget to have not only redundancy for disks but also separate enclosures. Very cool.
I’m in the process of replacing all my bad Disha from 8 4TB to 10TB. At 350$ each, it’s damn expensive. But, like you, I have valuable or priceless (old work files I can never get back), so I get it. For sure.
Photos fade, digitals do not ... music leaves popularity thus impossible download, same with films and tv shoes.
What bad devices did you do with?
but with a professional 4 or 8 bay server rigg it should be fine right? For raid 5?
I was setting up raids before you were born but you were clear decisive & I actually learned something about RAID 6 vs 5 from you nice brother !
With power failures (biggest issue for me) I have found that raid 5 with medium size SSDs RAID 5 is fine I always figured raid 50 would be the best before 6 what are your thoughts ?
Hey Joe, lets us consider the following scenario. what is I have 10x8TB array of disks configured with RAID6. So even if two disks fails( Since RAID6 ). I will be having 64TB of data to be scanned for recovering the last drives. So from your URE probability calculation, there is URE happening. Then RAID6 is also not useful. What would you suggest about this... !!!! Thanks in advance
The math isn't quite that simple, because URE probabilities are independent from disk to disk. Reading 64TB across disks that each have a URE rate of 1 in 10^14 does not guarantee that a URE will happen, because a URE not happening on one disk has no bearing on whether or not a URE happens on another disk. It'd be like thinking that rolling a 6-sided die six times guarantees that you will roll a specific number at least once. You'll get _more chances_ to roll a specific number, but failing to roll that number will never make it more likely that you will succeed to roll that number in the future.
All of that said, adding more drives to an array will always statistically increase the likelihood of a failure by a little bit, unless you're using RAID 1 where more drives always means more fault tolerance. And 10 drives is quite large for a RAID 6 array. You should definitely have regularly scheduled backups if you don't already. Off-site is best if it's feasible, but any backup at all is better than nothing.
i can no longer take this guy seriously i just assume its bullish
why are so many disabled people in the youtube comments
who can??!
ThioJoe is fake. ThioJoeTech is real
Right, but who's making the distinction...
WOAH I am literally just discovering this fact!!! Why isn't this video fake????
Sir, i f i may ask, if you copy or transfer files while Raid 5 is rebuilding, will it affect the system?
+ThioJoeTech
Thanks for the informative video :)
I'm confused with RAID 1.
I want mirroring and also better read speed (write speed is not that important).
Is RAID 1 always guaranteed to have double read speed?
How do we configure RAID 1 to guarantee double read speed?
That is why we use Raid 5 and 6, but need to have a Backup. I see your point. But I cannot see your conclusion. RAID and backup are not the same thing
Agree, no need for 3 parity drives. you need RAID 5 if you use something like 3 to 5 disks, or Raid 6 if you are using 5 to 32 disks AND your Backup.
Christoph Esch then what do you use as a backup?
@@jordanbanko5347 I'd go Raid 5 and look into Crashplan. for $10 a month it backs up what folders you ask it to with unlimited space. Easy to restore from and keeps file versions so you can go back to a file version from last week if you want to.
What's wrong with raid one 🤷🏻♂️
Usable space cost
thanks a lot mate. I did not know what the difference was and what parity was.
:) kudos to the video. very nice.
Informative. Thank you.
Is RAID 10 faster than RAID 5 and 6?
yes raid 10 is actually 2 raid 1 in raid 0... it's actually 1+0
HW RAID10 -> No.
Linux software RAID10 -> Yes and No.
RAID5 achieves 95% bare hardware read speed. RAID6 a fraction of that.
RAID5 and RAID6 takes a tremendous hit on write speed. Much more so on RAID6.
RAID6 is not worth it. RAID5 for upto say 6 drives, further than that RAID50. Much higher performance, same level of redundancy.
RAID != Backup.
That depends on your RAID controller card. My Areca cards that I use can run RAID 6 at nearly the same speed as they can run RAID 5. I've tested it, and with 8 drives it can read/write at ~750Mb/s in RAID 6. Cheap RAID controllers, however, won't be able to do that. The drives independently will only do around 130Mb/s, so it's losing very little to overhead if you figure that 6 drives (not counting parity) X 130 = 780Mb/s (max hardware speed) and I'm getting 750.
130 Mb/s sounds quite slow. My drives do around 120 MiB/s Thats around 8.3 times as fast. I would check your drives.
well you can just use RAID 10, simple, efective and reliable.. i know you have less space.... but its more secure for critical data ;)
And faster too
RAID10 doesn't use parity so it's technically less secure then RAID6 (RAID10 is for some redundancy + speed)
@@leexgx are you serious? Raid 10 is stripping+ mirroring so it is more secure and more fast, of course at expense of more disks and less space, it what's it is used in critical infrastrutures... trust me ... it's my work loool
There is something I don't understand about this: Assume you were to use 12TB HDDs with one URE at every 10^14 bits, your probability of an URE would approach 100% during a RAID-5 rebuild. However it would also approach 100% during a RAID-6 rebuild, because every HDD itself would likely have at least one URE. So no matter how many parity HDDs you use, you are always hosed and thus RAID-6 would give you anything over RAID-5.
What am I missing here?
The parity file is duplicated. That's the critical part.
I just bought QNAP TVS-472XT 4 bay Thunderbolt 3 4GB (for Editing FCPX on my MacBook Pro) with four (4) Samsung 1TB 860 EVO SATA SSDs... can I configure them more than RAID 0 like RAID 5 or 6 or 10?!?
eh... Delving into a grey area here... For most home users, Raid 5 is just fine, even if the array is larger than 12 TB. Lets say worse case scenario, and you loose a disk, and error correction failed to detect the URE, and during re-build the URE is discovered hence fails to rebuild. There is a another option that Joe fails to mention. Actually, there's 4 or 5 other options, but I'm not going to get into them right now. The easiest thing to do if your Raid array fails to rebuild due to a URE, is to copy off the data. Yes, that means you will likely have to buy a couple of huge drives, and stand up temporary storage, which is a pain, but you will be able to copy the data off.
Folks... Don't freak out over URE. Yes, the risk is there, but if you're buying quality drives the chances for URE is even smaller than what Joe stated. If you're Enterprise or your data is that important I figure you're going to go with 6 or Raid 10 anyway..... So, my point is just geared to the folks at home that may not have critical data and don't want to loose that additional disk space by going to Raid 6.
What are the other options he fails to mention? I’m trying to find a good safe way to back up my data and thought raid was the answer. After reading the comments, however, I’m seeing that it is not.
What you just said there is no simple task for home user, where if you had been using RAID6 to begin with a URE won't be a problem (or second disk fail) just replace the disk and let it rebuild
It be nice if Synology used 5 bay NAS by default (no 4 bay nas's) from 2 bay ones to cover the extra disk needed for RAID6/SHR2
Also UREs can just be skipped over. Unlikely you will get more than a handful. So worst case you rebuild the array with 5 files missing. Obviously if you have a backup then its fine also. Easily encounter same problem with RAID 6 with UREs. So video isn't exactly telling the whole truth. Rebuilds can be hard on drives so possible it can lead to other disk failures during it, unlikely but can happen. This video hides the full facts and tells only a narrative that he wants you to hear
trust issues.
Thanks, good information. I guess my only question is: How do you archive this much data when you do not need it on a local drive? Such as old youtube videos. I want to keep them, but not locally when I use up my 20-30TB NAS.
OK, what I don't get with parity 5 is if you lose drive 0 (2:48) what happens to 'Dp'? is that copy of the parity lost forever or is it actually copied onto one of the other drives as well?
Just use 6 drives in raid 0 for maximum craziness :) :) :), I live dangerously...
Wait wait, is this his joke channel or his real one lol?
What is your opinion using SHR-1 with BTRFS with four (4) 6TB NAS (Iron Wolf) disk drives in a Synology DS918+?
I am using RAID 5 system with 6x4T hardware. So I only got 20T of free space to begin with. After a few months(that is right , Just a couple of months), now there is only 500 or so free space. I am in taking this issue of data security very seriously that’s why I invest a lot of money to get a Promise Pegasus 2 for my iMac. Also, folder 30 years of using hard drive, I have encountered numerous case of URE. So what do you point out is very great for data reliability. However, since I already have almost 20 TB of data, what should I do to convert the whole system to RAID 6? I think definitely I need to purchase new hardware. I just don’t know if I need a whole new system gorgeous replace all my hard drive to a bigger ones. Also how do you someone convert a 24 TB RAID 5 to RAID 6 with minimum efforts and money? Thanks
Never use RAID5. If you are an IT tech chances are you will:
1. Get a call at some point in time to go deal with a RAID5 disaster
2. Never get a call about a RAID1 or RAID10 disaster
I always use Raid 10. Raid 5 rebuild is complicated. I've seen where it never finishes rebuilding. Raid 10 rebuilds usually finish in a couple hours. I know R10 is pricey, but it is 100% worth it. FYI, Setup some monitoring of your servers as well so you don't have multiple hard drive failures before noticing.
RAIDs? Isn't that something you do in video games?
lmao
+ThioJoe I thought RAIDs was an std
Yes u raid in minecraft factions
I THOUGHT RAID WAS A BUG REPELLANT!!
Actually RAIDS is the same thing as aids, but you get it from rape, that's what the "R" is for. Don't trust this guy's channel, ThioJoe will make a video explaining it.
Thanks, lot's of people say don't use RAID5, you're the first to explain why!
Hi Joe. Thanks for the interesting video, but please tell what filesystem cannot survive 1 bit error in 12TB aproximately?
The filesystem will likely survive, but an individual file will become corrupt and will need to be restored from backup. It's also possible that the corrupt sector will be in free space, in which case nothing bad will happen. The biggest concern is that when rebuilding the array, all of the other drives are under a lot of stress, so it's likely that a second drive will fail during the rebuild and all data will need to be restored from backup.
Do you ever blink?
i thought this guy is a troll what happen to him?
Mustafa Hashem he does troll.
Great Explanation of RAID. Clear, Concise, Interesting.
not sure if you are still monitoring this feed, but any updates on RAID systems and/or tech on this subject. Dusting off an old project and want to finish it this winter. Thanks
N A S
I know this video is a million years old in UA-cam years but there is one downside to raid 6, but you have to have minimum 4 drives to do that.
Shout out to anyone watching this video in 2020
Hello ThioJoeTech,
Thanks for making this video. I'm having a hard time finding URE: Unrecoverable Read Error in specifications for hard drives on websites such as Newegg and Amazon. Any way you can clarify how to find this information? Also, do you know how reliable Windows 10 is for setting up a RAID 6? To start out, I just want to have 3 12 TB drives in a NAS Enclosure. I'm looking at the Seagate IronWolfPro 7200 RPM drives. And I'll just back up to a BackBlaze account. I'm currently using 3 Google Drive accounts to store my archived projects and it's getting costly and a pain to upload. Sorry if this comment is long, it's a bit difficult finding the correct information and way to go about this. It is an investment, and I want to make sure I do it the best way possible. If you could let me know I would appreciate it. Thank you.
- Paul Harwood
what's the difference between raid 1 and raid 6? isn't raid 1 more reliable then raid 6?
and one more thing, Do you recommend using
2 x 3tb hdd in raid 1 or 6 PLUS using a 1tb ssd + Samsung NVMe 950 ssd for me new system?
idk just please tell me if this combination is good or not and what raid should i set the drives to.. THANKS FOR READING ☺
just read this comment that was posted 2 years ago. would be curious on what system did you choose to build your system. I woud personally recomment raid 1 on ssd . example 512 + 512 ssds on raid 1. would be a better bet.
your out to lunch!!! you don't know what your talking about!
hey guys i have heard that RAID 5 can tolerate multiple disk failures if we do something to parity group, is it possible this question is haunting me can any one explain thanks...
Is all the data really lost though, with the read error? Wouldn't you have like one or two corrupt files, but still get the rest back?
Raid 6 on 4 Drives cuts drives in half like Raid 1, right? so Raid 5 then… if Drives are more then four (>4) then Raid 6... did I understand that correct?
I know this was a few years ago, but correct me if I'm wrong. This really depends on the number of drives that you have? Right? What if I'm only using 4 drives instead of 7? This will certainly change my chances a bit. I agree with what's said, but the scenario certainly depends on a few factors.
What do you think of the Drobo drives? I have a Drobo and I was advised to ditch it. I was also advised to be wary of software controlled RAIDs. What do you think? Gerry in Calgary
so, looks like Raid 6 and 10 are the same as capacity still lost half. But Raid 10 good for Higher server like company. for home use like me, I needed backup data of family pictures and video. should I choose Raid 5 or 6 ? If one of the hard drive fail I still can recover the data. Budget is low. If I go with Raid 5. and the storage space have 5, can I later put in 2 more? or this would not work with raid 5 ? Thanks
Now, If I setup raid 6, use 4 hard drive. 5th space is a waste right? so just buy 4 rack/case only? how about I saw some guy on youtube turn his old mid case into a Raid storage? I have an old desktop that ran on window xp or so. Can I use the mother board and run it Raid 6? or The Raid motherboard is different? I understand that from my labtop can not connect to the 2nd pc to access data. is there a video or instruction about this? Thanks a lot guys...
I have an Asustor 2404TE, one of the very early line of Ausstor NAS that had an early Atom CPU. It had a 4x3TB setup (WD Reds) at RAID-5. The 3pin cable conencted to it wore and there was a power surge that caused damage to one of my drives. Eventually it was reporting a lot of rear errors on the drive, which started small then began to build up. I don't know if they were UREs, but there were many. The system rebuilt the raid twice with no issues during this time. I then replaced the faulty drive (4) with another, the RAID rebuilt. That was back in 2017 and there's been no issues since, though the other drives will need replacing soon.
So while there's a risk of URE it's incredibly low, and your drive will likely start reporting errors before this stage, at which point you should replace the faulty drive ASAP. Raid 6 has more redundancy, but I don't think it makes up for the exorbitant cost to get the same level of storage. I am planning to get a new ASUSTOR with a 4x4TB setup. Obviously my rebuild risk is higher, but I don't think it's worth me forking out another $400ish just for the same level of storage.
where do you buy a RAID 6 system? So 8 years later a RAID 5 is available anywhere. It can do RAID 1, 5, 10 in the same box but never 6. Have times changed?
I want to raid 3 x 4TB hard drives for a total of 12TB. Will RAID 6 work, and is it the safest?
Four drive minimum for raid 6
I wanto create a RAID 6 array with 5 disks in Windows 10, but HOW do I do that?
As I understand it windows 10 doesn't do RAID 6, and so far I haven't found any reasonably user friendly software that does.....PLEASE HELP :-)
Can you do a video on how to setup usb drives as raids, I know it is useless but I need it for a demo and it just don't works on windows. Or is this simply not possible under windows?
Greets J.
fantastic video....very helpful...thanks for sharing
I bought 4 drives 14 TB WD Ultrastart and QNAP what raid can I apply to have more space and not experience UTE .Thank you
So, the TLDR is you realistically have options of RAID 5 and RAID 10, and I would recommend RAID 10.
The longer version. RAID 5 means you stripe data across 3 discs and use the 4th for parity. This will give you a total effective storage of 42 Tb. In RAID 10 you stripe the data across 2 discs and use the other 2 to maintain a completely independent copy of the first 2. So in RAID 10 you won't have as much total storage, only 28 Tb, since you use 2 discs for redundancy instead of 1. Raid 5 will also get you more read/write performance from your array. Since the data is striped across 3 discs instead of 2, you could notionally get up to 50% better speed from the RAID 5 setup compared to RAID 10, although that will probably be limited by RAID overhead and network bottlenecks.
Now, as to UREs. WD Ultrastart drives are rated for an average of 1 URE per 125 Tb of data read (10^15 bits). That means during a rebuild (if we assume the array is basically full) each individual disc has about a 1 in 18 chance of encountering a URE (14 Tb/125 Tb x 0.5). The chances that 3 drives manage to rebuild without a URE are then about 5 in 6. So, if you have a drive failure with RAID 5, there is a 1 in 6 chance that the array fails to rebuild due to a URE and you suffer data loss. In RAID 10, you maintain a completely independent copy of the data at all times. So your data is protected without having to rebuild the array. In my opinion, a 1 in 6 chance of data loss if I have to rebuild the array is too high, its worth sacrificing the extra 14 Tb of capacity for better fault tolerance.
Great video. First time in your channel.
very good explanation, though I think mentioning the re-build time delta of RAID5 and RAID6 is worth it. Plus, RAID6 is required much more sophisticated RAID controller, so pure software RAID controller very likely isnt going to cut it.