FreeNAS Box ECC memory upgrade
Вставка
- Опубліковано 16 вер 2024
- After my initial FreeNAS build, many of you pointed out an important shortcoming of my system. I chose to use standard memory and didn't realize how sensitive ZFS is to memory errors. It was highly suggested I upgrade to ECC to ensure a reliable system. So, I took your advice and revisited my build with some upgrades including a new server board from ASRock and ECC memory.
You can find the initial build video here:
• Build a FreeNAS box - ...
The motherboard used can be found here:
www.newegg.com/...
I see you made a mistake circling the Items remaining MB rather than the ~11MB/s transfer speed. I'd suggest for a Gigabit network you need to have a look at what's going on there as you should be pulling 80MB/s+ on writes. Other than that nice video.
I still have no idea what is freenas or what it does lol, but its always awesome to watch you assemble a pc :)
10MB/s is not fast at all on a Gigabit Ethernet, it should be something in the range of 80-100MB/s...
+David Pérez i agree it should be much much faster... it should be around the 80MB/s range... he must have something that is throttling it....
+Nifty Mist well in case then he is capping out his switch... and should upgrade
Using a freenas system. 8 disk raidz2. Easy stable speed of about 100MB/s maintained while writing to the server. Not sure what he considers beefy but I'm at times writing 2 tb in one shot. For most of the home users such as myself I think 1 gigabit lan is going to be sufficient..
@@bhoot
Yeah, this guy has an old or defective network card/switch/router/or cable somewhere in his system creating the bottleneck.
He really needs to find the choke point since he's only getting 1/10 the available throughput speed, assuming the rest of his gear is "modern".
100 MB speeds over common commodity hardware is normal with FreeNAS. At least it is with all three of my FreeNAS rigs.
I love your choice of time-lapse music @ 12:12
Interesting, i have run my Freenas system for 5 years running continuously (except for when on vacation) without a problem without using ECC memory. There has not been one error that I am aware of. I'm storing pictures, movies, music on it for the whole household. I do back up off-site and onto another 4TB drive in another computer to be safe though.
+vnitto Thanks for that! What RAM do you use anyway?
18:10 1 celsius from tjmax, haha. Great video as always.
On the FreeNAS website, they have in big letters that you should use ECC RAM.
However, one of the ZFS developers said "There’s nothing special about ZFS that requires/encourages the use of ECC RAM more so than any other filesystem."
and the FreeNAS forums are full of people with dead zpools from non ECC memory use.
I ALMOST made the same mistake when building my Freenas box earlier this year by not installing ecc memory. Wise choice going back before a potential disaster.
I recently been looking into nas storage thanks for these videos keep up the good work!
It may go without saying but I figured I would mention that you should also configure a static DHCP lease on your router for the NAS.
How is 10MB/s a nice speed? My single WD Red hits 116MB/s over gigabit so your CPU/RAM is either insufficient or your FreeNAS configuration is off!
slow filetransfer. Sure you're on a gigabit network?
i see the hd home run on your network. Have you tried to install mythtv on a jail inside of freenas? if its on your things to try plz make a video of it.getting ready to try and tweak the settings and get it going on mine and would love some help in getting it going
The i3 CPU that you were using before wasn't compatible with ECC memory? Would have imagined that would be better than the Pentium. I had no idea that FreeNAS didn't like non-EEC memory. To be honest I don't know a whole lot about servers.
I don't like how the HDDs are vertical. It's my preference that they're horizontal; they just seem to last a lot longer like that, I find. Of course, just my preference haha.
I don't like how they trust me to take out the black CPU socket cover...I preferred it when they popped out. It's something I've seen on quite a few new motherboards. I've built quite a few systems in the last couple of months. Not for myself though, just at school I've been building them for the younger kids, and building a few for friends and family.
It's really hard to get a scale of the motherboard, until you put the heatsink in. Wow, so tiny and adorable! Haha
Cable management...Personally I don't like it with this case. Then again, with all the HDDs there isn't much to do. I guess with a case this small there isn't the room?
Talking about fans...The cheap POS Chinese one that I have on my gaming PC is slowly failing. Keeps making weird noises, and grinding (kinda like its rubbing on a cable). I checked, and as a I thought there aren't any cables there. Ah well, it's still working fine, so I'll use it until it stops spinning!
You're not helping my temptation to setup a FreeNAS box myself! As I said on Twitter, it's the price that puts me off and I don't really need a lot of storage. It would just be nice. Added redundancies are always good to have. :-)
Hi mate awesome set-up. This is the perfect set-up for me. Any chance of putting up a new list of all the products you have used? maybe a link? Thanks in advance.
Is it just me or does this fellow look like Allan from Two and a half men. Don't get me wrong I love the guy :DD. I've watched a couple of your videos and I will say I am a fan. Subscribed : P
Good video. If you already have freenas setup with 4 hard drives. Now if i want to upgrade the motherboard, memory and RAM. shall I need to run everything again. Like setting up from scratch for configuration.
What kind of performance can we expect from a set up like this. Can we build something like this to serve large 4k R3D and Black Magic camera footage to several computers? Any info will be greatly appreciated.
Yeah, you don't actually need ECC RAM for ZFS. The mighty “scrub of death” is bullshit.
There are basically 2 types of RAM errors you can get:
1. Random bit flip
A single random bit in RAM is flipped. Quite rare but it does happen.
Result: An integrity check might fail (even though the data is fine) and ZFS will mark a block of data as damaged and attempt to repair it. The repair will only take place if the redundant block passes the check, so the data is fine.
Worst case: Correct data will be (needlessly) overwritten with correct data.
2. Fixed bits
RAMs sometimes break in a way where certain bits will always return the same value regardless of what's been written to them.
Result: Every integrity check which uses this part of RAM will fail. If both a block being checked and any redundant block which could be used to repair it are affected, the block will be marked as lost and the whole volume as unrecoverable, but no correct data will be overwritten because there was “no correct data” to overwrite it with. You'll need to replace the broken RAM and recheck the file system.
Worst case: Basically a denial of service until you replace the RAM.
And then there's the fearsome “scrub of death” where your RAM magically becomes a first order SHA256/fletcher2/fletcher4 collision generator which garbles up any data read from the disk but only in such way that half the time the checksum is correct, so every correct block of data will be marked as damaged and then, when the redundant block is read, it will be damaged in RAM, but it will pass the integrity check and the correct data will be overwritten with garbage. Repeat until all data is gone.
Worst case: You lose all data, but you now have a first order SHA256/fletcher2/fletcher4 collision generator and any intelligence agency in the world will give you millions (if not billions) for it, which should help ease your pain. (Hint: In case it's not obvious enough, this will never happen.)
+hellterminator The issue is that ZFS will use the data in RAM, not the parity if neither checks out with RAM. It always assumes the RAM is correct. This will introduce bitrot into the array. Much like you get in RAID5 setups. But in RAID5 the controller just guesses at what drive has the right data on it. 50/50 shot of borking the data.
So, the whole array may not go down, but chunks of data can be lost with a flipped bit in memory. It can and has happened. You see it crop up on the FreeNAS forums from time to time when people come in asking about corrupted data and in many cases it boils down to non ECC memory. You wont even know it happened till the day you try to read the data.
Chad Bremer I was talking about scrubs because scrubs are the most common argument for ECC RAM. When writing data to the drive for the first, yes, if it gets corrupted in the RAM, it will be corrupted on the drive. But the same applies to literally every other file system and you don't see anyone saying that ECC memory is _necessary_ for NTFS or FAT32.
Sure, it's better (it's always better), but far from necessary.
***** it will do the same thing in a scrub. ZFS trusts data in RAM always.
Chad Bremer And how will it get the data into RAM during a scrub? It will load it from one of the hard drives. And what does ZFS do when loading data from a hard drive? It verifies the checksum.
For this to be a problem, the RAM would have to mess up a block, so the integrity check fails, load a redundant block correctly, so it passes the check and then immediately mess it up before it is used to replace the original “faulty” block. And even then, the original block is damaged (which will be discovered during the next integrity check) but the redundant block which was used to “recover” it is still untouched.
There is a a cached hash of changed data about the drives in RAM. That is the guide for ZFS. If a bit flips in that hash, you are in bad shape. It is always double checking against the hash. This hash is the checksum for the data. And that checksum tree is rooted in RAM for the whole system. There is a checksum at every layer as you move and each layer has it's checksum stored in the pointer from the previous layer. That is how ZFS self validates, unlike RAID. So, if data is bad on one drive or at some point gets corrupted as it moves up the chain a checksum from the last time the data was accessed with mismatch. That checksum is compared to all copies of the data. The one that does not match is then replaced. So, if you get a mismatch, and neither copy of the data in storage matches the checksum that cam from memory, both are considered wrong and the data is corrupted. The linchpin for the entire ZFS files system is that checksum hash in memory. If it goes bad in part or whole data on the drives becomes invalid because it does not match. It does not matter if you are reading or scrubbing either. Both will go through the same checksum list to verify integrity. All the scrub is, is reading all the data in a volume to check against the checksum rather then just what ever is being pulled from the drives. ZFS always will trust the root checksum in memory over all else. Every time. That is how it is designed and that is why you want/need ECC.
Chris, I think it's strange that you didn't test the memory for a week before installing FreeNAS. If you're willing to spend the money for peace of mind (because memory errors are maybe 1 bit per TB per year unless you live near a nuclear power plant), aren't you willing to spend the time to make sure the memory is actually good?
Did you fix the 100mbit issue as your copy was limited to 11mb?
Thanks for this, Chris.
I search and search through the internet for a clear answer, but I have yet to have a answer, if I am building a nas, is it better to have a i3 or xeon? I want to do 2-3 streams in 1080p. Eventually, be able to stream 4k contents. I am just not very sure which path to take.
If you have the budget for it get the xeon or an i7. I say this because an i3 will not really be able to handle multiple streams at 1080 or 4k. check www.cpubenchmark.net/cpu_list.php
That wasn't even a fast file transfer that file was 100megs big and you only copied at 11mb/s thats pathetic for a gigabit lan surlely? I'm sure you were being limited to 100Mb LAN you should check to see if something is incorrectly configured.
The premise of this video is utter horse crap. You do not *need* to use ECC RAM. The reason people freak out about it because they either don't understand the user case or the way FreeNAS / ZFS works.
If an error occurs in RAM, then FreeNAS will happily write that corrupted data to disk the same way any other computer will regardless of the file system in use. ZFS is an extremely powerful file system geared toward protecting your data at every step, but it can't protect against RAM errors. ECC RAM will automatically correct errors that occur within the RAM modules thereby closing that vulnerability. The FreeNAS / ZFS / ECC RAM combination are for users who are 'paranoid' about the integrity and high-availability of their data. If your not the kind of user to shuffle large amounts of data about (like a company would) you will quite likely never see a problem. ZFS mostly protects against disk errors and FreeNAS will run quite happily without ECC RAM. ZFS doesn't come with overheads though, so you'd have to wonder why anyone would go to the trouble of using ZFS and not use ECC RAM.
10MB/s? my 12year old notebook at 100mb nic is doing that. my linux nas with jumbo frame can go up to 115MB/s with a 10€ nic.
How much power does the system draw? Isn't it quitte expensive to run it 24/7?
ECC for ZFS... I approve...
Sorry but this is wrong. You don't *need* ECC RAM to use ZFS. Somewhere along the line some people have conflated "need ECC RAM and ZFS for extra data protection" with "need ECC RAM with ZFS for extra data protection". The point is to plug all the weak points.
11:57 any 1 see one dimm side not fully push down, or im blind?
Did you forget to apply your CPU TIM?
why doo people use those stock coolers
Servers look cool XD
the Intro music sounds so 90s
How can I know my Motherboard support ECC memory.
+Rolly C Check motherboard website.
+Rolly C You will need a server/workstation chipset. So, no Z, H, or B series on Intel. Basically no mainstream hardware. Most all Intel CPU's support ECC though. My server has a Supermicro board with the C226 chipset. The CPU is a little Pentium G 3260. You can always look up the Intel Ark page for any CPU or chipset to see ECC support. Just google said chipset or CPU and click the Intel Ark link. Usually one of the first couple links.
I have not used AMD with ECC so I have not researched it at all. Intel is so much more power efficient and more powerful per watt at the same time that I don't even consider AMD in a machine that will run 24/7.
Hey whats the song name at 4:20
hail hydra
Gigabit network?
Song name plz at 420 sorry for last comebt this app is buggy mxtube on ios
Crimson Fly. Its a Free Music Track supplied by UA-cam under the "Create" tab in Creator studio
nks i been trying to find it all this time
thankxx
i wish installing a teamspeak server on FreeNAS was possible
It is possible. forums.freenas.org/index.php?threads/how-to-install-teamspeak-3-server-jail.16900/
I'm also running Murmur and Ventrilo.
krowven i got it to work last week, it works perfectly fine :D
What is freenas?
CommentKid3000 www.freenas.org/
norge696 thanks
It's something cool that absolutely no household needs for any reason. A simple windows PC with file sharing and a backup external drive will accomplish just fine without all of the headaches involved.
That being said, if you like to tinker, have at it.
Going with 8GB of ram is NOT a good choice, ZFS requires 1GB of ram for every 1Tb of space.
+Gino Jansen This is a *very* common misconception about ZFS. The amount of memory you should use in a ZFS box should be the size of your usual memory footprint + the size of how much data you want RAM cached + how much free memory space you would like as a buffer.
You run into egregious memory requirements in ZFS (such as 1GB per 1TB) when deduplication is turned on, but dedup is such a corner case that it is completely inadvisable except to those who know what they are doing and are absolutely sure they require it, as it incurs a major performance penalty along with the inflated memory requirements (the checksum table is stored in memory, thus expanding the memory requirements).
It is completely valid to use ZFS on a low RAM system; you will just miss out on a lot of the ARC benefits.
Source: I am a system administrator who has set up and maintains several servers using ZFS.
cannot seem to find a mobo that's cheap, lots of sata, micro or mini itx, on board video and hopefullly make me a sandwich when i need it! ohh and obvs its for a freenas media with low power consumption :p suggestions anyone? :o
supermicro and Asrock have server boards for about $130 on newegg.