AR900I All NVME NAS Build - Alpine Linux and ZFS
Вставка
- Опубліковано 2 чер 2024
- In this video I assemble and configure an ALL NVME NAS running Alpine linux on ZFS using ZFS Boot Menu in a RAIDZ2 Configuration.
Documentation: github.com/letsRTFM/All-NVME-NAS
Power Supply: HDPLEX 500W GaN hdplex.com/hdplex-500w-gan-ai...
Motherboard: AR900i store.minisforum.com/products...
CPU: Intel Core i9-13900HX 5.2 GHz - 24 Core 32 Threads browser.geekbench.com/v6/cpu/...
*Built into motherboard
Affiliate Links:
RAM: Crucial 5600 96GB Kit amzn.to/49RVpaD
Case: DAN A4 - H20 amzn.to/49JtkTj
Storage: 6 x Crucial P3 Plus 4TB amzn.to/3Uc0iVs
Cooling:
- 1 x Noctua NF-A12x15 amzn.to/3uNb0Ju
- 2 x Noctua NF-P12 redux amzn.to/3w0QUMA
Screws:
- M2.5 Screws for Minisforum CPU Fan Bracket amzn.to/4djLXPE *Affiliate Link
- (Optional) Misc Assorted PC Screws amzn.to/4b4fPxQ
00:00 - Intro
00:20 - Parts List
00:43 - Performance
00:57 - Installing I/O Shield
01:30 - Installing Rear NVME
01:54 - Installing Front NVME
02:39 - Installing CPU FAN
04:14 - Installing RAM
04:24 - Installing PCIE NVME
05:10 - Power Supply Overview
05:35 - Swapping Power Plug
06:10 - Installing Power Supply
06:15 - Installing 10Gbps NIC
06:33 - Attaching 2x 120mm Intake Fans
07:35 - Installing Motherboard
08:32 - Installing 2x 120mm Intake Fans
11:20 - Installing Filters
12:10 - First Power On
12:20 - 6 Drive RAIDZ2
13:15 - ZFSBootMenu
14:00 - Issues with ZFSBootMenu Guides
14:15 - AR900i BIOS Memory Configuration
15:10 - Setting up Alpine Linux Live Environment
15:52 - Wiping Disks
16:10 - Creating Partitions
17:35 - Creating ZFS Pools
19:33 - Installing Alpine Linux onto ZFS Pool
21:42 - Adding Features to Alpine Linux Boot Config (NVME)
22:12 - Installing ZFSBootMenu onto EFI
24:00 - First Boot
24:30 - Scheduled Maintenance
25:25 - Network Access Setup
28:15 - Windows File Copy with 2.5Gbps Connection
28:56 - Linux File Copy with 10Gbps Connection
29:15 - Why All NVME?
29:25 - What happened to the BD790i?
Music from: CreatorMix.com
creatormix.com/album/mellow-t...
Song: Staying In
Music from #Uppbeat (free for Creators!):
uppbeat.io/t/prigida/moonshine
License code: OH2KU1QGFCZCZMM0
Music from #Uppbeat (free for Creators!):
uppbeat.io/t/aavirall/balance
License code: CKA4M2YHSZFBAPW4 - Наука та технологія
Very cool build that gives me some ideas for my NAS project.
Nice use for that Synology card, not cheap but they do the trick for a board with no bifurcation. My only concern would be the long term endurance of the P3 drives but to be fair for the price not that hard to grab one or 2 as spares.
Didn't know about ZFSbootmenu, I'll have to check that out.
interesting channel. post more videos
Great video; I've got a similar all-flash ITX NAS running TrueNAS SCALE (10700k w/ 4x Gen4 NVME's running at PCIe Gen3... 2x P3 Plus 4tb's and 2x Lexar NM790's 4tb's; and a 10gb ConnectX-3 NIC). I've been having some weird issues in relation to transfer rates.... basically, my desktop (13900k) w/ 10G (MMF) is able to copy from my desktop to the NAS at pretty much full speed (700-900 mb/s, NOT using jumbo frames), but copying from the NAS to my desktop appears to cap out at around 270-280mb/s (even for large sequential files)...
Now, it's not drive related on the desktop side - I'm running SN850X's, and also a P5 Plus, so throughput isn't an issue with things like sequential writes. The only thing I can think of is in relation to running the P3 Plus's and NM790's at Gen3 speeds... I'm starting to think the P3 Plus's may be the bottleneck with this reduced bandwidth limitation, as the NM790's are much faster drives... and you're basically limited by your slowest drive? Still seems a bit on the slow side, given the sequential writes. My plan is to temporarily borrow the wife's system (11900) and run the P3's at Gen4 speeds, 1 NM790 at Gen4, and the final NM790 at Gen3, to see if my transfer speeds increase... otherwise I'm at a loss for why the transfer rates aren't great.
Great video, and perfect timing. I am right now trying out a TrueNAS Scale VM under Proxmox (might also try out Unraid) on a BD790i with a LSI 9240-8i (with 4 older drives). Added 96 GB of Crucial memory. I did have to reset the BIOS once, but that is about it. I got a Corsair SF750 PSU and it's idle right under 100W, I would hope this to be lower. My MS-01 is idle around 18W (measured from the ac plug with a Shelly S).
Whoa that's a pretty high idle, on my bd790i I'm seeing around 30W once booted and 60W when booting. Under load I'm seeing 140W though.
I didn't find the performance governers made a big difference but that seems a bit high.
My dual Nvidia 3090 bd790i build idles around 80W but each 3090 consumes around 30W so the main board is only 20W (with the Corsair 1000w SFF-L measured from the wall)
@@LetsRTFM Thanks for the feedback, I had to investigate what was going on. Once unplugged my HDDs (that I assume was not sleeping) the idle went down to about 45W. Still higher than your 30W, but from what I can see the PCI card I use for the SATA drives is consuming about 13W. So that makes sense.
Are you by chance using the latest version of TrueNAS? The current release is Cobia but I've heard that people testing the Dragonfish version are having issues with much higher idle power consumption. I am pretty sure it is a defect that will get fixed soon.
@@iamamish Yes, that might be it also. I was running 24.04.0 (Dragonfish), interesting. Thanks for that feedback. :)
Nice video, what I can't figure out is how you could install the ssd-board and the 10 GbE Ethernet board, since tbere's only one availablecslot on the ar900i mainboard?! - Did you use a splitter or how did you do that? - Thanks.
It's actually a card from synology which has PCIE bifurcation via a PLX Chip (Works on any motherboard, even without bifurcation support) built in.
amzn.to/44BgW5X *Affiliate Link
It runs 2 NVME and the 10gbps connection off a single 8x PCIE and it's low profile too.
It's gen 3 PCIE, which means it's capable of about 8GB/s of data transfer so it shouldn't bottleneck on these drives
@@LetsRTFM Thanks, now I understand. :)
Bummer about those P3 series drives is they have terrible endurance. 200 drive writes isn't much these days. I just have some of the regular P3 4TB drives I am second guessing myself on because of the low endurance. Granted I only write 100-200GB a day tops. I'm just spoiled with my slower Intel DC drives than can do 1400 writes.
Question regarding the motherboard. Does the BIOS allow you to split or bifurcate the x16 slot to 2 x8 (or x8/x8)? If so would be quite interesting to do and add additional drives or a better/faster NIC
The AR900i does not right now, the bd790i does, but you give up the additional 2 nvme on the back
No L2ARC, swap, or anything except the pools and boot partition?
The CPU supports ECC evidently...question is, does the MB? Their product listing is terrible and I don't see a manual on their site.
On the support page it says no ecc support, here's the manual: www.minisforum.com/new/support?lang=en#/support/page/download/99
We all have our preferences. I personally would have opted for a 2U case. Couldn't you utilize the WiFi slot with a 'Key to 2-port sata' adapter? Connect some low GB SSDs to the sata adapter for a mirrored boot?
As you said, we all have our preferences, I'm moving away from Rack mounted solutions (I have about 4U right now)
I'd prefer something even smaller like the COOJ MQ5
www.reddit.com/r/sffpc/comments/t1y3al/cooj_mq5_show_and_build/
The AR900i has rear nvme? It doesnt look like it on their website. Am i missing something?
It does yeah, 2 slots on the front, 2 slots on the back I didn't realize it at first too.
thanks, I was confusing this one with the Ryzen version. I'm seriously considering pulling the trigger on a very similar setup. Is the nic m.2 combo card a cache only card and does that mean if I wanted to put it in RAID with the other drives on the board, that isnt possible? I was considering going with TrueNas as the OS@@LetsRTFM
They show up as if they are plugged in directly into the board, just like the other nvme drives, you should be able to do whatever you want with them.
If you don't need a 10gb nic and prefer the amd board you can use 4x4x4x4x bifrication with an Asus hyper m.2 btw, I've tested that configuration on both boards.
It works well on the bd790i but not the ar900i which doesn't have bios settings at all for pcie bifrication.
@LetsRTFM thanks man, extremely helpful info. Earned a sub from me! Hope your channel takes off
Have you heard about truenas? :)
The answer is less software not more :)
wouldnt 10gigabits/s be 1250 mb/s?
Yes, with more connections (I only used 2) and MTU adjustment from 1500 to 9000 you could make up the difference.
I'm demonstrating out of the box performance on my network with no configuration.
most of the time my use cases would be 4 clients each with 250MB/s connections.
Good video but showing black cases while wearing dark clothes it was difficult to see them...