Intel ME, PCIe bifurcation and more on the AliExpress Xeon Motherboard | Answering Your Questions
Вставка
- Опубліковано 6 тра 2024
- If you haven't seen the first video, please go check that out
• Building a Budget NAS!...
I think I discovered an incredible deal on AliExpress-a Xeon Motherboard with CPU and RAM for just $167! In this video, I’m answering all your questions. From PCIe bifurcation, to power consumption, to clearing Intel ME and more.
If you have more questions, please let me know in the comment section and I'll try to address it in the comments.
#nas #aliexpress #xeonmotherboard #nasbuild #budgettech #truenas #ubuntu #diyserver #techreview #pcbuild #homelab #networkattachedstorage #sata #mechanicaldisk #10gbe #pcie #techtips #budgetpc #serverbuild #computerhardware #geekculture #techdeals #gadgetreview #techtutorial #datastorage #techenthusiast #hardwarehacking #networking #speedtest #benchmarking #techcommunity - Наука та технологія
8:46 The heatsink on the bottom is for the chipset, and the slim heatsink on the top is for the VRM's.
Thanks for this follow up!!! I was eager to see your findings!!!. One of my questions that I think it was not addressed was the support for Wake On LAN (WoL). I have another chinese X99 motherboard and WoL sucks!!.
Really like to see a contrast against the and mini atx mini you had reviews for nas.
Thank's a lot for making second part video! I still don't know what to think about this board... it's quite powerfull one man army in one side but in other the power consumption on idle and no fully working IPMI in server board is a little crappy :( My current Unraid setup with i7-7700, GTX1650, 3HDD and 4SSD suck around 45W on idle with two VM and couple of docker container with spin up disks
I'm also with an i7 7700, 4 HDD disks, SSD and Quadro p2200, openmediavault, docker and Plex and searching for an upgrade setup 8 disk for my new Jonsbo N3.
I hoped this board will support ECC RAM and install TrueNAS but it looks like it is not.. 😢
@HaimPeretz Just make sure the ram is RDIMM Registered 2400 and you should be fine
@@HaimPeretz Might be hit or miss but RDIMM should work. I have a couple V3 systems and they can be picky about what ram works but they all have working ECC.
@Dvalin21 Are you sure.
I already have this RAM: hma82gr7cjr4n-vk
Will it work with ECC?
Also the power consumption is not good in Xeon.
@@HaimPeretz According to the manufacturer whom I spoke to, it has to be RDIMM or registered ECC 2400. However, since this ram you have is both registered and ECC, you maybe able to under clock it to 2400 and still use it. Try it as is first and see what happens. I look forward to hearing what happens.
The real question is "Why doesn't Proxmox enable iommu segmentation by default?". I spent a few days trying to pass through devices trying to figure out why the grouping weren't separated and available. Nothing in the documentation mentioned passing kernel flags in addition to the UEFI settings.
Thanks for this investigation! I've been running a proxmox server with truenas virtual on an old gaming rig setup with all OC options disabled and still having issues with stability, so I just ordered this kit today. I'm hoping I get as lucky as you, seems like everything worked out of the box. My first Ali order also so I'm super skeptical, but yolo, I guess! 😅
thanks for going through the bios
Just noticed at 13:34, under the IIO Configuration it has PCIE Hot Plug enable/disable (disabled by default) Also PCIE ACPI Hot Plug
PCIe hot plug always requires the drivers to support it as well, which is almost always only supported in specific "enterprise-validated configurations". It's almost guaranteed that hot plug won't work, but still would be interesting to find out. This board seems quite a surprise as it is, so who knows !
more video on this board noice still waiting for mine some issues with customs
kinda bummer ecc doesnt work with the ram the provided and pcie is only 2.0
@@YusufTech750 that pcie 2.0 limitation is going to hurt. The oldest pcie gpu that I have is a HD7770. I was hoping that I could upgrade the CPU to a E5 2699A v4, but that appears to be a disaster in the making.
In hindsight I should've bought a dual processor workstation, and added a 10Gb NIC.
I’ve been on the fence on ordering this board for the past 6 months. Not because I need it for anything, I just like weird niche boards. I’m glad that you bought it and played with it so I don’t have to, but realistically, I’ll probably order one.
Also, it has the bifurcation options im looking for. I emailed several of the vendors selling this board and none of them could answer me lol
Just found out that the ME Firmware that is currently on the system is as old as 2016 to 2017. The 009. There is an updated version the 100 which I guess is possible to see if the manufacturers will update this to the latest
Nice video my friend, but iperf3 is not suported in windows, even they create a exe, please use iperf v2, iperf3 is not fit for widows networking protocols, cheers.
Supply is still putting out voltage as long at it's plugged in. I always unplug the supply before I install cards. Burned up a motherboard once before I started following that rule.
Thank you for the wonderful review. My question is about enabling IOMMU for PCI passthrough on a specific motherboard. I use Proxmox and am interested in GPU passthrough. Does this motherboard support it?
That is pretty awesome to get 5 nvmes out of this. Also saw an adapter on AliExpress that connects to the NGFF slot (wifi slot) to pcie 16 slot. So it's possible to one more out of it. Thanks again for another awesome video, and happy to see the words of the naysayers put to rest. The only thing I didn't see and please forgive me if I missed it, but was it possible to set the server to return to last state if power was loss? Also does it wake on Lan? OH lastly, I have a link for this particular board to the bios. I can't post it here without UA-cam deleting it. I'll send ya the link, just let me know where to send it.
M.2 doesn't support hot swap, it might work with a pcie card but I wouldn't risk it unless it's like that icy dock card that's made for it. I'd be interested to see what power consumption was like with an L CPU assuming you can swap one in. That said I know to a lot of people that consumption is high but having ECC might be worth it for some people given the price point and bifurcation actually working is a nice bonus considering something modern like W680 will probably cost you double the price for just the motherboard to get ECC in a fairly low power system.
I'd still do a ton of testing before I trusted my data on it but honestly that board doesn't seem that bad. Have you tested things like WOL passthrough and USB ports? I know some of the other franken boards had problems with USB dropping out, passthrough being hit or miss and WOL not working properly. Not having IPMI isn't really an issue at the price point, but it would have been nice for another $20 or so.
Does the board support proper pcie power management? How does the C states look?
Update: The bios that shipped with my mobo is completely different (BIOS Date 2/20/2024). It is missing a bunch of options that you just showed in this video. Some of the options that are missing for example is Intel ME, Intel VT-d, PCI Subsystem Settings, trusted computing, CPU Configuration, etc. I would flash the old bios but I am afraid that this new bios is the only reason it detects my 64GB of RDIMM ram. I was able to boot into a live cd of Ubuntu 24.04 and ran some benchmarks. I did try installing the my E5 2697 V4 and had a weird issue with CPU sensors. For some reason Psensor only showed that I had 28 threads. I then installed the E5 2680 V4 back in the system and Psensor said I only have 14 threads. On both CPU's System Monitor showed the correct threads but still strange how the thermal sensors are missing threads. I would run the E5 2697 V4 in there 24/7 but I a afraid that for whatever reason the CPU overheats the mobo will not shut the computer off. I do have a E5 2660 V4 otw that is 105w so that should make this system more efficient and lessen my worry about the server overheating. One cool thing for anyone curious, my miniPCIe coral tpu installed in the mini PCIe slot is detected with lspci. I also tried a RTX 2060 and that was detected as well. I also ran lspci -k and it looks like 2 of the NIC's and the mini PCIe slot are in the same group. So if you are going to pass through a mini PCIe device figure out which NICS those are so you don't use them. Hope this helps someone out.
Thanks for sharing. It's strange, seems like they are shipping the new boards with a new/different BIOS, then. Is it possible for you to dump the BIOS like I do in the video and share it with me and others?
@@TechnicallyUnsure Debian 12 (and probably its derivatives) has the flashrom utility in their repos. I used it to backup the BIOS on a CWWK N5105 firewall/router box, before installing a later BIOS using a UEFI-based updater tool obtained from the CWWK website. It's likely I could have just updated the BIOS directly in Debian 12 with flashrom. So maybe you can use flashrom for a BIOS dump on this motherboard too.
my bios is the same - a lot of options missing. updated to the latest version on BKHD website and it still does not have the same options as in the video.
Also the claimed 64gb ECC memory support does not seem to work. I have 2 x 32GB RECC 2400 modules - identical part numbers and close serials - the system will only boot with either of the sticks in either of the slots. Putting in both gets me in an endless reboot loop (system posts and reboots)
I will see if the manufacturer can help with a different BIOS - the aliexpress seller said to contact them for custom bios and different options.
think you could make a video on this board using proxmox and OPNsense with passthrough ??? Got the board working with gaming memory 32 GB.
This motherboard has integrated chip aspeed ast2400. Potentially it can be managed. Any thoughts?
Is this mainboard 10 sata ports can hot swappable?
Great video! My board should show up soon. Thankfully I found some 2400 RDIMM. Do you think I could use a E5 2697 V4 145w with this board without a fire starting? The official manual says chips with 90W or less are recommended but they ship 105w and 120w chips with these boards. Having 18 cores would be awesome for virtualization.
Don't make my mistake. RDIMM Is not supported by the motherboard at all they claim it's supported, but after testing, no, it's not supported
Are you able to share a copy of your original BIOS from this motherboard by any chance? - the newer bios versions dont have any of the settings required for Virtualization :(
Did anyone confirm if this board restores to original power state after power is restored?
Well, 60W in TrueNAS idle with only two spinning drives is too much power consumption. If you assume roughly 5W per HDD, this is already 45 to 50W just for the Board with CPU and RAM but without any drives.
i think if he put on it unraid with pover governor and tweak C-states it might be idle at lower W, but like you said 2 spinning drives with idle cpu on 60W its to high for people like me.
Thanks for testing this, can you test if GPU integrated on motherboard have GPU video decoding or other ?
All information about it ? (For decoding In Plex or other)
Thanks.
Unfortunately there's no built-in GPU in this motherboard for Plex, either you can insert a GPU or just stream the videos as-is without transcoding.
What would be the maximum ram for this board? 64 GB DDR4 ECC RDIMM? 128 maybe in LRDIMM?
That would be awesome to test on this board
I understand correctly that only the factory provided RAM will work?
He's not sure yet. He ordered registered ECC to try. It sounds like the RAM he has simply might not be good enough.
So much power it is sucking up
Right? Was the idle power so high bc of how many drives were connected to it?
@@Rushil69420 those should not add that much to it, maybe 5-7w each.
@@bluesquadron593 why do we have to guess ? Boot it to an ubuntu USB stick… it's like 3 min to find out directly 🤷♂
So how much power does it take ? Or were you running all of your peripherals on a separate power supply ? It doesn't look like it from the video.
As you can see in the video, everything is running on a single PSU. Idle it's 60-70 watts and under heavy load and 100% CPU usage it shots up to 140w-ish
@@TechnicallyUnsure so no idea, got it
@@fiveangle what?
@@TechnicallyUnsure nothing has a more variable range of idle and transient power usage than SSDs and HDDs (are they idle, performing garbage collection, under load, linked at Gen2, Gen3 ?) Any guess would be exactly that, a wild guess. We just want to know how much power the board by itself takes when idle and under load. Not trying to be difficult... just saying what everyone else is thinking.
@@fiveangle No SSD/HDD, just NVMe SSD, booted into TrueNAS, idle after boot process, it's at 55W-ish
I can confirm that the NVME slot does not support a M.2 to SATA3.0 Adapter Card ASM1166 chipset 6-port Adaptor, bought one, plugged it in to expand my array and a no go. Then added same Adapter into a PCI-E NVME Adapter for the PCI-E slot and the unit was recognized. So now have 16 sata ports for my backup NAS. I expect the NVME only supports hard-drives only.
Thanks for testing, I was on the fence about buying that adapter.
got my board today, i have been trying to boot into various usb sticks with ventory tried ubuntu a few othe linux distros and they all crash. LDME linux also. These usually boot on my other system with no problem. I'm able to get into my bios and tweek it but nothing seems to work ! I even reset to defaults same thing. Seems to be a usb issue causing the problem.
Sorry to hear that. It could be the RAM issue, did you buy the RAM from them or you are using your own RAM. Seems like this motherboard ONLY works with 32GB and 2400 MHz RAM. Also instead of Ventoy, maybe try a USB stick with just Ubuntu on it.
@@TechnicallyUnsure So today I tested the memory on another system and that was ok the memory came with the system 32 GB. The only OS I was able to boot and instally on a SSD was OPNsense. Running with no problems. But I cant boot into no other OS be it Linux or Windows... Looks like only FreeBSD works with my system.
@@painkiller895 I had similar experience when I used ANY RAM that wasn't what vendor provided. Once I started using the original RAM, Ubuntu and other OSes started to work properly
Same here, I can boot into memtest86+ and test ram all day but I can't boot into a full linux distro. I've tried RHEL9, RHCOS, Ubuntu 24.04, an Avira Rescue CD I had laying around, they all hang with a kernel panic. :( Trying to decide if I want to go thru the trouble of returning it. Or giving them more money for RAM.
@@toneycassel4448 use gaming memory. I took out 2 sticks from my gaming pc and tried it on it and I was able to boot all linux distros. I'm using Corsair DDR4 2 16gb 3600mhz bought it at BestBuy in my area works great !