NEW AI Accelerated NUC with 16 Cores from ASRock
Вставка
- Опубліковано 10 чер 2024
- We go into the ASRock Industrial NUC BOX-155H. This new NUC has 16 cores with three different configurations on-SoC, a GPU, Thunderbolt 4, USB4, up to 3 SSDs, and built-in AI acceleration with a NPU. Meteor Lake in NUC is a really cool combination.
STH Main Site Article: www.servethehome.com/asrock-i...
STH Top 5 Weekly Newsletter: eepurl.com/dryM09
----------------------------------------------------------------------
Become a STH YT Member and Support Us
----------------------------------------------------------------------
Join STH UA-cam membership to support the channel: / @servethehomevideo
----------------------------------------------------------------------
Where to Find The Unit We Purchased
Note we may earn a small commission if you use these links to purchase a product through them.
----------------------------------------------------------------------
- 64GB Crucial Memory Kit: amzn.to/3tNiBY6
- 48GB Crucial SODIMM: amzn.to/3HiGLwx
STH Merch on Spring: the-sth-merch-shop.myteesprin...
----------------------------------------------------------------------
Where to Find STH
----------------------------------------------------------------------
STH Forums: forums.servethehome.com
Follow on Twitter: / servethehome
Follow on LinkedIn: / servethehome-com
Follow on Facebook: / servethehome
Follow on Instagram: / servethehome
----------------------------------------------------------------------
Other STH Content Mentioned in this Video
----------------------------------------------------------------------
- Ultimate 2.5GbE Switch Guide: www.servethehome.com/the-ulti...
- 2.5GbE Switch Mega-Round-up: • Finding the BEST Cheap...
- Minisforum MS-01: • The Pinnacle of Mini P...
- Intel C3758 Fanless 10GbE Server: • The EVERYTHING $300 Fa...
- CWWK Fanless Desktop System: • FANLESS 7W Homelab Fan...
- New 8x 2.5Gbe and 4x 2.5GbE and 2x 10GbE Firewalls: • Shocking New 10GbE and...
- R86S Pro Review - • This is a SUPER Homela...
- R86S Review - • Ultimate Homelab Revol...
- iKoolCore R1 N6005 Review: • This CRAZY SMALL 2.5Gb...
- Intel Core i3-N305 Fanless: • Best EVER Fanless Mini...
- Intel N100 and N200 Fanless: • HUGE UPGRADE! New Fire...
- Intel Alder Lake Fanless: • KING of Fanless 2.5GbE...
- Beelink EQ12 Pro with i3-N305: • Intel Core i3 N305 Min...
- Example of setting up Proxmox VE with OPNsense/ pfSense: www.servethehome.com/topton-i...
- ASRock 4x4 AMD Ryzen 7 5800U: • This AMD Ryzen NUC Alt...
- Three 6x 2.5GbE Fanless Systems: • 3x 2.5GbE Fanless Fire...
- Two Fanless N5105 Systems: • New Fanless 4x 2.5GbE ...
- VMware ESXi and pfSense on a 6x 2.5GbE Core i7 Fanless System: www.servethehome.com/topton-f...
----------------------------------------------------------------------
Timestamps
----------------------------------------------------------------------
00:00 Introduction
01:51 External Hardware Overview
06:13 Internal Hardware Overview
10:10 Intel Core Ultra 7 155H Performance
11:38 Trying the Meteor Lake NPU with OpenVINO for AI inference acceleration
13:38 Casual Gaming Performance with League of Legends
13:58 Power Consumption and Noise
16:39 Key Lessons Learned
19:45 Wrap-up - Наука та технологія
I have a few ASRock Industrial NUCs, and they all work great. I had one fan-less that I returned and replaced by a fan version, for the exact reason you highlighted on your video: the SSD was getting too hot for my tastes, and while it was temporary, it was unnecessarily hot. And I won't even consider that was with a 64W processor, versus the now pretty warmer Core Ultra 7 and its 115W max... Don't have a need for NPU at this time, but I would seriously consider one whenever the need comes.
This is a nice unit for home lab server
cant wait for meteor lake mini boards for my nas/plex server. then i can av1 encode/decode on the box
I totally agree. This is one step in that direction though.
check out the new ASUS NUC 14 Meteor boards.... it's the same as the Intel NUC boards.... solid and much better quality and IO support
Could you do a thermal camera test on these systems, could be very interesting
Would a mini PC like this one be good for Promox to run several Windows VMs at once?
I am particularly concerned about the mix of efficiency cores and performance cores.
Do you recommend something else? RAM and Storage are used fast.
I wonder if you could replace the bottom plate with a fan using the same screws fan with a grill on it. That would be pretty sweet
It's nice to have the power supply readout visible in the camera, but I am curious why you didn't mention it while talking about the NPU?
You can actually see the power consumption go from ~50-60W when using the GPU for inference down to ~45-50W when using the NPU for inference. The main selling point for the NPU (for now) is efficiency by Intel's own slides.
Also, something interesting about that 50% utilization on the NPU, I am assuming that is because it was using the systolic MAC of the NPU for inference and the Meteor Lake NPU has two Neural Compute Engines, each containing a systolic MAC capable of performing 4096 INT8 MACs per cycle. I wonder if it would be simple to force the NPU scheduler to utilize both NCEs for a single inference task, and if so, how it might benefit performance.
Did I see correctly that when you turned on the NPU CPU stayed the same and GPU increased? Whats the benefit with the NPU here? It basically just added another 50% usage chart to it.
Very positive apd optimistic man! )
What did they use to found out that stuff about the usb ports? Is there a tester for this or is it methodical trial and error with many devices.
If you are really that space constrained I suppose it is better than the alternative; but this size seems to be the point where you really hit diminishing returns from the fact that you can ram just about anything into a type-C port. If you want any displayport enjoy having either 1 or zero type-C ports(and if you need newer displayport prepare to lose thunderbolt) and permanent cabling on both the front and rear of the device.
It just seems like the amount of extra footprint required to put the video-carrying type-Cs in the rear; or cram in a vertically oriented DP connection, would be essentially irrelevant to most mounting/positioning use cases and make life a lot more flexible. In addition to this being at the size where even small changes in size can make a significant difference in how irksome a fan is.
I realize that "NUC" implies certain conventional dimensions, unless it's one of the oddball ones that aren't that size, so Asrock is presumably working under constraints here; but the NUC target size just seems to fall a little too far in the direction of small-to-prove-a-point-in-a-world-of-OEM-minitowers rather than "small enough to fit in small places; but no smaller". Something more mac mini sized fits in almost all of the same places with less oversubscription of connectors.
I generally agree that I would prefer slightly larger systems with quieter cooling and more expansion capabilities.
Seems like a good little box for home automation stuff. Start using the NPUs in these things for local voice control functions and also for security camera systems.
I think that is the idea with a system like this.
@@ServeTheHomeVideo Yeah, it's clearly not made to push a ton of data around with the 2.5G nics.
I mean, 5Gbps is not too bad actually with SMB3 multi-channel
@@ServeTheHomeVideo I was more thinking for use as a head for a JBOD rack where one nic would be for network connection and the other for the JBOD connection. I suppose it would probably be decent for home use for most people though.
Isn't it an overkill ? For home, all of that can be done with less powerful, more efficient CPUs (n100, etc...).
I'm very interested in this product, I'd like to know if the built-in NPU can passthrough to a VM (guestOS) and run smart/AI accelerated photo albums...maybe unraid, maybe trueNAS scale
Not just into VMs, but also into Containers and then you can use it there (if the application in the container supports it)
That looks like a sweet little ESXI lab node.
I wish them a great industrial designer for their industrial line.
That SATA bay could fit something like an NF-A4x10 PWM fan from Noctua. That's what I'd put in there. Well, and some holes in the bottom of the case.
Interested that when you show the NPU in use, the GPU is still up at 80%? As I understood it, the encoder for OBS counts as a different percentage in windows...
That part might be av1 encoding
Great review. Useful! Thx!
Thanks for the feedback. Have a great evening
I wish it had ECC
If it was AMD it could have. But this channel looooooooooves Intel any chance it gets
lmfao, good one
@@lllongreen to be fair intel has the super low powered stuff and intel is what companies use in machines of this form factor.
@bhume7535 you've not been following the latest amd CPUs.
@@lllongreen to be fair to the person who made the original comment when people say this they usually mean they want fully validated, error reporting and correcting ecc, which is generally not found on AMD platforms outside of epic.
Is there an AMD Ryzen PRO 8000 (because of the little bit improved AI stuff compared to the 7000 series) variant of this with unbuffered ECC (!) SO-DIMM support?
Hoping we get some options soon-ish. This also only takes a few minutes to get up and running since OpenVINO is packaged well.
All Ryzen SKUs support Unbuffered ECC RAM. The 8945HS is the 7940HS with a NPU.
@@BrunodeSouzaLino that's hardware support. ECC must also be enabled in the BIOS and often it is not even a setting
@BrunodeSouzaLino not the mobile SKUs unless they are Pro skus. Also for the AM5 8700G it also needs to be Pro sku to support ECC
@@lllongreen Not really. It needs to be pro to support Buffered ECC. All Ryzen SKUs support Unbuffered ECC out of the box.
What tool was used to display that usage? Can anyone tell me please? The one on that big screen at 14-15 min!?
I'd like a small quiet pc that could accept a single 3.5" drive, doing some searching lots of people are asking, but it appears no one is making. Thanks for the review.
There are PCs smaller than the NuC with dual Ethernet and Dual normal sized HDMIs. They use NVME as the drive though. Available in Pentium, i3 variants. I could find them as mini PC in Amazon. 3.5” drives almost history now.
@@tamildesan837 They are not history when you want to store more than one or two terrabytes of data. Zimaboard can do this, but harddrive will be noisiest part and it has no enclosure and Celeron is very weak CPU, not much RAM and it's overpriced considering it's performance. Or USB3 box for harddrive can be an option.
Another problem with the labelling: is that DC jack centre positive (yay!) or centre negative? And why not print the voltage range and current requirement? It would cost them less to do that than process problems from customers due to the lack of labels.
Great points
Great video!
Having a similar shaped mini PC I’m worried about overheating. These could benefit from a 120mm fan taking up the top. The ssds and ram cool in these things.
Any idea on how these boxes would run local LLM like the Llama 70B model?
Did anyone else notice the utter smoothness he had throwing the RAM in?
1:11
The footage is sped up xD and it's at 1:16 actually.
the standing at desk studio layout is giving me Patrick Norton vibes
Good name
@ServeTheHomeVideo I know that "AI" on home PC's has barely taken it's first gulp of air and from here can only grow, but I noted that the "AI engine" gave the same performance for your assigned task (but with the lower latency that you noted) as did the "GPU", bu I noted the small difference in GPU and CPU usage.! I watched it again and now have some questions. You clearly stated that there were multiple tasks going on, but going from the GPU to the APU: Increased the "AI engine" from 0% to 50%, decreased the GPU performance from ~84% to ~70% and reduced the CPU performance from ~23% to ~19%. this is all well and good, but is there any actual benefit.? What is the real-World benefit for reducing CPU load by 5% and GPU load by 15% for power consumption.? What is the real-World result of using the "AI engine" at 50% load for power consumption.? What is the end result on power consumption between the "AI engine" and GPU, and what is the real-World implication of the increased latency.?
.
Right now all I see for home users is "AI" being used as a buzzword, I have not seen any real world benefit, but as you have done a test I thought I would ask you for the power draw difference because that is ultimately what makes the difference where this CPU (Meteor Lake) is going to be predominantly used, laptops/notebooks/tablets that all rely on batteries unlike this mini-PC.
.
Many thanks and keep up the excellent content.
Patrick speaks so fast sometimes I think he will choke on his own tongue! :) This content is great! I’ve had some great ideas while watching those videos!
There is a test online where the idle consumption on windows is about ~3-4 watts .. is this other test invalid or is windows just better at idle c-states?
Correct.
I really need to get me a mini PC for Proxmox. How important would you say ECC support is for Proxmox? I'd imagine regular VM snapshots would kind of mitigate things, but I don't know of any mini PC with ECC support AND QuockSync for hardware transcoding.
I'm currently leaning toward the MS-01.
It depends how mission critical the system is. For a homelab where the result of failure is at least a bit of downtime while I replace the ram and at most having to restore some stuff from backup, I don’t see ECC as essential. I’m not running the financial servers for an international corporation, I’m running a handful of VMs for personal use. :)
@@BrunodeSouzaLino So when your backed up data has those errors that you don't know about, how do you fix it? If you don't have any data that you care about, that's fine. Stop speaking for others.
This is as close to the right size you can get and has both: ASRock IMB-X1231. Though to get the proper ECC reporting is another issue altogether. Intel said consumers are dumb and not worth our time, so you don't need ECC, hence that is where the industry stands.
Can you tell me if World of Tanks will play on it and how much power consumption it uses??
Finally meteor lake!! 😀
What’s the actual API to NPUs, is that any way vendor neutral?
How does this compare to the Nvidia jetson orin NX 16 gb in terms of AI applications such as chatbots?
Probably better off getting an Apple silicon Mac right now for that. NVIDIA is changing Jetson to the tensor acceleration architecture for LLMs starting with Jetson Thor
So does this "industrial" machine support industrial standard ECC memory?
What do you think of using used server hdd in consumer pc?
Generally I do not love used HDDs
On the labelling it would also be great if the management NIC was labelled so it didn't accidentally get used as a wan port
Fair point
When will it be available?
AsRock is the only company I actually trust to do good hardware. Everybody else fell off.
Nice machine!
I wish for some of these little units they added in IPMI. Would 100% pay an extra $30-50 for that feature.
Usually it is $25-35 in BOM costs. Plus add 5W of power consumption at idle.
They have Intel vPro/AMT, which is basically a lower cost IPMI solution.
AMT vPro shares the NIC so you can manage across the NIC and use it as well, not like a server where you have a dedicated management NIC. You can use "meshcmd" to do commandline power control but unfortunately IPMI protocol is different.
Question: Can I run Proxmox on this NUC?
They still dont sell to individual ?
You can buy these on Newegg (we put the UA-cam product tag around 1 min into the video
Gonna be interesting with Proxmox and heterogeneous cores.
afaik that should be fine with proxmox 8, it has kernel 6.x and later
love these little computers but they need more USB ports !!
They call it the easy bake oven. Cook everything cook everthing!
Wifi range problem might because it's an older revision 002? the latest is rev 006.
Would love to see this with a Ryzen 7945HX
I want one. wow
I would like to see a vid about Asrock 4x4 Box 8840u (AMD)
No ecc, no ipmi, no deal.
what kind of workload are you running on this desktop mini PC that would need ECC? honest question as I have none I'd care about.
oooo...nice
a step in the right direction tho...it is an asrock 🤔
would like to see other brands to jump in the fray (competition tends to help lower prices too)
I think ASUS will be there since they bought the NUC line. One advantage for many of our viewers is they have heard of ASRock versus some of the small mini PC brands.
I hate it when they don't put a USB-C DP on the back though, its stupid plugging it into the front and its much neater to use for a monitor on these small units.
Asus nuc 14 pro & 14 pro+ ? when can we expect review ?
Hoping later this quarter for a few of those NUCs. We just got the NUC 13 Rugged and I am hoping we get 1-3 more and then maybe do a round-up style.
Meteor's lake NPU is a tiny part of the silicon and offers a ridiculous 10 TOPS, compared to the 4090's 330 TOPS.
I wonder who exactly is this for.
I can see the reason why this 100% fan mode exists. It literally says Asrock industrial and you can imagine ppl using this box along with loud machines.
Great point
...a MPU miniPC ..great ...!
I wish these mini pcs came with at least a 10GbE port so that I didn’t have to use a thunderbolt to 10GbE adapter 💰 ☹️
Also: that thing is loud 😅
Very loud when we had it in performance mode, but mid-range for these kinds of boxes when it is in balanced mode.
And with 4x m2 ssd? 😅
>wants a 10Gbe port
>complains about noise
FYI a 10 GbE controller is going to need a dedicated cooler the same size as the CPU
The problem IMO is people that want 10GbE often want it in SFP form, so you can use it with RJ45, Fiber and DAC/Solid Copper by just changing the modules. The problem is not so much the first two, but the latter. People that use solid copper cable for short distances tend to use passive cables where the module and cable are one thing and those can get hot to the point of burning the SFP port on passive cooled systems. 10GbE is a format where you're starting to go beyond what twisted pair can do, so a single RJ45 wouldn't cut for certain cases, especially considering the prices of CAT7 and CAT8 cables.
Where was the CPU located? I didnt see it when he took the bottom off.
Soldered onto the top
Can you please use rest of the world messurement system for size please
Meteor Lake benchmarks only look good if you don't compare them against Zen4!
But the Intel NPU is easier to use right now
@@ServeTheHomeVideo How so?
The AI Guru software can get everything installed and running with a few clicks and such. Intel has been doing OpenVINO for a long time.
That PsU hahaha. God damnit. Why not USBC PD like the Asus PN64?
Just wait for the next ASUS NUC video. I think we are going from a NUC that uses ~10W at the low end to one that uses like 300W at the high end. BIG ~330W PSU IIRC.
I have a Coral in my Server as a pcie Card but having it in a CPU build in is nice but using it is actual pain. I think this will Change in the Future so i have high Hops and such system are perfekt for learning and developing Software with Edge tpu
Thanks for the video and review. Can it be run directly from a nominal 12 volt battery?
Could have literally just looked it up in the time it spent to ask a question that will not be answered.
@@LackofFaithify Yes I could and will, but I am also trying to encourage NUC size computer makers to offer versions that work on 12, 24 and 48 volt battery packs.
These can be perfect for off grid and mobile use. There is a 30 watt standby power consumption in the inverter so if the built in power system can deal with it, it's super useful.
Is this very different from a NUC 13?
Raptor Lake versus the newer Meteor Lake. This is a good step up.
I'd love to get such a thing without fans / passively cooled.
I agree that would be cool, but it would be bigger.
@@ServeTheHomeVideoFor my desk, that would be OK. I'm typing this on my Zotac ZBOX CI669 right now, with "only" 12 cores. ;-)
minisPC got the ITX one much better due to a PCI-e, so the MS1 with out a PCI-e same chip I guess.
We need AI benchmarks...
Did I miss you mentioning the price??
yup main question
@@zer0b0t $700
Could someone help me find the software that used to generated the block diagram shown at 19:35? Thanks in advance!
lstopo
@@pwkerney Thank you!
What apps are you using to stress test and visualize the load @14:50?
Asking the important questions. Anyone?
s-tui
He did mention stress-ng and earlier some said it looks like s-tui.
Proxmox and linux? It was me missing that or...
When I see Ultra name in new intel CPU inside the mini PC huehue ULTRA ehh.
"most people only use the out of the box bios setting" even if this is true (I don't think home users who buy these would not look at the bios) I think you should test with the different PL settings
Bit expensive at $700 but interesting system. Can't wait till CWWK starts copying these.
Interesting. NPU in a such small box.
Is the cost a secret !
Get a quote and come on down to play at the Price is Right. ;)
Maybe it's good for work, but Intel gpu... Meh.
I have minisforum with ryzen 7840hs + 64gb ram + 990pro for work and 970 for windows and games, so, I can play something like gta 5 sometimes. And I usually don't hear it!
Btw, I would put 96gb ecc ram, because I reboot it once per week and think about ramdisk for ide cache, but it's not so easy to find good 48gb so-dimm ecc modules(
Looks at the insufficient cooling...I give it 3 months life at best with a CPU turbo temp of 110 Celsius.
i don't understand why they don't put the power supply inside the case, if this is an industrial thing. the mess of cables is maddening.
You can do up to 25gbps per thunderbolt
The challenge is that 25GbE adapters use more power and get hot so you need fanned enclosures, but you are correct.
@@ServeTheHomeVideo my friend you really need to speak more clearly much of what you said didn't sound like proper English is this your 2nd language? Listen to your audio and see if you think your diction and pronunciation is accurate. I'm British actually Scottish and seldom have issues with American accents but you need to speak slower and more precisely to be understood by us all. Still a great channel and you're a real nice fella 👍🇬🇧🏴✝️
The WiFi 7 standard isn't even finished yet.. lets not ask for production computers to include that just yet.
Released Jan 8, 2024 and with each generation vendors usually have draft devices that can be firmware updated to the final specs
@@ServeTheHomeVideo
Yeah ieee released 802.11be as a draft. But
that doesn’t change that it won’t be finished before q4, and adoption will probably start to take place in the early 2025.
I work as an SE with major Lan edge vendors, they are starting to put out 802.11be capable devices but they’re nowhere near ready yet for production yet. This also goes for running STA or radio/AP in 11.ax, 11.ac or legacy with slower phy. Sure firmware can be upgraded. But in the enterprise there are still hardware revisions and factory callbacks for 6e -wich just now is getting mainstream.
My point is, it should be done and stable before we start asking for it in production devices, especially mini “servers”. Right now its out for testing, and is mostly poc.
Meteor Lake. A good name would be cluster lake considering how it is made. Then when they start failing people will give them a similar sounding yet appropriate name hehehe. They might call it cluster flake or cluster something else...
Intel's new naming is just confusing.
Agreed.
Both Intel and Amd in the latest gen...two megacompanys shouldent be able to screw naming up this bad.
Why wouldn't they put a thunderbolt on the back? SMH
Not how you had to leave out the "intel" out or the description, and only mention it a little while into the video. I think a lot of viewers simply wont bother if they new it was an intel product. I gave this a thumbs up anyway.
At the moment: Core Ultra are less power efficient and don't have a decent internal GPU. Why do you wouldn't prefer current Ryzen 7XXX/8XXX over a core ultra? Even NPU is not a dealbreaker user 8XXX...
the Fans ALWAYS suck in this form factor. I've replaced many of them
I wish they made the chassis bigger with better cooling in almost every mini PC
I liked this video.
Thanks!
$700? ouch...
I wish it had an SFP+
put the PSU inside and make cooling not junk please
also, putting DP on the front isn't exactly convenient, also overall putting C on the front doesn't work that well yet either
Maybe 40gbps
If they are moving the power outside of the chassis, then of course they can make a mini PC. But it isn't really a mini PC, it's a wink-wink nudge-nudge mini PC.
There are two types
Of computers
Fanless( Phones)
And noisy
I am just waiting for the new McDonald's value meal with AI. The AI marketing is a joke these days.
You seriously need a compressor for your microphone system.
$699.99, it's too expensive for this.