I had one of those setups for a bit and I must have spent 3 days trying to get it to post, I even got different CPUs, ram and tried everything and wasted days troubleshooting it and never got it working. Nice to see it in action!
I found the power draw comparison interesting, albeit presenting an incomplete picture. So you say Davinci Resolve took 5:58 mins rendering under Windows while drawing ~680 W. Linux took for the same task only 4:36 mins while drawing increased power of ~720 W. Which is better? Drawing more power at first sounds not so nice, but combined with using less time it might not be so bad. Assuming that the power draw given applies over the whole period of time in full gives us: Windows: 1/3600 h/sec * 358 sec * 680 W = 67,62 Wh Linux: 1/3600 h/sec * 276 sec * 720 W = 55,2 Wh So Linux, through being more efficient and thus completing the task in less time while drawing more power, consumes 12,42 Wh less energy (-18,4%) in the end. Nice.
I imagine it's similar to a GPU that's not being fully utilized. If the GPU isn't being maxed out it may step down its clocks to avoid wasting excess power.
The way in which the memory is accessed by the cores limits them having full usage. If you were instead for example to perform some calculations partly involving Matrices _(perhaps using "Open Source R" in coding for statistical analysis and data visualisation),_ you could eventually find a way to distribute the processing over the cores. One way to find out about R is at CRAN (Comprehensive R Archive Network for deep analytics), then RStudio. Also those (6282se) cores have a boost of about 3.3GHz (from 2.6GHz) and it is not hitting those _(and even then it is 3GHz when it is not 8 cores or less)._ They don't overclock. So you are spreading processing over the cores without each hitting the full boost (circa 3.0GHz and not the 3.3GHz). So they are only going (at best) at about three quarters (or less) of the clock speed and a lot of that is reliant on the cache and so do remember that what you are processing in video is heavily FPU based, not relying solely on what the cache offers efficiently. Each core has 16KB of L1 data cache per core but 64KB of L1 instruction cache is shared per two cores (per module). 2MB of L2 cache is shared per two cores (per module). However, 8MB of L3 cache is shared per Eight cores (per die). 14MB total L3 cache is available when using HT Assist (for memory bandwidth optimisation). Just as a try, check for patches that have changed Hyper Threading (meltdown/spectre and other pertaining old patches). Try an old linux distro on Debian or knoppix or slackware or fedora, from a distro of 2017 or older. Some of the video encoding with be heavily FPU based such as FFT (such as discrete cosine transform being a famous example). I'm surprised you did not try Ogg vorbis theora codecs. As an aside, this test you have done would be cool on your QuadFather PC comparing Linux and MSWindows in similar tests but also sometimes with a vulkan good GPU. My comment has no hate in it and I do no harm. I am not appalled or afraid, boasting or envying or complaining... Just saying. Psalms23: Giving thanks and praise to the Lord and peace and love.
The results on Ubuntu are quite impressive, but I would personally use a more lightweight distro of GNU/Linux, mostly because this would be better as a Rendering Rig or Server instead of a regular desktop.
Windows scheduler has indeed been shown to not scale too well beyond single socket systems and in fact even single socket with high thread counts can be constrained by the subpar scheduler. I guess it is to be expected. Linux has always been geared towards server use while Windows is a desktop first OS. Year of Linux desktop could be as close as 10 years from now though.
More precisely, it struggles with more than two NUMA domains. This is why CPUs like the 2990WX were amazing on Linux and often awful with Windows. Windows also has a max of 64 threads per NUMA domain, so top end chips with more than 32 cores can be problematic, especially if running two such CPUs in a single system. The Linux scheduler isn't perfect itself though. The year of the Linux Desktop for me was 2003. I've been using Linux for 23 years. My phone runs Linux as well (Android).
Check out the Anandtech article "896 Xeon Cores in One PC: Microsoft’s New x86 DataCenter Class Machines Running Windows", which includes a screenshot of Task Manager showing 896 cores, 32 sockets. BTW, in regular windows, it's best to use Windows 10 Pro for Workstation or Windows 11 Pro for Workstation; it handles dual CPU procs better.
I would be interested in seeing some Proton games running on this. I have heard from some people the Linux gives better performance that Windows when using Proton. It may be due to DXVK translating DirectX to Vulkan and Vulkan better using the hardware as seen in the Doom benchmark.
hey man, could you try running games in proton in linux on this rig? some of the native linux games arent updated or optimized to the degree that youd expect, and proton generally is pretty impressive with its compatibility. also subbed. neat video
Interesting coincidence to see this, just as I was starting to consider an upgrade from a dual CPU setup (Asus z8na-d6, dual x5675). I like the case on the one you show, being a desktop form factor is important for me
I don't know if you used it but Feral gamemode can help a lot with performance in games (it optimizes the os for gaming), other than that running Manjaro and Fedora has been shown to be faster than Ubuntu in a lot of cases.
The one thing I wonder after watching this is how thoroughly was Linux set up. The cliched saying that Linux is free if you don't value your time is somewhat true - there is a lot of 'under the hood' time spent tuning the system and making sure the OS has what it needs to perform. I left Mac OS for Linux (Elementary/Garuda/Pop) in order to game better - but, other than Garuda (which had the issue of not always being happy after an update, like 'I'm not gonna boot'), it took me a lot of time to tweak the system. Using Proton (and Glorious Eggroll) - emulated Windows (or native Linux) gaming outperformed most of the fast dwindling Mac native games. Eventually, I went to Windows 11 on my main PC and back to Mac OS on my retro tinkering machines. I'm curious what would happen with Linux if the system was tweaked to work with this beast - I mean that computer isn't exactly a garden gnome. Cool video though, as was the first part.
Fully buffered : Wendel from Level1Techs already confirmed what I know since the AMD FX 2011 era in a video from 2 years ago about the Threadripper 2990WX....which is that Linux scheduler is around 50 % more efficient that Microsofts....hence a more realistic and better performance of multi-core than in Windows OS.
That's interesting although known behavior designed like that on purpose, both OSes relay on devs to configure and optimize their scheduler per platform, if config for platform you're running isn't present windows will default to more single threaded operation since it's more stable and consumer apps tend to run better on it, linux default to as much parallelization as possible because it's mostly used on servers and it's assumed that people using it will have at least basic programing skills to write scheduler ruleset that will move program on single thread if it's crashing. still very cool to see it in action.
My recommendation is to make video where you use virtualization to run Windows and Linux at the same time. Something like Windows running on first CPU and then having Linux on other three which you can access via VNC and setup server. Just as an example how such system could be used.
Very interesting review, thank you. I wonder if you were running the proprietary video driver for Nvidia. Performance can certainly be worse, but usually not that bad.
Well with windows 10 Pro you have a max socket limit of 2 and 64 CPU cores. So a quad socket system just isn't going to be allowed to use them all due to software limitations. For quad socket support Workstation pro or enterprise versions are required or just go with windows server.
@@erkinalp As 8 socket systems and up to 4096 physical + logical core systems exist i kind of thought that was implied that it was implied it was the "first 2 sockets and first 64 logical cores"
glad you pointed that you; very few know about the Workstation edition, just the Server editions. Also you'd probably be interested in an older article on Anandtech called "896 Xeon Cores in One PC: Microsoft’s New x86 DataCenter Class Machines Running Windows".
You should rerun all the tests again. there has been huge improvements to Ubuntu and Steam since you last tested. I would love to see an update if you ever decide to do one.
As i mentioned in my comment on your other video. You'll want to run benchmarks with Node Interleaving turned off and on to see what the difference in performance is :)
I think the problem with your Linux gaming performance was probably related to your video drivers. If you had Nvidia drivers installed properly, performance should have been similar to what you have on windows. The fact that you have such a massive performance disparity means that you've probably been running open source Mesa drivers. Which are, of course, slower (since they've been built with a "reverse-engineering" approach, they lack a lot of important features (such as frequency scaling) and, really, are meant to be more like just "VGA adapter" drivers to give you some video output and mostly basic support for things like 3d webpages, especially if we're running them on Nvidia hardware, which has closed-source drivers.
the gaming performance could probably be improved by setting the cpu affinity of the games to only one cpu, so that the caching is more efficient. maybe the scheduler is clever enough to do this anyway, but manual intervention might be needed here. it's probably not going to push it to competitiveness with windows with older opengl games, but maybe it's worth a try :)
Very nice build, I too ran a G34 setup but with the Supermicro H8DG6-F, which had these specs: Valkyrie SMP 2014 2x 12 Core AMD OpteronMP 6180 SE D1 Mangy Cours, 24 Cores @ 2.5Ghz 2x Noctua NH-U9D0 A3 48GB NUMA Quad Channel DDR3-1600 ECC Reg HP Server Ram @ 1333Mhz Supermicro H8DG6-F AMD Maranello Blade Server Chipset 2x AMD SR5690 Northbridges,1x AMD SP5100 Southbridge Sapphire Nitro+ Radeon RX Vega 64 Limited Edition 2.0 PCI-E 2048 Bit HBM2 Auzentech X-Fi Bravura PCi-E 7.1 Corsair Force 3 240GB SSD Micron Crucial MX500 2TB SSD Plextor PX-B950SA BR-RW Drive, Singlelayer & Duallayer BR-RW + Lite Scribe 2x Intel 1000 MBit Ethernet RJ45 Lan Jacks SeaSonic PlatinumX 1000 Watt EPS 12V + 24V PSU Lian Li TYR PC-X2000B Server Design Case 1 of 200 Made Worldwide Lian Li W-X2000-B Window Kit 1 of 200 Made Worldwide Hewlett Packard ZR24w 24" 16:10 Aspect Ratio HS-IPS Panel AMD VSR @ 2560x1600x32 @ 60Hz Microsoft Windows 7 Professional x64 UK + Service Pack 01 AMD Adrenalin 21.5.2 Beta for Win7x64 Personally if using Linux like Clear Linux or even Kubuntu or Ubuntu a card like a RX Vega 64 or even a Radeon VII Could give some very interesting numbers, even a RX 5700 XT or a RX 6800 XT could be used, but these may be a bit too much for the system, for direct Compute stick with the Vega 56, Vega 64, Vega Prontier Edition or just the Radeon VII. Drivers are miles better for AMD Radeon than NVIDIA GeForce when it comes to Linux,, as other people have mentioned.
You should try gaming with proton from stream, if I'm right in my understanding, windows versions with proton do better than the native Linux versions of a game, at least from what I have seen, I'd love to see the results
i know that's maybe a bit much to ask for but still worth a try: Could you do tests with 4 6180 SEs and 4 6386 SEs? For Deerhound (K10) and Piledriver (K15.2) respectively? Bulldozer (K15) was just so bad.
Have you tried proton to run steam games? Because I'm running things like doom eternal on Ubuntu. This would make the available library on Linux like 200+x the size of what it is when you only run native titles. Protondb is a site to check the compatibility with the games, Linux and sometimes even configs you change to make things run properly. And the steam deck also runs Linux... If you try it on your main PC then you'll be shocked how good those games run (like no stutter in Elden ring at all)...
It's been some months, but maybe try the new ubuntu 22.04 with kernel 5.15 and proton 7.0 in steam. I use an 9900k with 3060 and the performance is almost the same as that in windows 10.
As for the gaming results, I am unsure why they differ so much. Did you use the native Linux ports or run the games as Windows versions through Proton (or even Proton GE) on Steam? While Feral Interactive's Linux native ports were great back in the days, they are ill maintained and polished over time. Also Tomb Raiders native port uses openGL still, running DXVK to translate DX11 to Vulkan by using the Windows version may get better results. Also enabling game-mode for game start-up might use the cores more effectively for games (have a look on ProtonDB and Lutris to see tweak suggestions). Rule of thumb: * older games using DX11 (Win) / OpenGL (Linux port) --> use Proton/Wine + DXVK * newer games using DX12 (Win) / Vulkan (Linux port) --> using Linux native port most often runs better than the DX12-Wine translation * if the game is older or the port not well maintained after an initial port --> going with Proton/Wine to run the Windows version may net better results since Windows received continuous attention in maintenance and polishing yet What Kernel was used for this benchmark? What Wine/Proton version? What's the graphics-card in this system and what drivers where used (In the previous video you swapped the graphics card for a Gigabyte GTX 1080, now I see yet another graphics card here ... a smaller MSI. Is this the GTX 750 Ti again that the system came with)? How does DOOM run in comparison on both OS's (since DOOM seems to be making insanely good use of a well implemented Vulkan)? In general, I don't see why Linux should run that much worse other than the application being specifically polished to run under Windows. I think there can be something done to increase your gaming experience under Linux on this machine. (Some thought: if sticking with Linux, it might be beneficial to give an AMD graphics card some consideration instead of a NVIDIA one due to the open source and MESA drivers, the kernel implementation, better Wayland support, no screen tearing out of the box etc. - depending on your use case of course)
Great video. An older version of Windows Datacenter or Red Hat version 7 or 8 might produce better results. You've got to get an OS from the time this system was at its best. Anyway, thanks for posting!
I don't know if you're comfortable with the command line, etc but could it be possible to try a BSD? Something like DragonflyBSD, which is supposed to be the Clear Linux of BSDs. I don't know if it would be faster than Linux, but there's a possibility, Dragonfly is supposed to have almost everything multithreaded.
What nVIdia driver was being used for the gaming runs? Proprietary or open source? Purely from the perfomance shown, I'd guess open source. nVidia proprietary drivers perform quite well in Linux when they aren't breaking everything. And like others said, AMD graphics are best on Linux. Run all the open source drivers as nearly every distro has by default and it all works great without fuss.
I am thinking whether you can put a 1kW solar installation to run this server with battery power, giving the opterons an EOL retirement as a game server maybe ?
Have you considered Windows Server 2016 / 2019 Datacenter or Standard converted to Workstation (windows 10 like GUI) enabled? While expensive, they were designed for this set up out of the box I use it as so on home high end multi core workstations for large RAM jobs. multi app rendering & VM server. Back in the days of Canterwood dual i875P dual pga-604's we had to. Windows XP couldn't cut it over 4g RAM or x64 (not till XP x64 2 yrs later). Server 2003 as workstation was awesome as an XP alternative for overclocking & gaming by time nVidia caught up w/ X64 SLI driver support.
I did consider Win Server, but since the main selling point of Win 10 Pro WS is its quad socket capability, I assumed it would have similar optimizations as Win Server for such a multi socket config.
this will be interesting to see how well it runs as a nas and attempt to edit videos, plus how well does it do creating virtual machines and of course look in gaming on that virtual machine also use amd as drivers seems to work better on linux overall than nvidia.
Why do you use sudo with geekbench? Is it faster in this way? P.S: my own PC is build on Socket F(LGA1207) MOBO with 2x Opteron 8439(6 cores) and 128Gb DDR2-1333(Quad-Channel), and FirePro S9050. Using Ubuntu. Also, I have some Socket G34 systems too, and LGA 771 mystem, and 2 Socket 940 systems.
Those games you tested runs on a OpenGL (3 if I remember correctly), so they are not very great comparisons. And, yes, as others have pointed out, yeet the Nvidia card.
bro, in the gaming section u used a gtx 1080 in the windows video, and you are clearly using the gtx 770 this time on linux, of course it will work way worse... also, nvidia drivers, even though they are not the best compared to windows ones, they perform better in some cases, but using amd is the way to go in linux.
I would have re-tried those games with the windows version running through proton. Some linux ports are pretty shit and the windows version in proton gets better performance.
Would love to see these benchmarks under win server OS (2012r2, 2016, 2019 & 2022), regardless of kernel versions, (win10, win11) windows mainstream is not a good OS choice
Let's start from beginning AMD never intended Opteron for any tasks you performed Opteron would actually work very well for VM host I have run multiple VMs on AMD Phenom II X6 1100T and 2x AMD Opteron 6180SE And despite having much more hardware on Dell PowerEdge R715 it is more power efficient running twice to four times as many VMs as Phenom II X6 1100T desktop This is power of such chips. Not running something faster, but more efficiently. I can run whole Dell PowerEdge R715 server in lower power usage then whole AMD Phenom II X6 1100T desktop. Whole AMD Opteron 6180SE is 125W TDP, same as AMD Phenom II X6 1100T, which makes it 51% faster then Phenom at same power or 34% more power efficient
Play some vulkan game and use an AMD GPU, if nothing else the AMD driver has lower overhead so it is probably more suited for this system. Even a game under proton is probably better than an OpenGL game when this cpu limited
Windows has had really sub-par NUMA support for years. I guess Linux being an awful lot on multi-socket servers has focused support on these sort of NUMA setups.
Check out the Anandtech article "896 Xeon Cores in One PC: Microsoft’s New x86 DataCenter Class Machines Running Windows"; it includes a screenshot of Task Manager with 32 sockets, 896 cores. Windows 10/11 Pro for Workstation also handles dual CPU processor boards better than Windows 10/11 Home.
Wow, probably garbage that eats kilowats((( 12600k 120wat, so you can have 2 or 3 pcs with those CPUS and destroy this 4 CPU monster) but nice to see what they did this time.
I wouldn't say 'terrible". Look at the Anandtech article "896 Xeon Cores in One PC: Microsoft’s New x86 DataCenter Class Machines Running Windows"; it shows task manager handling 32 sockets. There's also Windows 10/11 Pro for Workstation edition; it's what I use for my dual socket 44C/88T systems.
Maybe try a Radeon card instead, Nvidia drivers tend to be held back a bit on Linux
Also try proton for windows games, would be interesting to see the compatibility layer performance difference
Yeah, remember what Linus Torvalds said about NVidia... 😂
I had one of those setups for a bit and I must have spent 3 days trying to get it to post, I even got different CPUs, ram and tried everything and wasted days troubleshooting it and never got it working. Nice to see it in action!
Hi Timmy, many thanks for the comment! Sad to hear you couldn’t get yours to work - it’s a very finicky platform sadly..
Yooo its Timmy. I used to watch your vids all the time man
I found the power draw comparison interesting, albeit presenting an incomplete picture.
So you say Davinci Resolve took 5:58 mins rendering under Windows while drawing ~680 W.
Linux took for the same task only 4:36 mins while drawing increased power of ~720 W.
Which is better? Drawing more power at first sounds not so nice, but combined with using less time it might not be so bad. Assuming that the power draw given applies over the whole period of time in full gives us:
Windows: 1/3600 h/sec * 358 sec * 680 W = 67,62 Wh
Linux: 1/3600 h/sec * 276 sec * 720 W = 55,2 Wh
So Linux, through being more efficient and thus completing the task in less time while drawing more power, consumes 12,42 Wh less energy (-18,4%) in the end. Nice.
I imagine it's similar to a GPU that's not being fully utilized. If the GPU isn't being maxed out it may step down its clocks to avoid wasting excess power.
it's drawing more power because hardware is being properly utilized!
that's less than 10% extra power draw for like 30% more performance o.o
A small correction: the FPS mentioned on Tomb Raider was the Max FPS instead of average FPS. The differences are roughly the same though.
That GPU right there doesn't look like the one that been used with windows
It would be cool to try out the Steam Deck OS on this machine and see if it can improve gaming performance on it.
Thanks for the suggestion!
@@FullyBuffered i second that.
@@FullyBuffered you would need to use HoloISO, but I'd love to see it!
Did you install the NVIDIA proprietary drivers?
The way in which the memory is accessed by the cores limits them having full usage. If you were instead for example to perform some calculations partly involving Matrices _(perhaps using "Open Source R" in coding for statistical analysis and data visualisation),_ you could eventually find a way to distribute the processing over the cores. One way to find out about R is at CRAN (Comprehensive R Archive Network for deep analytics), then RStudio.
Also those (6282se) cores have a boost of about 3.3GHz (from 2.6GHz) and it is not hitting those _(and even then it is 3GHz when it is not 8 cores or less)._ They don't overclock. So you are spreading processing over the cores without each hitting the full boost (circa 3.0GHz and not the 3.3GHz). So they are only going (at best) at about three quarters (or less) of the clock speed and a lot of that is reliant on the cache and so do remember that what you are processing in video is heavily FPU based, not relying solely on what the cache offers efficiently. Each core has 16KB of L1 data cache per core but 64KB of L1 instruction cache is shared per two cores (per module). 2MB of L2 cache is shared per two cores (per module). However, 8MB of L3 cache is shared per Eight cores (per die).
14MB total L3 cache is available when using HT Assist (for memory bandwidth optimisation).
Just as a try, check for patches that have changed Hyper Threading (meltdown/spectre and other pertaining old patches). Try an old linux distro on Debian or knoppix or slackware or fedora, from a distro of 2017 or older.
Some of the video encoding with be heavily FPU based such as FFT (such as discrete cosine transform being a famous example). I'm surprised you did not try Ogg vorbis theora codecs.
As an aside, this test you have done would be cool on your QuadFather PC comparing Linux and MSWindows in similar tests but also sometimes with a vulkan good GPU.
My comment has no hate in it and I do no harm. I am not appalled or afraid, boasting or envying or complaining... Just saying. Psalms23: Giving thanks and praise to the Lord and peace and love.
The results on Ubuntu are quite impressive, but I would personally use a more lightweight distro of GNU/Linux, mostly because this would be better as a Rendering Rig or Server instead of a regular desktop.
Windows scheduler has indeed been shown to not scale too well beyond single socket systems and in fact even single socket with high thread counts can be constrained by the subpar scheduler. I guess it is to be expected. Linux has always been geared towards server use while Windows is a desktop first OS. Year of Linux desktop could be as close as 10 years from now though.
More precisely, it struggles with more than two NUMA domains. This is why CPUs like the 2990WX were amazing on Linux and often awful with Windows. Windows also has a max of 64 threads per NUMA domain, so top end chips with more than 32 cores can be problematic, especially if running two such CPUs in a single system. The Linux scheduler isn't perfect itself though.
The year of the Linux Desktop for me was 2003. I've been using Linux for 23 years. My phone runs Linux as well (Android).
Check out the Anandtech article "896 Xeon Cores in One PC: Microsoft’s New x86 DataCenter Class Machines Running Windows", which includes a screenshot of Task Manager showing 896 cores, 32 sockets. BTW, in regular windows, it's best to use Windows 10 Pro for Workstation or Windows 11 Pro for Workstation; it handles dual CPU procs better.
I would be interested in seeing some Proton games running on this. I have heard from some people the Linux gives better performance that Windows when using Proton. It may be due to DXVK translating DirectX to Vulkan and Vulkan better using the hardware as seen in the Doom benchmark.
Sometimes it does, but not always... Still to me it's good enough
hey man, could you try running games in proton in linux on this rig? some of the native linux games arent updated or optimized to the degree that youd expect, and proton generally is pretty impressive with its compatibility. also subbed. neat video
Interesting coincidence to see this, just as I was starting to consider an upgrade from a dual CPU setup (Asus z8na-d6, dual x5675). I like the case on the one you show, being a desktop form factor is important for me
Prores is a well known, frequently use format for high res , near lossless. Most content creators know this.
Have you tried using a AMD card? I could be wrong, but may get better performance in Linux.
Dude, you forgot to install graphic drivers, at 1:58 it can be seen you are running at 1024x768, so the lackluster gaming performance
I don't know if you used it but Feral gamemode can help a lot with performance in games (it optimizes the os for gaming), other than that running Manjaro and Fedora has been shown to be faster than Ubuntu in a lot of cases.
The one thing I wonder after watching this is how thoroughly was Linux set up. The cliched saying that Linux is free if you don't value your time is somewhat true - there is a lot of 'under the hood' time spent tuning the system and making sure the OS has what it needs to perform.
I left Mac OS for Linux (Elementary/Garuda/Pop) in order to game better - but, other than Garuda (which had the issue of not always being happy after an update, like 'I'm not gonna boot'), it took me a lot of time to tweak the system. Using Proton (and Glorious Eggroll) - emulated Windows (or native Linux) gaming outperformed most of the fast dwindling Mac native games.
Eventually, I went to Windows 11 on my main PC and back to Mac OS on my retro tinkering machines.
I'm curious what would happen with Linux if the system was tweaked to work with this beast - I mean that computer isn't exactly a garden gnome.
Cool video though, as was the first part.
Thanks for the comment!
Fully buffered : Wendel from Level1Techs already confirmed what I know since the AMD FX 2011 era in a video from 2 years ago about the Threadripper 2990WX....which is that Linux scheduler is around 50 % more efficient that Microsofts....hence a more realistic and better performance of multi-core than in Windows OS.
That's interesting although known behavior designed like that on purpose, both OSes relay on devs to configure and optimize their scheduler per platform, if config for platform you're running isn't present windows will default to more single threaded operation since it's more stable and consumer apps tend to run better on it, linux default to as much parallelization as possible because it's mostly used on servers and it's assumed that people using it will have at least basic programing skills to write scheduler ruleset that will move program on single thread if it's crashing. still very cool to see it in action.
My recommendation is to make video where you use virtualization to run Windows and Linux at the same time. Something like Windows running on first CPU and then having Linux on other three which you can access via VNC and setup server. Just as an example how such system could be used.
I wonder how well that would run Proxmox. I bet you could load up a bunch of VMs and containers on that and work quite well.
Very interesting review, thank you. I wonder if you were running the proprietary video driver for Nvidia. Performance can certainly be worse, but usually not that bad.
Well with windows 10 Pro you have a max socket limit of 2 and 64 CPU cores. So a quad socket system just isn't going to be allowed to use them all due to software limitations.
For quad socket support Workstation pro or enterprise versions are required or just go with windows server.
The system will boot up but it will only allow you to use "first" 2 sockets and "first" 64 cores.
@@erkinalp As 8 socket systems and up to 4096 physical + logical core systems exist i kind of thought that was implied that it was implied it was the "first 2 sockets and first 64 logical cores"
glad you pointed that you; very few know about the Workstation edition, just the Server editions. Also you'd probably be interested in an older article on Anandtech called "896 Xeon Cores in One PC: Microsoft’s New x86 DataCenter Class Machines Running Windows".
You should rerun all the tests again. there has been huge improvements to Ubuntu and Steam since you last tested. I would love to see an update if you ever decide to do one.
As i mentioned in my comment on your other video. You'll want to run benchmarks with Node Interleaving turned off and on to see what the difference in performance is :)
The 6300 series Opterons add AVX 1.1 support. That should make a big impact on gaming performance. Love the rig!
I think the problem with your Linux gaming performance was probably related to your video drivers. If you had Nvidia drivers installed properly, performance should have been similar to what you have on windows.
The fact that you have such a massive performance disparity means that you've probably been running open source Mesa drivers. Which are, of course, slower (since they've been built with a "reverse-engineering" approach, they lack a lot of important features (such as frequency scaling) and, really, are meant to be more like just "VGA adapter" drivers to give you some video output and mostly basic support for things like 3d webpages, especially if we're running them on Nvidia hardware, which has closed-source drivers.
the gaming performance could probably be improved by setting the cpu affinity of the games to only one cpu, so that the caching is more efficient. maybe the scheduler is clever enough to do this anyway, but manual intervention might be needed here. it's probably not going to push it to competitiveness with windows with older opengl games, but maybe it's worth a try :)
Very nice build, I too ran a G34 setup but with the Supermicro H8DG6-F, which had these specs:
Valkyrie SMP 2014
2x 12 Core AMD OpteronMP 6180 SE D1 Mangy Cours, 24 Cores @ 2.5Ghz
2x Noctua NH-U9D0 A3
48GB NUMA Quad Channel DDR3-1600 ECC Reg HP Server Ram @ 1333Mhz
Supermicro H8DG6-F AMD Maranello Blade Server Chipset 2x AMD SR5690 Northbridges,1x AMD SP5100 Southbridge
Sapphire Nitro+ Radeon RX Vega 64 Limited Edition 2.0 PCI-E 2048 Bit HBM2
Auzentech X-Fi Bravura PCi-E 7.1
Corsair Force 3 240GB SSD
Micron Crucial MX500 2TB SSD
Plextor PX-B950SA BR-RW Drive, Singlelayer & Duallayer BR-RW + Lite Scribe
2x Intel 1000 MBit Ethernet RJ45 Lan Jacks
SeaSonic PlatinumX 1000 Watt EPS 12V + 24V PSU
Lian Li TYR PC-X2000B Server Design Case 1 of 200 Made Worldwide
Lian Li W-X2000-B Window Kit 1 of 200 Made Worldwide
Hewlett Packard ZR24w 24" 16:10 Aspect Ratio HS-IPS Panel
AMD VSR @ 2560x1600x32 @ 60Hz
Microsoft Windows 7 Professional x64 UK + Service Pack 01
AMD Adrenalin 21.5.2 Beta for Win7x64
Personally if using Linux like Clear Linux or even Kubuntu or Ubuntu a card like a RX Vega 64 or even a Radeon VII Could give some very interesting numbers, even a RX 5700 XT or a RX 6800 XT could be used, but these may be a bit too much for the system, for direct Compute stick with the Vega 56, Vega 64, Vega Prontier Edition or just the Radeon VII.
Drivers are miles better for AMD Radeon than NVIDIA GeForce when it comes to Linux,, as other people have mentioned.
Strange it uses such a middling percent of all cores instead a higher % usage access the board, I was expecting it to max the lot.
did you ever do a follow up upgrade to this machine?
i would suggest water cooling , it would cut down the noise
my guess for the reason that for the rendering its not using 100% of the core power, probably maxed out the cache thats being shared
On Ubuntu you can install a performance selector, set it to high.
Should do gentoo next. If you find the right compiler flags and tune builds for opteron then you'll notice a big difference.
"right compiler flags"
Surely it would be MAKEOPTS, right?
cflags are a meme lol
@@gøckłøłe all of them. You can go look at the gcc documentation and see what's available. Just takes playing around.
I look forward to a layman's distro of Clear Linux
You should try gaming with proton from stream, if I'm right in my understanding, windows versions with proton do better than the native Linux versions of a game, at least from what I have seen, I'd love to see the results
honestly you should try messing with the cpu-freq files and energy power files, set everything at 100%
Thank you for testing this! It's a bit sad that Windows is so bad at utilizing so many NUMA nodes.
You're welcome!
2:06 god thats a long hostname
i know that's maybe a bit much to ask for but still worth a try: Could you do tests with 4 6180 SEs and 4 6386 SEs? For Deerhound (K10) and Piledriver (K15.2) respectively? Bulldozer (K15) was just so bad.
An upgrade to Abu Dhabi chips is in the works!
Have you tried proton to run steam games? Because I'm running things like doom eternal on Ubuntu. This would make the available library on Linux like 200+x the size of what it is when you only run native titles.
Protondb is a site to check the compatibility with the games, Linux and sometimes even configs you change to make things run properly.
And the steam deck also runs Linux... If you try it on your main PC then you'll be shocked how good those games run (like no stutter in Elden ring at all)...
Sorry I missed the first installment. Did you try Process Lasso to keep the NUMA architecture happy? Or NUMA aware programs?
That server is best used for virtualization where the cores will be better used...
It's been some months, but maybe try the new ubuntu 22.04 with kernel 5.15 and proton 7.0 in steam. I use an 9900k with 3060 and the performance is almost the same as that in windows 10.
As for the gaming results, I am unsure why they differ so much. Did you use the native Linux ports or run the games as Windows versions through Proton (or even Proton GE) on Steam?
While Feral Interactive's Linux native ports were great back in the days, they are ill maintained and polished over time. Also Tomb Raiders native port uses openGL still, running DXVK to translate DX11 to Vulkan by using the Windows version may get better results. Also enabling game-mode for game start-up might use the cores more effectively for games (have a look on ProtonDB and Lutris to see tweak suggestions).
Rule of thumb:
* older games using DX11 (Win) / OpenGL (Linux port) --> use Proton/Wine + DXVK
* newer games using DX12 (Win) / Vulkan (Linux port) --> using Linux native port most often runs better than the DX12-Wine translation
* if the game is older or the port not well maintained after an initial port --> going with Proton/Wine to run the Windows version may net better results since Windows received continuous attention in maintenance and polishing yet
What Kernel was used for this benchmark? What Wine/Proton version? What's the graphics-card in this system and what drivers where used (In the previous video you swapped the graphics card for a Gigabyte GTX 1080, now I see yet another graphics card here ... a smaller MSI. Is this the GTX 750 Ti again that the system came with)?
How does DOOM run in comparison on both OS's (since DOOM seems to be making insanely good use of a well implemented Vulkan)?
In general, I don't see why Linux should run that much worse other than the application being specifically polished to run under Windows.
I think there can be something done to increase your gaming experience under Linux on this machine.
(Some thought: if sticking with Linux, it might be beneficial to give an AMD graphics card some consideration instead of a NVIDIA one due to the open source and MESA drivers, the kernel implementation, better Wayland support, no screen tearing out of the box etc. - depending on your use case of course)
Thanks the comment, something to look into
Great video. An older version of Windows Datacenter or Red Hat version 7 or 8 might produce better results. You've got to get an OS from the time this system was at its best. Anyway, thanks for posting!
I wonder how it would last as an virtualization server
linux will work fine on this for actual server tasks. This thing makes a damn fine vm testbench or docker core
Have you tried to have Linux Native, and run Windows in VM..? I saw some spicy test once Level1Techs did on some Eypc CPU's.
Quirks and features. Nice.
I don't know if you're comfortable with the command line, etc but could it be possible to try a BSD? Something like DragonflyBSD, which is supposed to be the Clear Linux of BSDs. I don't know if it would be faster than Linux, but there's a possibility, Dragonfly is supposed to have almost everything multithreaded.
What nVIdia driver was being used for the gaming runs? Proprietary or open source?
Purely from the perfomance shown, I'd guess open source. nVidia proprietary drivers perform quite well in Linux when they aren't breaking everything.
And like others said, AMD graphics are best on Linux. Run all the open source drivers as nearly every distro has by default and it all works great without fuss.
Fantastic results for a 12 year old chip of which their design is dated back to AM2 Phenom Chips.
You should compile a kernel !
Cool rig - good vid
I am thinking whether you can put a 1kW solar installation to run this server with battery power, giving the opterons an EOL retirement as a game server maybe ?
Have you considered Windows Server 2016 / 2019 Datacenter or Standard converted to Workstation (windows 10 like GUI) enabled? While expensive, they were designed for this set up out of the box I use it as so on home high end multi core workstations for large RAM jobs. multi app rendering & VM server. Back in the days of Canterwood dual i875P dual pga-604's we had to. Windows XP couldn't cut it over 4g RAM or x64 (not till XP x64 2 yrs later). Server 2003 as workstation was awesome as an XP alternative for overclocking & gaming by time nVidia caught up w/ X64 SLI driver support.
I did consider Win Server, but since the main selling point of Win 10 Pro WS is its quad socket capability, I assumed it would have similar optimizations as Win Server for such a multi socket config.
Have you tried a Windows Server install to see if Performance is better?
and did you change the GPU for Ubuntu test or did the 1080 worked the same?
if you build the linux kernel to this cpu it should gain more performance
Dual boot to make it all great.. but that's a pain too....
this will be interesting to see how well it runs as a nas and attempt to edit videos, plus how well does it do creating virtual machines and of course look in gaming on that virtual machine also use amd as drivers seems to work better on linux overall than nvidia.
Why do you use sudo with geekbench? Is it faster in this way?
P.S: my own PC is build on Socket F(LGA1207) MOBO with 2x Opteron 8439(6 cores) and 128Gb DDR2-1333(Quad-Channel), and FirePro S9050. Using Ubuntu. Also, I have some Socket G34 systems too, and LGA 771 mystem, and 2 Socket 940 systems.
Running in sudo mode got me slightly better results IIRC
Have you installed the nvidia proprietary driver?
Those games you tested runs on a OpenGL (3 if I remember correctly), so they are not very great comparisons.
And, yes, as others have pointed out, yeet the Nvidia card.
iirc there is a bios that can let you oc opterons
bro, in the gaming section u used a gtx 1080 in the windows video, and you are clearly using the gtx 770 this time on linux, of course it will work way worse... also, nvidia drivers, even though they are not the best compared to windows ones, they perform better in some cases, but using amd is the way to go in linux.
I would have re-tried those games with the windows version running through proton. Some linux ports are pretty shit and the windows version in proton gets better performance.
Chce to ještě jeden test...POP os:) Pro mě nejlepší distro.
when you can upgrade the cpus that would be awsome theanks for the vid.
Would love to see these benchmarks under win server OS (2012r2, 2016, 2019 & 2022), regardless of kernel versions, (win10, win11) windows mainstream is not a good OS choice
While it was true with win 2003, I was quite disappointed after comparing win2012 vs win7. Server OSes from M$ got worse.
Would be worth trying on Windows 10/11 Pro for Workstations (a different edition) because it's meant for multi-socket systems.
"OOBANTOO" does not exist...
Let's start from beginning
AMD never intended Opteron for any tasks you performed
Opteron would actually work very well for VM host
I have run multiple VMs on AMD Phenom II X6 1100T and 2x AMD Opteron 6180SE
And despite having much more hardware on Dell PowerEdge R715 it is more power efficient running twice to four times as many VMs as Phenom II X6 1100T desktop
This is power of such chips. Not running something faster, but more efficiently.
I can run whole Dell PowerEdge R715 server in lower power usage then whole AMD Phenom II X6 1100T desktop.
Whole AMD Opteron 6180SE is 125W TDP, same as AMD Phenom II X6 1100T, which makes it 51% faster then Phenom at same power or 34% more power efficient
It can compile very fast!
Play some vulkan game and use an AMD GPU, if nothing else the AMD driver has lower overhead so it is probably more suited for this system.
Even a game under proton is probably better than an OpenGL game when this cpu limited
Nice!
Birşey sorucam i5 460m 16gb ddr3 1333mhz ram 240gb ssd photshop kullanabilirmiyim
- When I show that I can compile, render, calculate and play game simultneously, people get their brains blown off :))))))
i think you should have used windows server for the windows test. windows server is numa aware.
W10 Pro WS should be identical, given its main selling point is support for quad socket systems (as opposed to 2 socket max on regular W10 Pro)
Interesting but someone would have to pay me to use this computer
Use the proton windows verison DUH (talking about the gaming)
Windows has had really sub-par NUMA support for years. I guess Linux being an awful lot on multi-socket servers has focused support on these sort of NUMA setups.
Check out the Anandtech article "896 Xeon Cores in One PC: Microsoft’s New x86 DataCenter Class Machines Running Windows"; it includes a screenshot of Task Manager with 32 sockets, 896 cores. Windows 10/11 Pro for Workstation also handles dual CPU processor boards better than Windows 10/11 Home.
that noise lol computer is loud.
For something it will be that in large systems, Windows is NEVER used.
Windows is a toy operating system.
I bet if you threw a radeon in there it'd work. linux just really hates nvidia sometimes.
cpuって4つ搭載できるんだ
An Intel distro tested on an AMD CPU 😁
It's all fun and game until you use it for mining Monero
but it feels to tell me: buy a new machine and dont waste all the power ;) ... hope you get me. xD
id like to see this thing again but run the tests in windows 10 with the ATLAS os installed and all the cpu vunerability fixes Disabled.
and change the benchmark prioritys to higher priority.. iv whitnessed cinibench giving higher scores when you elevate its priority in the taskmanager.
Wow, probably garbage that eats kilowats((( 12600k 120wat, so you can have 2 or 3 pcs with those CPUS and destroy this 4 CPU monster) but nice to see what they did this time.
tux smash! :)
Fedora anyone ???
Windows is still terrible at handling any NUMA systems. Just like the Threadripper systems still suffer on Windows.
I wouldn't say 'terrible". Look at the Anandtech article "896 Xeon Cores in One PC: Microsoft’s New x86 DataCenter Class Machines Running Windows"; it shows task manager handling 32 sockets. There's also Windows 10/11 Pro for Workstation edition; it's what I use for my dual socket 44C/88T systems.
That piece of junk consumes more power than a Tesla 😂😂
- When I show that I can compile, render, calculate and play game simultneously, people get their brains blown off :))))))