Getting The Most Out Of Your Epyc Server With Proxmox!
Вставка
- Опубліковано 11 лип 2024
- www.linode.com/level1techs
For more information:
forum.level1techs.com/t/how-t...
**********************************
Thanks for watching our videos! If you want more, check us out online at the following places:
+ Website: level1techs.com/
+ Forums: forum.level1techs.com/
+ Store: store.level1techs.com/
+ Patreon: / level1
+ L1 Twitter: / level1techs
+ L1 Facebook: / level1techs
+ Wendell Twitter: / tekwendell
+ Ryan Twitter: / pgpryan
+ Krista Twitter: / kreestuh
+ Business Inquiries/Brand Integrations: Queries@level1techs.com
IMPORTANT Any email lacking “level1techs.com” should be ignored and immediately reported to Queries@level1techs.com.
-----------------------------------------------------------------------------------------------------------
Intro and Outro Music By: Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 3.0 License
creativecommons.org/licenses/b... - Наука та технологія
I'd be down for another L1 vid on configuring HA on Proxmox. Also would love some XCP-NG content :)
VIRTUALIZE ALL THE THINGS
This is my motto
Yes wendle. Please. More xcp-ng and proxmox stuff.
Seconded. I'd love to be able to step up a 3 node homelab with commodity hardware.
@@OldKing11100 3-node HA cluster at home is when you've kinda crossed the line into "having a problem". But it's OK, this is a safe space.
seconding this, XCP-ng performing great in homelab for me
Watched this yesterday on my lunch break at work. Good video. Looking forward to more content on Proxmox and EPYC.
Thanks for this! I don't have an EPYC quite yet, but more Proxmox content is really nice to have. It's something I use enough that I love having videos explore what is possible with it and how to use it.
I just started looking at proxmox for a small company... We have it installed on a couple R720's to play with. When we finally got to the point were we pulled the network cable on the main machine running the windows vm's and they magically appeared running on the 2nd node (setting up replication and HA).... Awesome! We certainly have a bit to learn but love these videos! (mainly to know what we are doing wrong) ;)
Your knowledge on Linux and servers blows my mind. I heard your voice on TekSyndicate years ago, and saw you on LTT. "Wait, I recognize that voice!" haha. Love the channel. Getting some spare parts to put together a vm server for webhosting. Glad to know there's a channel like yours I can refer to. Keep it up!
Very interesting and you’ve got my interest in Proxmox. Excellent work.
I wish this guy be my older brother. Omg such a pleasure sit, watch and listen to him. keep it going man!
Yay! Linux content! Love proxmox
Super helpful! Thank you! Would love to see a video about setting up containers on proxmox. Or a video about how to consolidate old hardware
Yes please clusters and ceph storage setup tutorial!Thank you Wendell.
uppppppppppppp
Absolutely this. Ceph would be great to have a L1 video on.
I would love to see you make more proxmox content. Really good information in this video
I feel like every time I find something amazing, Wendell has already done a video about it.
Great video. I have fun with proxmox and pass through nic for pfsense
I've used proxmox for years, on dedi's and going to use it on my threadripper when I upgrade to a real gaming pc instead of workstation in the next years, I love it!
i love these linuxy server-type videos!
I would love a Wendell comparison of these major virtualization implementations; Proxmox, Unraid, FreeNAS, XCP-NG, Xen (standalone).. I am planning a home server to do various things and my buddy in the hosting industry suggests Xen (standalone) to me, but whenever I research it online, everything is about XenServer or XCP-NG.. A detailed explanation of these different products would be amazing!
:)
I second this. ☝️
Has this been addressed yet?
I've recently spent some time playing with Proxmox in a VM, and I'm impressed. I think this is what I'll install on my next server.
When you install Proxmox (with no subscription), after you fix the APT repository and pull in updates, run this:
sed -i.backup "s/data.status !== 'Active'/false/g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js && systemctl restart pveproxy.service
Ta-da, no more subscription nag when you log in to the web UI (might have to clear browser cache). Put that in a shell script so you can run it each time you pull updates.
I only watch these videos to hear your ramble Sempai!
it would have been great if you could do a comparison with xcp-ng , or in terms of performance, with a container only solution like coreos. thanks for the video
We need more Proxmox in our lives, Wendell.
Oh yeah more videos please! One question. How is GPU passthrough?
Oh and Yes! I would love to see setting up proxmox cluster. Especially because AFAIK there is no easy way of doing host backups on single node setup (only vms and cts backups).
I'm new to Proxmox and I'm interested in the best way to do host backups too. There may not be a lot of files added/changed on the host compared to a fresh install, but it would still be nice to avoid identifying all the files that need backing up & just make a full backup. This could be a reason to choose ZFS when installing Proxmox, since you could then snapshot the host & back that up.
@4:06 the antec 900 case! That's my case. Soooo much airflow.
This will be very useful for the epyc server I can totally afford
Some of the first gen stuff isn't too bad now. Am looking at a 32 core with motherboard and 64GB memory for $1200 AUD.
amazing content, more of this!
great video's wendell! please make a vid for HA, Ceph, and perhaps compare Proxmox, XCP-NG, & Xen.
I'm setting up some new hardware in my homelab for virtualisation, and it would be greatly appreciated.
Wendel! You're just amazing ! Compare proxmox with xcp-ng, planning on a homelab server to run bitwarden, plex, nextcloud, 2 VMs for gaming and some other VMs to test windows and linux distros for work!
I'm excited for the bluetooth linux stuff, I seem to always have issues with streaming audio via bluetooth from linux where I have no issues under windows.
just so you know, i've been using linode for at least 5 years, which was based on a comment you made about using them (not even a sponsor at the time).
Love the intro into Promox!
Any chance of digging into oVirt or Vsphere?
Hi Wendel, would love to see a video on the Dell server with Graphcore IPU cards in it!
I was the 5000th viewer and 500th upvote. It's a sign!
You've restored balance to the force.
Linode comin in clutch
Hey Wendell, could you do a video on using docker-compose with Cockpit or portainer, Then setup a couple of services like a web server, with a jwilder letsencrypt docker reverse proxy and maybe a password and bookmark server like passbolt and xbrowsersync on a little low powered device like say an intel nuc? Would love to see your take on this and most importantly how to backup data on services running in docker containers. Thanks :)
I missed this channel
Are there any "cheap" NAS-like cases in the server area with more than four HDD slots on the front like the ones QNAP uses with their bigger models? I need something like that for my planned DIY NAS.
Yes, please do a HA video for Proxmox. I found it very wierd with Ceph and the wierd units you get. I'm never sure how much usable space I actually have. Also, the quorum can be problematic sometimes.
For the determinism slider, if you want the highest performance possible you will want to choose Power, not Performance. I know it sounds counterintuitive, but check out AMD's Power Performance Determinism whitepaper, or some of their tuning guides which provide a better understanding of how Power Deterministic allows you to extract the maximum performance from your processor.
Hey Wendle, why didn't you go with a virti-io drive type at 15:43?
A couple of things to be wary about Proxmox and HA - live migration often isn't supported between nodes that are running different major versions of Proxmox, which is a complete pain if you're trying to avoid downtime during a major update of Proxmox. I've even had a few *minor* release differences of Proxmox fail to live migrate between them, which is even more maddening.
Another HA gotcha is that Linux VMs survive only about 25 seconds without their filestore being present before it goes read only and the only way to fix it is to reboot (or power cycle if it's really bad) the VM. So your shared filestore has to also be configured for redundancy, via whatever your preferred replication method is.
This is fairly interesting. My limited understanding thought unRAID might be a better solution, but after looking a little more at it, only if storage configurability is a higher priority while the VM features were added later. So proxmox would be great for the host, it looks like, if I build a homelab.
HA on ProxMox is very appealing to me, especially considering some of the features that ProxMox enables without really expensive licensing
What are the pros and cons of Proxmox over XCP-NG .
Most importantly , will it let me import a 48 disk freenas pool ?
I tried importing from freenas to proxmox without luck. Different feature set, so it was listed as incompatible. Could've changed now though, this was a year ago.
Proxmox is finicky with hard drives. If there is ANY partition data left on the hard drive, Proxmox will not display it in the gui at all. You’ll have to then use the command line to format it completely before Proxmox will recognize it. There’s no place in the gui to format drives that have partition data so it can be confusing when you have so many drives and none show up because they have a single photo on it. I know because I migrated from freenas also
You could run freenas in a vm and pass though all the drives. Craft Computing did a video on it.
I've moved from FreeNAS to Proxmox without issues. But I was rather conservative in choosing flags for my FreeNAS pool and I've never run "zpool upgrade".
Fortunately, the lastest Proxmox update added ZFS on Linux 0.8.3 support, so it probably supports more features now.
Please as many of these proxmox videos as possible. I want to see if it would be feasible to replace my standard server solution of Windows Hyper-V and Veeam for backups, with proxmox and ZFS for hypervisor and snapshot capabilities...but need to find a reliable multi-tenant solution for backups that is as easy to deploy and monitor and recover from as Veeam
Some PowerShell SCM or Ansible vid? You already did chocolatey so those would be great addition. As for proxmox i use it at 2 servers on my company and as NAS at home. Ive noticed that community repositories for paid repos become more and more like Fedora is for Red Hat - testing ground for updates. So anybody who wants to try proxmox needs to drop that illusion that its for free. Because the price here is stability.
Why under Hard disk section during creating the VM you didn t check Discard (Also if you had enabled the advanced option it would give the ssd emulation option)?
Last at System section Qemu manager shouldnt it be checked also for better understanding of the host of the guest h/w needs.
I'd be trying yo get work to look at eypcs from using dell Intel hypervisiors currently. With the cluster storage; guessing it's using iscis with network raid I.e raid 10 etc? Similar to vmware vsan perhaps?
How do I setup a really dense NVME server. I'm thinking of using a workstation motherboard with threadripepr and lots of hyper m.2 v2 then use it as a large postgresql database server with ECC ram. Not sure if this is the right way to just splash money at supermicro for stuff like this
Could you please do a review of the FreeWrite modern typewriter ? Maybe you could get them to ship you a testing version of their upcoming secret 3rd generation model ??
I would like to get some useful power consumption numbers... especially in idle/quasi idle status.
Hi nice work can you help me ....i am using proxmox 6.2 with 4 window n 2 linux guest...i wonder can i fix a corn job for window like i need 4 window up at 10am each day n shutdown at 7pm....
getting started with docker swarms on top of proxmox
Isn't writeback cache considered unreliable when compared to writethrough? What happens on the system when a SSD dies? How does it recover?
Useful!
Can you provision VMs in Proxmox using Vagrant or Terraform?
Proxmox clusters and ceph storage setup tutorial please!
You can play with Proxmox in VirtualBox 6, as it supports nested virtualization.
You can play with Proxmox in Proxmox too 😂
So are there limits with Proxmox the number of servers? Say 200 or more?
could you make a video on how to setup a high availabilty cluster with replicated storage
We've just done a large AWS migration of 95% of our infrastructure, but we're aiming to buy some new stuff for the remaining services. I'm definitely trying to get the management to purchase some Epyc hardware. We've been talking about VMWare licensing again, but Proxmox looks really awesome.
Quick question, does Proxmox support iSCSI targets for backend storage? If so, we might be in business!
Remember that VMware is changing their licenses and limiting them to 32 cores packs. So if you were to buy a 64C EPYC, you'd need 2 licenses. Tho comparing to Intel, it's still probably worth it.
@@LtdJorge That's definitely good info to know about. It's annoying companies like Microsoft and VMWare are going to a core/socket licensing model to get more bucks.
@@chbrules yep. And VMware products are definitely not cheap, to begin with.
Yes, it does support. We are able to run a 3-node Proxmox cluster with iscsi storage backend running off a 12-bay nas
Do that proxmox cluster! Please?! D:
Coming from an embedded software engineering world, this talk of virtual hardware is bewildering to me. I'd need someone to hand-hold me to walk through brutally open source systems with no GUI whatsoever, until the VM is ready to boot.
More Linux please :)
Finally!!!!!!!
this is so interesting
Can you run 128 instances of PrimeGrid on it?
01:17 Instant Like the moment i realize that Wendell has constructed a new technological terror .... (please mate, don't turn into Palpatine)
Does anybody know if nVidias Quadro P2000 works on Proxmox with vGPU as cheap solution to run multiple VMs with graphics acceleration?
please share a screen of htop with 128 cores 256 threads
😂 Just did a Proxmox install this morning
what about RHEV?
I wonder does Mörton Högarth in Denmark know that he could get a EPYC Data center unit.
How about Red Hat Virtualization?
What are your thoughts on GPU pass through on proxmox?
Having done it myself recently, I can say it works quite well performance wise. Once you get past the initial process of blacklisting so that the host isn't trying to use the device, its pretty simple to pass through, but you then lose the ability to interface with the VM in the browser. Nvidia consumer geforce gpu's are a separate problem that can be worked around. I run my test Proxmox drive headless, with an autostarting Windows VM with GPU pass through.
@@blkspade23 is it an am4 motherboard?
@@nutterztube Yes X470.
@@blkspade23 which one?
@@nutterztube Asus ROG STRIX X470-F GAMING
Exactly how does Proxmox allocate cores to the physical CPU? With KVM I can do CPU Pinning to tune performance. How does Proxmox handle this level of tuning?
Imo, proxmox does not do this, it is the underlying Hypervisor that does it. In this case KVM.
Proxmox vs TrueNAS?
i really want to do a make buildworld -j 128 in freebsd with this...
We’re planning a massive server deployment with a group running processing digital interactive content processing and markup.
what about xpc-ng ?
Been running FreeNAS, Windows 10, Pi-Hole and a UniFi controller on ProxMox for awhile and don’t ever plan on going back.
Wendell you're close but this isn't my KVM 3950x lab build 😉
Prefer KVM with Arch Linux and many self organization. So I know where all necessary points are, can setup everything in the way I want and I'm not limited or dependent on proxmox (packages, kernel etc.)
Proxmox is KVM
My single application workload will DEFINITELY access and/or address 128 cores without any problems. As I've explained to my dad, in what I do, there is and there will NEVER be a shortage of cores, storage space, RAM, etc.
I will always be able to suck that up and demand twice/ten/hundred/thousand times more because we just keep making our problems/cases more and more detailed.
That being said - I play with Proxmox on an old HP Z420 workstation and it works great with Linux, but trying to run Windows 7 in graphical mode was dog slow.
I don't have that kind of an issue when I tried Oracle's VirtualBox.
Proxmox appears to be great if you're running stuff without GUIs, it can be blazing fast.
But I also think that virtualization really doesn't have much of a use outside of the datacenter, and not even for SMB/SMEs. Yes, you can consolidate racks of equipment into a few servers, but this presumes that you even have racks of servers to consolidate initially.
For most "normal" users, that's probably less likely to be the case.
What's also less likely is that most "normal" users will have multiples of this type of server running (for HA, as described) in order to consolidate their half-dozen (or so/or more) towers into a rack server and even then, you still need a desktop to be able to log in or open up the console to be able to actually do stuff from the Proxmox virtualized compute environment.
And then there's this:
"Don't use ZFS. It's that simple. It was always more of a buzzword than anything else, I feel, and the licensing issues just make it a non-starter for me.
The benchmarks I've seen do not make ZFS look all that great." - Linus Torvalds
Linux Torvalds has no idea what he is talking about when it comes to ZFS.
@@andljoy
You're more than welcome to review the benchmark results performed by Michael Larabel on his website phoronix.com.
You can search for ZFS and review all of the published benchmark results there and see for yourself.
If performance is all you care about, I wouldn't choose ZFS. If you care about certain features and reliability / data integrity, ZFS may be a fine choice. Saying that _nobody_ should use ZFS just makes you sound like a crank (even if you're Linus Torvalds).
@@vonkruel
In regards to reliability/data integrity - it depends on which specific feature you are referring to.
If it is about data checksums, there's ZFS, Btrfs, ReFS, SquashFS that will do that (in addition to HAMMER and NOVA).
If you're talking about, for example, copy-on-write, even XFS can do that (by request).
Even for data reliability though, ZFS would NOT be my preferred choice because ZFS cares a LOT about the UUID that it writes to the block devices such that if you scramble its position, it WILL put the pool into a degraded state and depending on how you have set it up, it can also completely destroy your pool.
Per the feedback that I received from Sun Microsystem's own kernel development engineers, it is ALWAYS recommended that you have backups (usually referring to tape backups) of your system/data when you are using ZFS because it CAN catastrohpically fail like that.
And let's be real here - MOST people who are deploying ZFS in their home lab, AREN'T going to be dropping 5 grand on a LTO-8 tape drive backup system so that they can back up upto 120 TB over 10 tapes.
This means that MOST people who are deploying ZFS DON'T have the most fundamental and critical element, as recommended by Sun Microsystem's own kernel development support engineers - make sure you have backup(s).
From Linus' perspective, he would be right due to the licensing issue and HIS own inability to maintain due to the LEGAL differences that arise from the use of the different licenses.
I would HIGHLY encourage and advise that people should read his entire comments especially given the fact that Linus references the CURRENT Google v. Oracle America case which is going to the US Supreme Court and therefore; the LAST thing that Linus would want is for Oracle to sue Linus and the Linux Foundation if Oracle wants to make a pass at COMPLETELY quashing the ENTIRE Linux OS completely.
Therefore; to recapitulate:
ZFS is:
1) Not the greatest for performance
2) Not the greatest for reliability (you can watch other UA-cam videos of people trying/struggling to fix their zpools)
3) Most people who are playing around with deploying ZFS DON'T have the requisite data retention/backup infrastructure in place in order to be able to properly and appropriately tolerate a completely failed ZFS pool/array.
4) Oracle is highly litigious.
Windows 7 being stupidly slow is Windows Update. It got broken badly about 18 months ago and it just sucks CPU and disk IO. It does it on real hardware too, and other virtualisation systems. Server 2008 R2 does the same thing.
Disabling the various Windows Update services brings it back to sane speeds..... And since there's no more updates now anyway, it doesn't make much difference.
So you can run docker containers directly on proxmox?
I don't believe so. I mean not without setting it a up yourself like on any other Linux distro.
I wish I could afford this kind of stuff.
Music is too loud!
Have you found a way to override the cTDP limit beyond cTDPMax listed here?
www.storagereview.com/amd_epyc_rome_7002_series_launched
Having no windows legacy in your infrastructure feels amazing.
We *had* a Proxmox setup. It was *replaced* - together with other systems and virtualisation platforms, by *XCP-ng v8,* the free XenServer alternative, full name is *Xen Cloud Platform.* And we're using AMD EPYC, currently 24-core Zen1. Our setup includes a frontend system (firewall, monitoring), 3 main server hosts in one XCP-ng Pool and a backup system. Reasons for XCP-ng: Free, fast, easy, lots of enterprise level features and high stability.
More about XCP-ng here: xcp-ng.org/
Isn't Proxmox free, fast, easy, lots of enterprise level features and have high stability?
@@TheLovinator True, but the features are not identical. For example, both platforms can live-migrate a running VM between hosts, if they are organized in a server cluster. XCP.-ng v8 (or the Citrix equivalent, if you want to pay a lot) can do this even when local storage is involved (gets live-migrated along with the VM). Also, XCP-ng v8 supports VM migration between different pools (XCP-term for cluster).
@@omsi-fanmark Note that Proxmox 6 now supports live migration of VMs that use local storage, but obviously it could take a long time depending on the size of the VM and the speed of your network/disks.
@@rklrkl64 Sounds good. We are using local NVMe/U.2 storage and the first (and only) migration we needed was surprisingly fast, seemed even a bit faster than the pure network speed. May have been due to on-the-fly compression. Didnˋt notice any service slowdown either, but load wasnˋt high anyway. . Link is a single 1 GBit/s. So expected ~ 120 MB/sek.
proxmox vs unraid ?
You are among the people whose "aliens" will use their UA-cam content to identify humans.
Thank you
Gimme dat ProoxMoox 😎
Nutanix should add these servers to their HCL...
"Merely hundreds" of vms, hehe.
commenting for youtubes algorithim
I'd just like to interject for a moment. What you're referring to as Level1Linux, is in fact, GNU/Level1Linux, or as I've recently taken to calling it, GNU plus Level1Linux. Level1Linux is not a youtube channel unto itself, but rather another free component of a fully functioning GNU channel made useful by the GNU corelibs, shell utilities and vital channel components comprising a full channel as defined by POSIX.
Many youtube viewers watch a modified version of the GNU channel every day, without realizing it. Through a peculiar turn of events, the version of GNU which is widely used today is often called Level1Linux, and many of its viewers are not aware that it is basically the GNU channel, developed by the GNU Project.
There really is a Level1Linux, and these people are watching it, but it is just a part of the channel they watch. Level1Linux is the kernel: the video in the channel that allocates the channel's resources to the other videos that you watch. The kernel is an essential video of a youtube channel, but useless by itself; it can only entertain in the context of a complete youtube channel. Level1Linux is normally viewed in combination with the GNU youtube channel: the whole channel is basically GNU with Level1Linux added, or GNU/Level1Linux. All the so-called Level1Linux videos are really videos of GNU/Level1Linux!
I thought this channel was dead.
16:30 Imagine how fast it would be if you weren't using godawful RAIDZ1. Pool of mirrors, my man! Pool of mirrors! :)
I rather SPARC M8 (32 core, 256 threads CPU) ... With Solaris ;)
As great as this is, Zen3 Epyc (Milan) is only 6 months? away... should be even better than Rome !
milan ...you can buy dell poweredge with epyc milan 32 core now