Migration Madness: Virtual Machines, Containers, Pain!
Вставка
- Опубліковано 5 чер 2024
- === Links ===
Scott's video on getting Incus installed and setup step by step:
• Incus Containers Step ...
Scott's video on setting up LXConsole (a Web GUI for Incus)
• LXConsole Web Interfac...
Scott's video on creating custom images:
• LXD Custom Images
Other Tools I used in this video
ZFS Utils for Linux - packages.ubuntu.com/search?ke...
Webmin - webmin.com/
PiKVM - pikvm.org/
Get the AwesomeOpenSource Merchandise
awesomeopensource.creator-spr...
Support my Channel and ongoing efforts through Patreon:
/ awesomeopensource
Buy Me a Coffee or Beer
paypal.me/BrianMcGonagill?cou...
=== Timestamps ===
00:00 Beginning
00:09 Introduction to Why I'm Migrating Everything
01:45 The Hrdwaare I'm Migrating From and To
05:23 Thank you to my Patrons over at Patreon, to my Coffee Buyers, and my subscribers on UA-cam
06:35 Looking at my new setup
07:35 Webmin - Linux server admin GUI
09:00 Incus (the Forked open source LXD containerization)
12:55 Proxmox Cluster Migrations are Pretty Easy, but
14:40 LXConsole, a web user interface for Incus
17:45 All of my Instances (VMs and Containers)
19:25 Configuration, the console, and Exec in LXConsole
22:25 Create a new Incus Container in the LXConsole
26:18 Everything I'm moving, and how the Migration went
=== Contact ===
Twitter: @mickintx
Telegram: @MickInTx
Mastodon: @MickInTx@fosstodon.org
Try out SSDNodes VPS Services! Amazing Specs for incredibly low costs. I'm running a 32 GB RAM / $ CPU Server for only $9 a month! Seriously. FOr long term server usage, this is the way to go!
www.ssdnodes.com/manage/aff.p...
Get a $50.00 credit for Digital Ocean by signing up with this link:
m.do.co/c/a6a61ae55242
Use Hover as your Domain Name Registrar to get some great control over you domains / sub-domains:
hover.com/SHPaiirr
Support my Channel and ongoing efforts through Patreon:
/ awesomeopensource
What does the money go to?
To Pay for Digital Ocean droplets, donations to open source projects I feature, any hardware I may need to purchase for future episodes (which I will then give to a subscriber in a drawing or contest).
=== Attributions ===
Intro and Outro music provided by www.bensound.com - Наука та технологія
Every once in a while you come across something that you've never heard about, and your brain goes POP! This is one of those moments!
There are so many vids about setting stuff up, not much about maintaining and migrating, so props to that.
As always, awesome presentation.
Super glad you found it useful!
yep, the Scotti-Byte channel is definitely the go to for learning LXD/Incus. This is where I started, his videos are so clear and easy to follow.
Scott is a rockstar when it comes to infrastructure and LXC, LXD, and Incus.
The last time I looked at an Webmin instance must have been about 15-20 years ago and looked in the PHP era very different. Back then it was considered to be not very security minded as in being unsafe to use on production stuff!
These days it uses your auth for the machine, so still recommend not running it outside the LAN unless it’s over a VPN. That said it’s a superb tool.
@AwesomeOpenSource Ah you can have proper LDAP and SSO workflow.
If you're going to use root everywhere it's same as using root in normal ops and security issues are inherited as well. Same applies to other users and similar risks that comes with ssh access.
In nutshell it can be as good as it's system admins capability to harden the system. Assuming it's a design flaw is not something you do when you're free to config it for very different usecases having different security requirements.
Indeed. SSO rocks, and I'll eventually get it setup on my Webmin, but for now I keep it local only, and I never allow root login if posssible.
When you migrated to different hardware on your second node did you change any of the VM or LXC options such as CPU type, virtio driver settings etc... I noticed performance improvements when I did so.
I didn't. Just kept it all the same. Guess it never struck me that I should do that since it was virtualized.
Since setting up 3x nodes Proxmox cluster out of N100 mini PCs - migration between nodes is a breeze. And if I need to do something on one of the nodes - other two will pick up and run all VMS while 3rd one is out of action. As soon as 3rd node is back up - VMs and LXCs getting. Oved back (HA and groups taking care of that automagicaly)
That's awesome. Proxmox is a completely amazing open source system for sure!
Just picked up a minisforum ms-01 here and about to start the process of migrating my little nuc I have running my lab over to it.
That's awesome. Minisforum seems to be a a pretty great maker of mini-pcs for sure. I'm always looking to see what I can find that's a good buy for the buck.
webmin!!! Its been a long time since I last used it. Is there any advancement / innovation in order to deploy zfs raid on it or still with mdadm method? From cli I m sure it can be done, but in GUI there would be no info at all.
So you moved from proxmox -> proxmox or from proxmox -> ubuntu / incus. Why would you do that?
Not sure there's much in the ZFS / RAID arena, but it's still a really nice setup.
Why are you running separated lxc containers for each docker app? Is there any purpose for that?
You definitely could run multiple apps inside one LXC container. One huge benefit of going this route is to avoid Docker NAT with IPv6.
@@PopularWebz I know that and if you want to avoid docker NAT in docker bridged network, just use macvlan or ipvlan network. But I'm just wondering why he is running multiple lxc containers with docker engine, each for one app. You can run multiple apps on one docker engine inside only one lxc container.
My bigger thinking on nit is what services I'm willing to have be down when I have to reboot a container. For instance my pi-hole containers are always separate from the others, and my Nextcloud isn't tied to my vaultwarden, authentic is separate etc. So if I need to mess with one container, the others aren't affected (in theory).
@@AwesomeOpenSource but you can do this inside one docker engine, just separate their networks. And if you want one of them to have own IP adress in your subnet instead of using NAT inside Docker bridge, then use macvlan or ipvlan for exposed container and bridge for entire backend
I agree for network separation that works, but what I mean is if I need to reboot the host, in this case the Incus container, for kernel updates, or other updates, then the single docker application inside that host is all that is down while it reboots. If I have all of them inside a single host, it becomes a little more difficult when rebooting the host. Doesn't matter at all of course when the main host PC has to be rebooted, because they all go down, but for now, just separating concerns a bit.
I also had this problem with the Proxmox 8.1 iso. I knew that Debian usually upgrades really nicely so my solution was to install the previous version of Proxmox and then update.
I wanted to do this too, but I couldn't access the 8.0 iso anymore, even from their FTP style index site of images. I thought about 7.4, then upgrading that way, but just decided to try Debian with PVE on top. Just didn't work quite like I hoped, and it gave me a kick I needed to check out Incus, which for my home lab is working super well.
VMs and docker containers on VMs are easy to backup or migrate because all data and mac address etc are all enclosed , the problem is with LXC container which most of the data is on the server it's self.
This is definitely true of my Incus setup right now. I moved everything to the 1st mini pc and got it all up and running again, then re-installed the Mac Pro with the same setup using Ubuntu 22.04 and Incus, and moving some of the containers off of the 1st one was as simple as export container to tar.gz, rsync it over, and import on new machine. All up and running with all the same IPs and settings as I set it up with.
I trust, if you had replicated your storage per container/VM you could have migrated it without any problems?
Ditto on replicating the VMs and CTs. Learned the hard way when we disabled / removed replication (works if your disks are zfs only) before migrating some 1.5tb VMs. Took 1.5hrs so move across, vs minutes when the data is pre replicated. Takes less resources also. We go to 98% on 80 cores and load above 150 during the migrations without a pre replicated volume on the target node.
Great tip for the future, but definitely had not replicated my storage. I did have it setup in ZFS though.
Theres still so much to cover regarding Open Source in business use...... also noone seems to care except one state here in germany whos going all out on it rn migrating 30K PCs to LibreOffice...... Lets see how long it takes till they go back or how long till someone gets bribed by Microsoft to go back......
Noone has really done an in-depth video yet how to manage Linux systems on a big scale in a business, user management, update management, permission management, logkeeping, compliance, security u name it......
That would be a whole series all it's own, no doubt. I would assume something like RedHat EL or Ubuntu would come into play, perhaps Suse Linux with their Enterprise offering for really large scale deployments and all. Excited to see that place in Germany succeed. I know it was tried in the past and sadly reversed, but hopefully they'll get through it this time.
@@AwesomeOpenSource Still noone seems to care and left and right companies get themselves stuck in big tech cloud services handing control over their data to random people and governments......Overpriced, incompliant and vendor lock-in...... like seriously the newest Outlook app sends the login credentials and mail content to Microsoft......
I wish id have a homelab at times but time and well 50 MBit VDSL doesnt get you far id have to get fiber first and that would cost quite a bit......
Maybe you could tho not do a series but build out a business or Enterprise grade network in theory what it could look like and what software might be used for it
Hey. That’s a great idea. If you really wanna do this my friend @AwesomeOpenSource, I can serve you a bare metal server here in Brazil with a Public IP and any OS of your choice so you can present a real world scenario with world wide connectivity. If you want to tell me how to reach you out.
I'll think on it a bit, and see what I can come up with.
I don't see the advantage using lxd over lxc. I am not ready nor convinced that using lxd is better. I am also not ready to switch from an enterprise rated machine to some consumer product. I like my power hungry machines and their reliability.
I completely understand this. For me, it's my goal to cut down my power usage this year. Being in Texas, and particularly south Texas, it's very hot here most of the time. My 3 Dell Poweredge servers generated a ton of heat, and used a ton of power, which in turn made my Air Conditioner run harder trying to keep the room cool (even in the winter). So to move to less power hungry hardware that generates overall ess heat by a lot, is going to help in several ways. Everything is now moved after about 10 days of slowly getting a little done at a time, and everything is running well.
incus has gotten some buzz lately
It's pretty solid, and really nice.
Now you got to buy more servers to fill out that screen.
Little by little maybe.