Virtualizing OPNsense on Proxmox as Your Primary Router
Вставка
- Опубліковано 17 кві 2024
- Are you curious how to set up OPNsense on Proxmox as your primary router? In this video, I show one way you could go about doing it using the Protectli VP6650, but you may adapt this guide to the hardware you wish to use.
This guide assumes you have an existing network and that you are planning to migrate to a virtualized instance of OPNsense. The guide also does not do a full network configuration in OPNsense in an effort to keep the video shorter. Once you have OPNsense up and running, you can use other guides I have created to configure OPNsense for your network.
The focus of this video is getting Proxmox configured properly, creating the OPNsense virtual machine, and showing how to physically connect everything to your network. You must be careful to not conflict with your existing network once you have the OPNsense VM set up (conflicting IP addresses, multiple DHCP services running on the same network, etc).
I recommend disconnecting the Proxmox server from your network or connecting it to a dedicated VLAN with different IPs to avoid issues with your existing network if you wish to do more testing before swapping out hardware.
See also the addendum video • Addendum: Virtualizing... where I demonstrate a few things based on some feedback and questions that I have received.
For a written version of this guide, please visit:
homenetworkguy.com/how-to/vir...
Hardware used in the demonstration (affiliate links):
* Protectli VP6650 mini-PC: amzn.to/49NVFXP
* Grandstream GWN7806 (non-PoE) switch: amzn.to/3PTdWvl (link for the GWN7806P PoE version)
* ZimaBoard 832: amzn.to/4ax8xCw)
* TinyPilot 2a: tinypilotkvm.com/?ref=homenet... (for screen captures)
Chapters:
01:32 Physical connections
05:36 Configure Proxmox server
20:54 Create OPNsense VM
35:54 Demostration of assigning VMs/CTs to VLANs
EP42 - Наука та технологія
Finally, a video that shows what the REAL WORLD looks like, and takes it from step ZERO. Well done
Thanks! I have more real world examples coming up soon! In fact, most of my guides are based on real world examples (I like to base them on real examples that I have done for my own home network either currently or in the past and sometimes I create examples in a lab environment to try new things and to verify the process works properly).
Thank you for showing the physical real world implementation. As a beginner, I've always struggled with this and this is the only video that shows from "zero to one to 100"
I’m glad that was helpful to show all those steps! I’m going to doing the same when showing how to set up a basic 3 node Proxmox cluster soon.
Using OPNsense for years, I never knew you could delete the interface which holds the vlans. Nice video. 👍🏼
Yeah you can if you don’t plan to use the untagged parent interface. Since I use a different interface for the LAN for untagged traffic, I don’t need a second untagged interface and just only need to use VLANs on that second interface for just tagged traffic.
I will note one potential gotcha that I encountered when testing out some things. If you want the VLAN interfaces to use a MTU that is higher than the default 1500 used by all interfaces (to enable jumbo frames with MTU of 9000, for instance), you will need to have the parent interface assigned and enabled so that you can set the MTU value on the parent interface. This is likely a rare scenario since typically jumbo frames are used on isolated networks with higher speed interfaces (10Gbps+) rather that for routing traffic across 2 networks with larger frame/packet sizes.
Fantastic video. I learned a ton watching and following along. Thank you so much. I appreciated you walking through each option and briefly discussing why or why not you had chosen said option. Cheers!
Thanks! Glad you liked it! I think it’s helpful to explain the options instead of just picking them. I tend to go in more detail in written guides on my website. I have to be a little more terse in videos to try to stay on topic and keep the length shorter.
Excited to watch in full, now, for learning and entertainment.... Already saved to watch again as a guide
Thanks! I hope I covered enough to help people along. It’s a lot of info to cover (and there could be even more but I tried to keep the length somewhat reasonable). Takes a lot of time to produce content in general, let alone during your limited free time. Haha.
Perfect timing on this. This is exactly how I plan to setup the mini PC that is out for delivery right now. :)
Sweet! I love it when it's perfect timing for my subscribers (and others). Someone else said it was also perfect timing earlier today.
Fantastic that you release this video literally the day i get everything together to do exactly this myself, you also helped me with the PCI pass through that nobody else talks about. Thankyou!
That's great! I'm glad the timing worked out. Sometimes I'm just in time for some users and too late for others. haha. I thought I would mention PCI passthrough in the video even though I didn't do it in the video to keep things a bit simple but I also tried to ensure that the instructions should still work if you plan to use a Proxmox cluster. Things get more complicated when doing PCI passthrough with a cluster. I have yet to try all that out as well. Bridges are safer and you will only notice performance issues with 10G interfaces or faster. You can still get 5-6Gbps with the VP6650 I used in the video so it's still faster than the 2.5G interfaces (and really you should try to not route 10G NAS and other traffic when possible to reduce the load on the firewall by having a separate 10G network).
Maybe use the managed switch and create a WAN subnet using a VLAN 🤔 connect the WAN cable to the switch and then any Proxmox node can access the Internet VLAN for a virtual bridge ?!? Just a thought. Might be more complex using the newest SDN feature on Proxmox . . . Guess it's time to experiment around a bit . . . Great work 👍🏻👍🏻
Just when I needed the video, no excellent info available on UA-cam IMHO, this _is_ _great_
Thanks! I hope it has enough info to get started because there is a lot of information to cover. I tried to keep it focused on the topic at hand.
Brilliant work. I'm building my own home network and your guides are excellent.
Thanks! I hope they help you along the way! I have been evolving my network for many years (more so in the last 6 years).
@@homenetworkguy , do you have any thoughts on IPFire ? For example, can I use it to achieve something similar to your "opnsense for beginner" video/post ?
I have thought about learning more about other firewalls (OpenWRT, IPFire, etc) once I have exhausted the main topics I want to cover in OPNsense but after writing on my website for nearly 6 years (and more recently, UA-cam videos), I still haven't exhausted everything I'd like to learn about. haha.
I think IPFire could be a good Linux based alternative. There are a lot of similar features but also some things it doesn't offer via plugins. I would like to test out the performance of it because it's possible Linux could perform better than FreeBSD depending on driver support, etc.
This is the video that gave me the reassurance to switch my own home network over from firewalla to a virtualized OPNsense instance this past weekend. It genuinely surprised me that it was a clean cutover with all of my vlans/APs, Thank you!
OPNsense has 4 Performance cores of a 14700T, 32GBs of ram and a bridged Intel X550 T2 dedicated.
Great to hear! Glad it gave you reassurance! Make sure you have a good backup plan if you only have one Proxmox server (but even if you have a bare metal installation, it’s good to have a backup plan). If all is configured properly the virtualized instance should function essentially the same as bare metal as you have discovered!
thanks for all the opnsense and proxmox content. As a opnsense / Truenas scale home user and a vmware enterprise user @ work i enjoy all this content. Proxmox and ncp-ng are in our work test labs for possible move to from vmware. Thank you again!
Thanks! I'm glad you appreciate it! I hope to dig more into Proxmox clustering with OPNsense and how I think I'm going to go about it on my home network so that I can do live migrations (it will be very awesome to have the ability to move my main router/firewall over to a different physical machine with only a split second blip in downtime for my network!). I don't care about high availability/failover as much as being able to live migrate the VMs (because with VMs it's easy to restore from a backup from my PBS system, which is another nice piece of software). The configuration and requirements for live migrations is less intense which I think will suite my needs perfectly.
This video shows step by step installation Proxmox to OPnsense.
Thanks
You’re welcome! I plan to into configuring a Proxmox cluster soon and show how you can live migrate OPNsense to different nodes with very minimal downtime.
I lust got my PVE/ OPNSense machine running and in my rack a couple days ago, and I just found this today! I also used your Pi Hole PVE guide and set that as my DSN server. I used an 8th gen Dell OptiPlex with a dual 2.5gb card, and am thinking of setting up a second machine for a HA cluster.
Nice! If you set up a cluster with 2 nodes, you need to make sure you have a 3rd device as a “Q” device (a 3rd voting member) so you can have quorum. You need an odd number of devices so you can reliably know which nodes are available.
@@homenetworkguy Good to know!
OMG right in time ... Thank you a lot :)
Love it when the content is release just in time!
I use this on my server in the datacenter. Works perfect!
Following my instructions or you already have an OPNsense VM in your datacenter? Either way, that’s awesome!
@@homenetworkguy In production for +-1,5 years. I can also acces ipmi with a vpn that is not running on the server ;-).
Thanks for the content. Playing with some similar setup on mini pc's right now.
You’re welcome! Have fun!
well done, many thanks for this great video. Will help me a lot on the way to my first home-lab :)
Great that you found it helpful in creating your first homelab!
This is an awesome video. I am trying to learn about this stuff so I can do it in a few months after a move.
Thanks! I’m planning to expand upon this and show clustering in Proxmox. I will demonstrate how to manually live migrate the VM to another Proxmox node as well.
I didn't know you could do raw passthrough on PCI devices without Iommu. That's cool. 👍
Yep you still need to have virtualization features enabled in the BIOS but if you don’t also enable IOMMU on Proxmox, only raw device pasthrough is available.
Excellent video 👍🏻 I needed this 6 months ago (figured it out the hard way!) 🙄 Have a smoothly running virtual opnsense on an R86s for some time now 👍🏻 quick question, I have a cluster of nodes and want a fallback scenario in case main node with opnsense dies - how would you propose moving the virtual instance to a different node and still keep network settings?!? 🤔 Might make for a great follow up video ?!? 👍🏻 keep up the great work . . .
Thanks! I’m planning to show I will do this in a cluster. With the limited research I’ve done, you would want to ensure the bridge names are the same on both nodes so the 2 machines would need to be configured similarly in that regard. Also if you’re not using shared storage, you would need to restore from a backup (and there might be a step to “manually migrate” the VM to a different node by messing with the config files since the VM wasn’t migrated while the node was still alive- not sure about that one yet until I try it out and/or do more research).
Thank you!
You’re welcome!
Fire 🔥!
Haha, thanks! Took a bit of effort some to get it made but my favorite videos are real world examples pulling multiple concepts together.
@@homenetworkguy man this was perfect and honestly I appreciate the content it’s helpful for poeple who want to try this and the examples and explanations is perfect for beginners . Will be showing my friend as well who’s trying this to
Nice video.
Thanks!
You should enable "Discard" (for trim) for thin-provisioning to work properly. If you disable "Pre-Enroll keys" then Secure Boot won't be enabled so there's no need to disable it later.
OPNsense (and pfSense) recommend to disable all off-loading settings. At least for virtual NICs.
Thanks for those tips! I should’ve looked up Discard to better understand if it was necessary or not.
Funny thing is that the pfSense documentation shows to do it that way for disabling Secure Boot (docs.netgate.com/pfsense/en/latest/recipes/virtualize-proxmox-ve.html#booting-uefi). I figured their docs would also work fine for OPNsense for recommendations for VMs. Since I always use OPNsense in a VM for demo/testing purposes I didn’t care about optimal settings as much but if I use it as my main router/firewall, it becomes more important!
Hardware offloading is disabled by default in OPNsense which is why I never typically mention doing it. I think for pfSense it may be enabled by default.
@@homenetworkguy Yeah that's funny but one can't know everything. The PVE docs (I apparently can't link things without the comment being deleted) say this
> pre-enroll-keys specifies if the efidisk should come pre-loaded with distribution-specific and Microsoft Standard Secure Boot keys. It also enables Secure Boot by default (though it can still be disabled in the OVMF menu within the VM).
To elaborate on the discard as far as I understand it. On most linux OSs there's a weekly "fstrim" timer which calls "fstrim" which gives unused chunks back to the underlying storage. Assuming the virtual disk is on thin-allocated storage and "Discard" is enabled, of course. I believe windows also needs the "SSD emulation" option. I'm not sure how pfSense/OPNsense/FreeBSD handle trimming. I'm very far from an expert with BSD. Trim seems to be disabled in my OPNsense VM according to "tunefs -p" but I'd recommend to enable "Discard" for every disk on thin-allocated storage.
Yes, I appreciate when others let me know about details such as this so I can continue to learn as well.
To clarify from what I looked into this morning-- without discard enabled, the initial VM storage doesn't take up the full 64 GB when I looked at the disk usage. It's sitting at 3GB and I have a few CTs set up as well.
However, I'm assuming discard will help free up space on the host when data is deleted from within the VM. It's good to know that it doesn't fully allocate the 64 GB even if discard is left disabled. I'm not sure how trim is handled in OPNsense either.. I think I've seen others talk about it at some point but not sure if it is something that needs to be enabled to make it function properly.
Thanks for this. I'm definitely wanting to setup Opnsense and Proxmox, I just don't know what on. I like the chassis design and ports on these Protectli units, but god they're expensive. The Minisforum MS-01 gives you a mobile i9, the same two SFP+ ports (it's even the same model of Intel NIC), two 2.5G RJ45 ports (also same model of Intel NIC), two USB 4.0 ports that can do 40 Gbps, three NVMe slots (albeit only one of them is PCIe 4.0 x4) instead of an NVMe and 2 SATA slots, for like $220 less than this. If you get the i5 version (which still has a better CPU than this one) it's $460 less. It's pretty ridiculous how expensive this thing is to only have an i5 in it. I'm not sure the extra 2 RJ45 ports, better chassis, better firmware support is worth paying so much more to lose out on hardware. It's quite a dilemma.
I only have 1 Gbps for now, so realistically I'd be fine with one of the cheap Protectli boxes if I was going with barebones Opnsense on it (aside from running ZenArmor and such), but I want to upgrade to 10G LAN at some point so I'd like to have the support for it to make routing between VLANs faster among other things.
Yeah you have to weigh the pros and cons. I definitely wouldn’t use the MS-01 as a dedicated OPNsense box. Systems like these are too powerful not to use virtualization to make full use of the hardware. Not all of the services in OPNsense take full advantage of all the cores. In fact some of them may fight for the same couple of CPU cores (the Zenarmor team has noted as much to me).
Thanks.
You’re welcome!
I've thought about doing this.
Cool! I hope it goes well if you do!
good video. One thing you touched on but did not get into is if your PVE (with OPNSense) goes down, you lose your router. It would be good to understand how you would migrate this over to a second PVE without losing routing. I suspect you would need a machine with the same number of LAN ports which have the same virtual bridge names in order for it to migrate properly. (I want to use OPNSense but I want to be able to migrate it between PVEs in a cluster).
I mentioned that you need a backup plan if you only run a single node since it will take your network down which I mentioned near the beginning about how I prefer bare metal because I’m considering using a Proxmox cluster so I will feel more comfortable about virtualizing OPNsense for my primary router/firewall. I plan to show my cluster configuration in the future. It will be pretty awesome to be able to live migrate my primary router/firewall with less than 1 second downtime!
@@homenetworkguy that would be nice, would love to see this. Especially how to do this when your provider allows only one device with the public ip
@@homenetworkguy I always double router... I keep the ISP provided router in front with family wifi. Then have a proxmox/opnsense router behind, so I have my own network I can freely break without affecting the family. Which is good because sometimes I break it a lot 😅 I've heard double router can cause problems but so far I've never faced a single issue caused by double router so not sure what that's about
@@user-ll7rk8mk4q I also play around with OPNsense VMs on a separate lab network for the same reasons. I try to keep the main network stable for my family and also because I work from home (and my wife does some work from home too). Having a separate lab network is nice because I can play around with stuff so I can make guides/videos and I don’t get tech support tickets if something breaks. Haha. But I will move to a virtualized OPNsense once I set up a Proxmox cluster because it will provide me with more redundancy so I will feel more comfortable virtualizing the main router. It will allow me to migrate to different hardware much more easily since I tinker with different mini-PCs and other hardware on a regular basis.
@@user-ll7rk8mk4q yep when you have a family, you are basicly on -call 18 hours a day 7 days a weeek, the more complicated, you have as part of the family networking, the more likely it wil break, and of course it always breaks when you are the busiest. If do this i'm going to use opensense on a virtual network that doesn't leave the system then i can have more bandwith between VMs and I can play around with rate limiting, and other firewall features.
From experience, even running on a multi node cluster with full DRS running, virtualising your firewall is not a good idea in an home lab. It sounds like a good idea, its a good project to get your head around, but just don’t do it. Save yourself a world of pain. Thats said, this is probably the best Proxmox setup video for new users I have seen.
What kind of pain? In planning to mostly keep OPNsense on one on the nodes so I can live migrate it. I’m not going to do any of the high availability features nor mess with shared storage or Ceph to keep it simple as possible. I just want to be able to move VMs between nodes if I take a node down for maintenance or if it fails. I’m not concerned with automation failover scenarios which is another reason (among other reasons) I haven’t implemented high availability with OPNsense itself.
@@homenetworkguy If anything at all goes wrong with your host infrastructure, either physically or you make with a mistake your config then you lose your connectivity. In an enterprise environment which is strictly change controlled then I am happy with virtual firewalls, but in a home environment unless you have similar controls, built and proofed in an dev environment and then rolled out to production, invariably you will make a mistake, mess up a VLAN assignment, trunk, host or the OPsense VM and then you are dead in the water as you will have no connectivity across your VLANs and no internet connectivity. That was my experience and attempting to get my environment back up and running at 4am in the morning and was not fun. It looks like those who have this working as a solid solution have a much better at home based change control than me. Love your video's BTW and thank you for this video in particular.
Yeah, I understand the need for tight control for configuration management in the enterprise, but home networks typically aren't nearly as complex so it should be easier to manage. I don't make major architecture changes very often but I plan for some down time when I do.
Also Proxmox clusters can be relatively simple and not be configured with all of the high availability features. At the bare minimum, you can simply group systems together so you can manage them all from a single UI and you can migrate VMs between them. That's mostly what I would be interested in because it's quicker than backing up VM, shutting it down, and restoring the VM on a different independent Proxmox node (if not using clustering). There is a less than 1 second cutover from what I have seen from others which is pretty sweet.
Since you mentioned DRS, you might be more familiar with the VMware world which perhaps may be more complex to configure/manage clusters (I don't have personal experience in that area).
I'm going to give a Proxmox cluster a shot soon, but I could always keep an extra box with a bare metal installation to swap out if need be. Wouldn't hurt to have a hardware backup!
DETAILS, details!
“It’s hard, complicated and error prone!” (Only for “some”).
I did run my main pfsense, plus 2 more for HA, under ESXi, for a few years and there was NO SUCH PAIN!
The main reason that I run pfsense on a dedicated machine, is because I found cheap used quad core mini PCs that work perfect.
The “people” that utter vague claims like this, usually don’t know the stuff well!
Even if you only start with one proxmox host it is advisable to create a cluster before creating the 1st VM. Not used v8 but this was the case with v6 and v7, A host with a VM cannot join a cluster.
Good point I hadn’t considered yet. I haven’t created my cluster yet but plan too soon. I can easily back everything up to my PBS system and restore it back on the cluster.
I looked into this further. The primary mode where you create the cluster can have VMs/CTs running but any new nodes that you are adding to the cluster must be empty to avoid naming conflicts between nodes. Makes sense. I was worried I would have to start over with a clean slate to create a cluster. I have backups on PBS so it’s easy enough to start over if need be.
Super movie! I only have 2 network cards in my computer. Can this be done?
Yeah you could dedicate one interface for the WAN but the other interface will have to be the Proxmox management interface, the LAN interface and if you want any additional VLANs. You could experience some bottlenecks using a single interface. The configuration will be a bit different than what I demonstrate but the concepts should be the same (you would just use the same bridge interface for all internal networks instead of separate interface(s)).
hey at first, thx for yout tutorial. i got one question. at the network config you give 4 to queues. why? can you explain int to me pls. i m new in the game and dont find a easy answer in the inet. thx
It allows the guest virtual machine to have virtual CPUs process the network traffic which can help improve throughput. According to the following link, it is recommended to set the multiqueue value only when anticipating a lot of network traffic since it increases the CPU load of the host/guest as network traffic increases: forum.proxmox.com/threads/multiqueue-inside-of-vm.66321/
@@homenetworkguy thx for your fast help 🙏
I have 8g symmetrical at home, as a non network guy, if I want to use IDS/IPS and pihole / unbound dns + wireguard. is that something Protectli VP6650 can handle. I don't know how much power you really need. I most likely won't vlan too much more of a simple router -> switch to nas and computers and then router-> 2.5 directly link to nas port for DMZ sharing
Without IDS/IPS, it shouldn’t be a problem but it can’t do IDS/IPS on OPNsense at 8 Gbps because not all of those services are fully optimized to take advantage of all the cores on the CPU. You may potentially have better luck with other operating systems. I haven’t tried other firewalls such as IPFire yet. It’s Linux based so it may perform better. I should try it before I start using the VP6650 in my future Proxmox cluster.
@@homenetworkguy would love to see it. i wish there was a chart that just said you need x for y feature somewhere, the information is always vague or refers to buying some enterprise level hardware, pretty sure my wife would not be happy to that purchase. vs something smaller one could build out ^_^
I’ve thought about creating a chart/table for the hardware I have personally tested to help others determine how much hardware they need for certain services in OPNsense. I wasn’t able to test all of the older boxes I have quite as thoroughly but it’s getting easier for me to set up test cases since I have more sponsored hardware and other hardware that I purchased available for testing.
First, thanks for the video. I think this has me most of the way there but am unsure on something. In my case, I will have Proxmox on a server colocated in a datacenter. I passed thru my NIC to OpnSense and it's booting and working. However, how do I allow Proxmox and other VMs to use OpnSense? I'll have a VPN running so I can remote in, and hopefully use the LAN IP address to still access Proxmox from afar. - thanks
You’re welcome! As for your question, I’m not sure of how many interfaces you have in the colocated server. Of it’s only one, then you cannot use passthrough because that means only the OPNsense VM can use that interface. You will have to use the default bridge interface in Proxmox. With only one interface it’s going to be tricker to set up a WAN/LAN interface but it’s possible using VLANs.
If you follow the basic principles in the video, you will be able to use the default bridge for both the Proxmox management and the OPNsense LAN interface. You simply just assign the same bridge to other VMs so they can be on the same network.
Excellent but you missed out on showing the temporary firewall rule to allow all vlan networks to see each other.
Yeah, basically I had allow all rules on every network. I had to decide if I wanted the video to be 40 minutes or 1.5 hours, etc to show a full build (which I have done twice already.. I may do a 3rd in the future as I slowly work to improve overall production quality, etc).
Is it possible to have 2 VMs of OPNsense working as Active/Passive mode? For high availability and reliability on home network? If so, how it would be look like? Would you pls consider making a video for that? You can mention me as “Abu Rayyan from Baghdad” next time 😅 never been called out on UA-cam algorithm 😂
Yes but it’s almost has value to do so especially if it’s running on the same Proxmox server. VMs are very easy to backup and restore and you can take advantage of deduplicated snapshots with Proxmox Backup Server as well to get you back up and running quickly if something goes wrong. Could make a HA video because it’s interesting to learn even though I wouldn’t personally use it especially since I only have 1 public IPv4 address.
How does the Protectli Vault Pro VP6650-6 Port do on power at idle?
I notice it uses about 20-22W but I had a couple network interfaces plugged in and I have a second disk (SSD) which would add to the base wattage. However I think that’s a good basic use case for real world wattage. It has faster single threaded performance than my Ryzen 7 1700 Proxmox server but at 1/4th the idle power consumption. It uses about twice as much power as their 4 port models but it’s also much more powerful too. I have the VP2410 and VP2420 and the two systems combined uses nearly the same power at idle as the VP6650.
How can I hide Proxmox behind an OPNsense firewall if I only have 2 Ethernet interfaces (WAN and LAN)?
Thank you!
You would have to use a bridge for the LAN interface similar to how I demonstrated in the video. It would be the same interface you use to manage your Proxmox server. You can’t use PCI passthrough on that LAN interface and also use it as the management interface for Proxmox because that interface will be dedicated to the OPNsense VM if using passthrough.
Hello, very nice video, but could you make another video about OPNsense KEA DHCP New
Thanks! I've had a few requests for Kea DHCP. I'll get around to it eventually since it will be the new way forward but currently I do not believe it is considered feature complete so I do not see myself personally migrating any time soon (but I will likely do a video on it before I make the transition on my own home network).
@@homenetworkguyI switched right after update with Kea support and it's not that hard to set up. And it's working without problem. Only con I see is no hostnames, only theirs IP adresses (in DNS server, monitored communications etc.).
Forgive my elementary level question, but how (and on which device) do you manually generate an ip address to disconnect from your LAN and continue management? Thanks!
I just used another PC from the one I did the recording (I have a couple of mini-PCs I use for demo purposes). If you only have 1 PC/laptop and 1 Proxmox server, you’ll have to temporarily connect your PC/laptop to the Proxmox server to configure it. Once you’re done and have OPNsense installed, you can connect it back to your network. I’m assuming in the video you’re using the default LAN network for both the Proxmox management and the OPNsense LAN interface. That network interface is 192.168.1.1/24 (which means usable IP addresses between 192.168.1.2-192.168.254).
@homenetworkguy: Are you giving "pve-test" the new address of 192.168.1.50?
My current network uses a 10.27.27.x scheme, but I don't know how to locate my new proxmox node when I remove it from the network. 😕
You can assign the Proxmox static IP to be whatever you want so you can make it 10.27.27.100/24 if you like (make sure it’s outside your DHCP range to avoid potential IP address conflicts). If you plan to put OPNsense and make it your primary virtualized router and you still want to use that network address, you’ll have to change the default LAN IP addresses or create another interface with the appropriate IP address ranges. I tend to keep the default LAN network of 192.168.1.1/24 since it’s keeps things simple (but I add other VLANs too, of course, with different IP ranges).
Just figured out that it's the ip address for the "stand alone" pc used to configure the proxmox device that needs the new static ip in the proxmox device's network scheme. 🤯
Thanks for all the info (and patience)!
i feel like it's not really your "primary router" if proxmox is still in front of the OPNsense router and using the wan for management, i did it today with proxmox behind OPNsense and it's much safer, just not sure how to set up the pve>system>network,DNS,certificates thing as i am absolutely new to proxmox
It is your primary router but just virtualized. You can plug your modem/ONT directly into the interface used as WAN on Proxmox just like you would on a bare metal installation plugging into the WAN interface. Proxmox is not doing any of the routing or firewalling for your network-- OPNsense in the VM is doing that task. This is the nature of virtualization. Proxmox is not "in front" of the OPNsense router. Rather, Proxmox is simply hosting the router/firewall software in a virtual machine (all routed network traffic flows through that VM just like a bare metal installation). I am currently using a bare metal installation of OPNsense, but I will probably move to a virtualized installation (in a Proxmox cluster) so that I can have more flexiblity to "move" my router to different hardware without doing a separate bare metal installation. I can just migrate it over to a different machine. Since I test out various hardware, that flexibility will be great to have.
As far as security is concerned, the main security risk with virtualization vs bare metal is escaping the VM sandbox. If an attacker can break out of the VM, they can get on the host system. Those sorts of attacks are very rare. Other than that, the security is generally pretty much the same. I understand virtualization is not for everyone. I have guides that show both bare metal and virtualized instances of OPNsense.
@@homenetworkguy did you notice that after installing opnsense and setting it up as the main proxmox router that pve>system>network,DNS,certificates etc have to be changed to match the new network?
Hey I'm new to networking and I just build my first home server. However after setting Proxmox up. I can't seem to access the webGUI using the PC to configure the creation of the OPNsense VM. I have assigned a static ip to my laptop. Any idea of what I'm missing?
Thank you!
Are you plugged directly into the Proxmox management network interface? Or connected to a network switch? You will need a static IP on your laptop only if you’re plugged directly into the Proxmox management interface. Otherwise you can use DHCP if you’re on the same network as the Proxmox management interface.
@@homenetworkguy im plugged directly into the interface. Followed your guide.
Did you configure the subnet of the static IP to be 255.255.255.0? Also make sure it’s not accidentally the same as the Proxmox IP address as well. You could try different interfaces on your Proxmox box in case you have a different one configured than the one you’re plugged into.
@homenetworkguy could I contact you on a discord or something alike to grt a bit more help. I'm really stuck and can't seem to figure out what is going wrong
I do have a Discord account. I don’t always hop on it but you could use that. Keep in mind that it’s becoming a bit more difficult to keep up with everyone’s messages. I still have a couple week backlog left in my email (I caught up on a couple weeks worth of email last night).
I currently have a topton N5105 with 4 ports 2.5gb i226v. Would I be able to do this? Been reading around reddit that people were having random crashes? Is this still the case? I currently run opnsense as bare metal. But want have snapshots/ backups for quick restore
I believe this was addressed in newer versions of Proxmox. I know many had issues with the N5105 and the N6005 but I’ve used Proxmox with the N6005 without issue several months ago.
@@homenetworkguy that’s great thank you for replying. I plan to change over to proxmox. Can I use a Ethernet adapter (2.5gb) to use for proxmox/setup and setup opnsense. So I can set up my 4 built in ports as follows: WAN, LAN 1 , LAN 2 and LAN 3
You could I suppose but keep in mind if you use bridges, you can share the same port with your Proxmox host/VMs/CTs as demonstrated in the video. You don’t necessarily have to dedicate all the ports to OPNsense (you may need to use passthrough on the N5100 to achieve 2.5Gbps but faster hardware can handle 2.5Gbps even with bridges just fine).
@@homenetworkguy I think I understand it now. I have 3 Ethernet cables from my opnsense, LAN 1= server (unraid) LAN 2= WiFi access point upstairs and LAN 3 for lounge. And last port is my WAN. So if I understand correctly I can e.g use LAN 1 to install/setup proxmox and opnsense and then have my ports work in the same way?
Yes if you use the default vmbr0 bridge that Proxmox sets up during the installation. That’s the great thing about bridges but there is a performance impact depending on your CPU and the speed of the network interface. I’ve discovered that bridging performance in Proxmox is greatly impacted by single threaded performance of the CPU.
Any idea how to show the connected devices on my network? I just switched from a off the shelf router to OPNSense, but I can't seem to figure out how to see all my devices and their IP addresses.
Under the Services > ISC DHCPv4 > Leases page, you will see a list of all devices and IP addresses of the clients using DHCP. You won’t be able to see any devices that are using static IP addresses but you should be able to see everything else.
you can also use nmap to scan your networks
I have been able to follow the instruction, install Proxmox / Opnsense and everything is working fine. I however am unable to update proxmox, and keep getting a message “download failed unable to resolve host” when trying to download Lxc for pi-hole. I feel proxmox isn’t able to access to web. What can I do to solve this. Please help.
You’re using the same bridge in Proxmox as the LAN interface in OPNsense? As long as you don’t pass through that interface which is used for Proxmox management and you have the LAN configured properly in the OPNsense VM, it should have access to the Internet just like any other device on the LAN network.
Realized my error; reinstalled proxmox / opnsense and all is well now. I guess this all is the learning process.
@@cyrilpinto418 Nice! Glad you got it resolved. Sometimes missing a minor detail can cause a problem.
Could you make deep dive OPNsense firewall video next I'm having trouble understanding the firewall. I have OPNsense running on top of Proxmox with two NICs passed trough (WAN/LAN) and VLAN interfaces (10,20,30,40,50). I'm trying to allow Proxmox hosts in ManagementVLAN10 (10.10.10.0/24) to temporarily (or permanently) access my Unraid NAS VM web GUI in ServerVLAN30 (10.10.30.0/24) but I'm having no luck with it. In the future I also need to allow Proxmox hosts in VLAN10 network to reach Unraid (in VLAN30) for NFS purposes. I'm using Mikrotik SWos switch.
The firewall just doesn't click with me. I've watched some of your OPNsense and firewall videos but I'm still struggling. It's feels like OPNsense doesn't know the routes between VLANs since the firewall rules I create seem to do nothing.
It's hard to say where the config is going wrong without seeing any of it. Perhaps you could take a look at my website which the videos are based off of for more details since there may more explanations that will help you understand it better. It does take some time to wrap your mind around firewall rules when you are new to them (at least it did for me): homenetworkguy.com/how-to/set-up-a-fully-functioning-home-network-using-opnsense/
why do you create bridges instead of passing the device through to the vm?
It depends on what you are trying to accomplish. Performance is best with passthrough but you can’t use the interfaces for anything else. With bridges you can have other VMs and CTs be on the same network by sharing the same bridged interface. It’s very flexible but there is a performance penalty. Since I’m planning to cluster it makes it easier to migrate VMs between nodes.
Just watch the video, but you did not showed how to configure firewall rules as by default opnsense block all the traffic i am also having the same setup as you showed in the video but i cant access my internet on lan network
Can you please give me some inputs here
You can create a rule on each interface to allow all access (protocol any, source any, destination any) for testing purposes.
@@homenetworkguy okay let me try it
Yoooo let me just swoop one of those $1300 mini computers 😂
May as well go buy an sonicwall TZ570w with a year of professional support for the same price.
Can you install a hypervisor on the Sonicwall? New prices seem like $3500? I’m assuming you’re referring to used hardware prices.
You could also do this guide with a $200-300 mini PC which has 4 network interfaces. It depends on what you need.
The VP6650 is faster (single threaded performance) than my old Ryzen 7 1700 Proxmox server at 1/4th the power consumption. I could easily replace my huge 4U server with the Protectli if I wanted but I’ll probably just cluster a few of my systems at some point.
@@homenetworkguy each to his own my guy. Great video and I'm sure it'll be very informative and helpful to a lot of people.
Thanks! It seems like the video is being well received by those interested in the topic.
Also, I was genuinely curious in my previous comment if you can run a hypervisor like Proxmox on it and get the device plus a year support for $1300?
I wasn’t implying the Protectli box is superior to the Sonicwall but rather it’s an apples to oranges comparison (one is a general purpose computer while the other is a firewall appliance). For a home network, having a general purpose low power mini PC is great for virtualization servers, etc.
I see the same steps on my dell mini pc but for some reason I get a no boot device found no matter what.
No boot device before or after installation? Sounds like a boot order issue possibly?
@@homenetworkguy it appears I have been using the incorrect ISO. I would download the image from Open sense straight into my Proxmox and just realized that this was some kind of a zip file. I unzipped and now uploading manually but this method might take an entire day to complete. Stay Tuned!
Ohh yeah, you will need the DVD ISO image and have it unzipped before importing into Proxmox.
@@homenetworkguy Thank you again, Second Question if I set this up as virtualized just for learning can I keep it strictly isolated to my proxmox vms and not have it manage my main network/wifi. And what would be the best setup for that scenario?
Yeah for sure. I have a couple of OPNsense VMs I use for demos/testing, etc. The main thing you need to be careful of is not putting the WAN interface of the OPNsense VM on one of your primary networks while also having LAN interfaces on the OPNsense VM using the same IP addresses because the LAN interfaces will take priority over the WAN interface. It’s hard to explain but I’ll give an example. If you put the WAN interface of your OPNsense VM on the 182.168.1.1 network, the WAN of the OPNsense VM will be assigned something like 192.168.1.100. But if you also have a LAN interface in the OPNsense VM with 192.168.1.1/24, your WAN interface in the OPNsense VM will not be able to communicate with your primary network because the gateway address of the WAN interface will be 192.168.1.1 which happens to be the LAN interface IP address. One other gotcha is you will likely want to enable query forwarding under Unbound DNS if you are running into DNS issues. I’ve found that running a recursive DNS resolver behind my primary OPNsense box doesn’t work (probably since I am using DNS over TLS on my primary OPNsense so it can’t recursively resolve to the root DNS servers).
Why does almost everyone choose “Linux” as OS type when creating an OPNsense vm, when in fact OPNsense is FreeBSD 🤔
It’s either that or choose “other”. I think it affects the options that are available for the VM configuration since some options aren’t available for certain OS’s. Not sure if it makes a difference for FreeBSD based VMs or not.
The very first sentence of the video is how IT sounds to people who have zero clue about IT.
Yeah… it’s a mouthful of IT jargon.
Firewall on a VM is not a good idea. The "bad packets" must be forwarded through the physical server to the VM. This means that the physical server for the VM is always unprotected. (As an example a bad IP packet triggers a buffer overflow on the kernel)
Greetings Marco
I usually run bare metal but I know a lot of people like to virtualize for various reasons.
Do you have any documented examples of what you are referring about compromising the hypervisor on a virtualized firewall? I’d be interested in reading up on it.
@@homenetworkguy The IP packet arrives at an interface on the server and is analysed by the server (OSI Layer 2 & 3 analysis) and forwarded to the VM. These steps take place on the server before the packet arrives at the VM.
Only the IP tables of the server forward the packet to the VM. This means that the IPTables including the kernel are before the firewall.
Draw the path for each OSI layer once on a piece of paper and write who is responsible at each point.
I understand what you are saying. I am just curious how many documented cases of compromise due to virtualizing the firewall. So many people do it that I’m surprised more people say “don’t do it!”
@@homenetworkguy Security is not a question of the frequency of events! The host server is not protected and is therefore directly connected to the "bad" Internet.
Why use a firewall then?
@@marcodoehler4089 Because the OPNsense VM uses interfaces that are connected to bridges on the physical Proxmox VE interfaces, Proxmox VE doesn't analyze anything.
It will only receive Ethernet frames (layer 2 only), the bridge will look up the destination MAC address (of the OPNsense virtual interface) and simply forward it.
Iptables (or soon nftables) on Proxmox VE will not be used for this at all, unless you want to block traffic to and from the OPNsense VM from the host.
If you do not set an IP address on any of the bridge interfaces to which the OPNsense VM virtual interfaces are attached, there is no way to communicate with the host.