There's a reason people don't leave bedrooms and other places people actually live in unfinished. Also, all that equipment with fans in it is just stirring it up.
One really cool thing about these Tiny machines is they are a normal socketed CPU that you can upgrade (there an amazing serve the home listing about these and there specs and generations) so you can slot in little tiny GPUS and have a super power dence little PC
You mentioned that 20% isn't much, and I agree. However, if the task is single-threaded and you're using a 4-core CPU, 25% usage would mean one core is fully utilized. I Learned that the hard way with a 16 Core CPU. It said the CPU is at 6%. Which is nothing but still it was one core fully utilized.
True my virtual pfSense has 2 cores of a amd 7700. You need to tweak some things to use more then one core efficient. I think you bottleneck things when only assign one core of a 8400t.
@@chinesepopsongs00 Yes. I did not wanted to say, that less cores is better. I just wanted to say, that the CPU can be the bottleneck despite having just a 20% load, because a single core is too slow for a single thread task.
@@wiziek it counts the percentage usage in a fixed timeframe. If thing much shorter then that timeframe are not done fast enough because for example your cpu single thread performance is too low. Then you can have a cpu bottleneck with low utilization. I know i had that problem when my pfSense was running on a older hypervisor (i7 3770) my solution was going to 2 cores and i just migrated that over to my new hypervisor. I had about 35% utilization and was limited in troughput on the old hypervisor with a single core assigned. I doubted my isp at first but because i am on 1gbit plan it is very easy to test your own router local by just putting a machine with gbit connection on the wan side and test without your isp. That was proof enough for me the bottleneck was in my setup.
These are my favourite mini pc for a few years now. Reliable, expandable, and can run a decent ish cpu with potential for many gbs of memory. Not many other oems managed to get all these features into a true 1 litre chassis ( can’t think of any really) without making them much larger (HP/Dell extended models namely) I’ve got 3 nvme/ and a SATA drive in one of mine which runs a really power efficient nas (truenas)- circa 11w idle. Also got one as a standby router with quad 2.5 gbe should my other more modern 10gb mini pc pfsense router fail. Waiting for the M90qs to reach my price level then I’ll go to those! Akin the Lenovo t/e series laptops which I’ve also got a few of - not fashionable designs or fancy but workhorses.
With custom riser card you can have x16 slot (with x8 lanes from cpu) toghether with 2 x4 nvme slots from chipset. Which adds the possibility to have 4 nvme drives (5 in fact if you use wifi m.2 e-keyed slot too) and 2x10gbit ethernet in this single little PC. Those little PCs (920q/x especially) are great for home LABbing. Even the newer version of them aren't so great.
M920Q doesnt have 2x M.2 slots either. To get that (from factory) you'd need to get a M920X/P330/P340/P350/P360/M90Q. There is however some people that have figured out that you can solder the "missing" M.2 connector (and some other components) to the M720Q and the M920Q to get the second slot. More info about that in the linked STH thread. You can even hack the bios to support bifurcation on the PCIe slot. (8x to 4x4x) An easier alternative is to buy one of the community PCIe risers that exist for these boxes. They utilize the extra 4x PCIe lanes available on the chipset to add a extra M.2 slot.
Finally, a decent video that goes over some stuff that I've been looking into for awhile. I prefer a visual presentation along with the reading material I've been gathering. Thank you, thank you a ton.
Super cool setup man! Also: thanks for all the really cool Tailscale videos: I am setting up loads of selfhosted stuff, being helped by my 10 year old kid - and he is practicing his English listening to you + me translating to Finnish as needed. Keep up the good work!
So you left pFsense because open source is better (Opnsense) and now you moved to proprietary Dream machine? I hope you explain such a big shift, on the next video.
Wow, just found this channel, this is first video that I watched. Wow, you've got some great content. I need to build some type of Router / Firewall for home network. I'm just using a cheaper D-Link router that I got when my Xfinity was slower, but now, it's faster so I might need to get something with more horsepower. So, that's a good idea that system you have. Brilliant idea.
Congratulation for such a fast Fiber Internet Speed, up&down! 💪🏻 I am from germany and that is only a Dream here, perhaps only companies have such a fast Fiber Internet… Great Video, really nice homelab!
I have been running this for a while now too. Part of me wishes I only passed-through one of the SFP+ ports so other VMs and LXCs can share the internal facing port.
Does this tinies support IOMMU to virtualize PCIs passed through, so that other VMs have access to the devices? ( I am just entering the forest of proxmox and co, so that's a legitimate question)
Since the Lenovo M720q is running such a critical mission in your home, maybe you should have two? If you had second one in the "rack" next to the first, you could just disconnect the wires from the main system, switch the devices and reconnect the wires. The only question is how to backup the SSD so that you don't miss anything even if the main box totally fries thanks to lightning strike.
I do have the dream machine Pro Max with a 8Gbps symmetrical but with IDS on I'm getting around 5/6Gbps, any idea how much your firewall can handle with Suricata or any kind of IPS? would love opnsense in my Proxmox :D also id love to see your setup after installing the UDM Pro max! (PS LOVE TAILSCAL, have that on most of my LXC /VM now)
Very helpful video. One thought / question: I noticed at around the 8 minute mark you mentioned about how warm the system is. I wonder if you might have considered (or perhaps tried) undervolting the CPU - since later in the video you mentioned most of the time the cpu was being underutilized - and see if that might help with the heat issue? Especially since it seems to get quite toasty in the basement where all the equipment is housed (I imagine the fiberglass insulation seems to be doing its job rather well of keeping the warmth from going to the upper level(s) 😊)
What device do you use to measure power thru wifi? (18:36) And, then, how do you measure the temperatures of rooms (19:36)? That is really incredible. I live in Houston, Texas, and plan to upgrade my insulation in attic, because it is very costly monthly with my A/C bills. So, I'd like to monitor (like you do) from outside house to see if my attic tricks lower my electric bills. Also, can you (maybe) make videos on those? I've got cheesy devices from Amazon to monitor temp thru Bluetooth, but it doesn't hook up with PC's or share data over network. Be nice to see how your implementation works. That's really an incredible innovation that you've built.
Hey man, where did you get the cute little 3d printed cover for the network card? I checked the untrusted source site and don't see them there. My intel dial 10gbe nic is just sticking out the back of my Lenovo at the moment.
Not that long ago I Iearned that testing networks in a browser is limited to the browser as a bottleneck. The speedtest has a win app. And recently I updated my fiber to 2Gb and disabling the networkcard buffers is at the moment of testing better. I wash thinking to use a second new zimaboard to use opnsense, what do you think❓🤔💬 as I like these Lenovo very much. 🤩
How do you put the containers (caddy, pihole) on your LAN if the SFP ports are passed through to the opnsense vm? Did you put it on Proxmox's virtual switch? And then plug the 1Gbit port into your physical switch?
Can you please link where you found that 90 degree PCIe adapter? I have an M720q that I'd like to add a 10G NIC to as well and am having a hard time finding one that would fit in the case.
Appreciate the content. I admit, I did not watch the entire video. So, I may have missed where it's explained the reasoning behind virtualizing the firewall. I feel this over complicates things while also losing performance. There has been many attempts on my end to virtualize my networks firewall but it always ends creating unnecessary issues. For example, you reboot the host and the pass-through for some reason fails. You will then need to jump through a few hoops to regain access and resolve it. In my experience it's just not worth the headache.
Just to get the maximum info from your setup. Is HA running as another VM on the 720q? Do your other VMs only get access to the 1gig ethernet? Just trying to visualise the full setup. Thanks.
Oh wow I have one of those. I didn’t know you could put 64 gigs of RAM in there. Which network card is that, does it have to be a certain model or any Intel or similar.
@@markstanchin1692 Any model will do basically. It’s just PCIe, but it has to fit physically. Take a look at Supermicro AOC-STGN-i2S. They are both half height and half length dual SFP+ cards. It leaves room in the Tiny’s for even a shucked SATA SSD.
Since I’m thinking about adding a SFP+ card myself I can give you one more thing to consider: power consumption & heat. From what I’ve seen Intel X710 based cards might support ASPM with high C states (C7 according to what I’ve read), while many other NICs either don’t support ASPM at all or only reach C3). Might or might not be an issue for you…
I bought several HP EliteDesk 800 G4 Mini's for a proxmox cluster - dual M.2 NVMe 2280 slots (and SATA), but sadly no exposed PCIe slot like the M720q. The M720q and M920q both seem to be in same price range on ebay as the HP, but it's a real shame that Lenovo didn't offer a 2nd M.2 slot in these models. As someone posted below, the M920x provides the 2nd slot, but then the price goes way up.
Recently picked up a m920q. Can't decide for my 10gbe NIC if I want to go with RJ45 or SFP+. I don't have fiber like this lucky guy. I think maybe RJ45 will have to do for the moment with my lousy 1gbps/down 150mbps/up ISP plan
I recently got at&t fiber but only did the 1gb up/down because I got older ubiquity hardware so can't go faster than that on my network. Time to upgrade to 10G!
just seeing your channel, check ecoflow grid inverters so you can pop on Solar panel to greatly off-set during the day your hardware consumption! Plug-and-play literally just plugs into wall socket.
What 2x10gbps SFP+ NIC did you use ? About the Box getting quite Hot (besides the obvious ... it's PACKED :D), another possibility is that you are using a NIC that does NOT support ASPM, preventing the CPU from entering higher sleep / lower-power C-states (basically anything higher than C2/C3). Unless you have an Intel X710 and to a lower extent a Mellanox ConnectX-4, ASPM will NOT be supported at all (and Mellanox ConnectX-4 is only "Partial" ASPM, while the Intel X710 can achieve up to C7 Power States apparently). EDIT: I don't think that the Intel 82599ES supports ASPM (even though some Documentation say it should). No direct Experience from my Side, but lots of Research basically only point to the Intel X710 (Full ASPM) and Mellanox ConnectX-4 (Partial ASPM). The Intel X520 and X540 do NOT support ASPM apparently. I'll be testing soon the former 2 plus the Intel XXV710 (25gbps) as well ...
@@ep_dimi Intel X710-DA2 worked pretty much out of the Box even in CPU-connected PCIe slots, getting a Intel Xeon E3-1230 v3 in a Supermicro X10SLL-F Motherboard down to Package PC6. Mellanox ConnectX-4 I still gave it a Try Yesterday, but it's not working at all in the CPU-connected PCIe Slot. Whereas you could get away in a PCH slot especially for 10gbps and a Supermicro X11SSL-F & Intel Xeon E3 v5/v6, because the DMI link is PCIe 3.0 x4 (NOT PCIe 2.0 x4 like in the other Platform I tested).
@@ep_dimi No worries. For 1gbps I had to "Force" the i350-T4 (because Linux complains about PCIe 1.1 Device or something) but it does appear to work. Note however that I didn't have any Cable plugged in while I was running the Test. You furthermore will/might have to struggle with Linux Configuration Quite a bit. I put some stuff into Several GitHub Repositiories about how to Patch, Troubleshoot, what to look for. Although, to be honest, on some Systems it still will NOT work, no matter how hard I try :(.
Maybe I missed you explaining it in the video but wouldn't it be possible to plug the SFP straight into the Lenovo box instead of using the AT&T provided media "modem"?
I have the AMD Ryzen pro, and it has been very useful, I run jelly fin on it and it's my prefered "daily driver" I have added 12TB of storage and upped the ram to 24GB, I must say it runs like a champ. I don't game, so no worries there, photoshop runs fine. I like it, saves on electricity and space. my towers just sit to the side!
@@ktzsystems Google "RTX a2000 one Slot cooler diy". 😉 You need also undervolt it. Its Tricky But Work. For Gen9 Intels you need Coffeetime to add Microcode for the Gen9 CPU.
@@ktzsystems also you can DL a 3D STL File to make a Case that Suite the A2000 without cooler Mod. A4000 ADA SFF Work also. Main drawback is the 75W Powerlimit of the PCI-E Slot and also PSU. Without GPU you can use also Non K and KF CPU's with High Performance cooler.
I inserted a GPU and drilled many small holes so that the GPU fan gets fresh cold air to blow over the heatsink. You sould du similar for that custom blower
hi , i am from india (an architect) knows nothing about networking . i want to setup the firewall and learn by doing so . can the chat please guide me to any ref thats tells step by step process ?thank you
Hey Alex (great name, by the way :D) Why virtualize the firewall? Why not run it natively on the hardware? My PFSense box is running on a Xeon E3 with 8 gigs of RAM, including Suricata, Tailscale, dual WAN (1 x 1G fiber and 1 x 400mbit cable) and it's very very stable and quick while running below 40 Watts in total.
@@ktzsystems it may be less responsive as the hypervisor needs to balance all workloads and schedule all vCPU requests. So ping times may be impacted. Also, throughput may be less. I notice that when I run OpenSpeedTest on my virtual environment, I get a max throughput of 8 gigabit on a 10 gig line. So it may have limiting factors.
get spray foam insulation or enclose the ceilings - neither option is a huge project and you're a handy fellow ;) the pink stuff only degrades over time and gets everywhere and is just awful lol
Nice video, I suddenly consider my 4/4 gbit to be very cheap for €67,50 a month. (I checked what 5/5 costs at AT&T, $245) Intel 82599ES is getting really old, though a good solution, I went for a i5 MS-01 because of the X710 chipset instead of the really old 82599 you'll find on afforable cards. Later on I found some cheap X710 card though. Good thing you don't have nasty PPPoE, really costs a lot of CPU.
if you look when he does the lspci command (12:57) it's an intel 82599, which is very similar to the x520-da2. Both are pcie gen2 x8 cards, with no ASPM support, from 2009. If I were to build a machine today with a dual sfp+ nic I would go with a little higher tier card, hopefully with aspm support, like a connect-x 4 LX.
I have seen a max of 4.5gbps routing through here with no issue. The on device speedtests from the modem don’t go higher than that so I’m satisfied that I I’m not bottlenecked.
your grasteful to have fiber... we had DSL for many many years until recent with 56k speeds.... thanks AT&T. I gave up and called AT&T business and am getting dedicated internet (ADI) this week. We tried using a hotspot but we are to far away from any useable towers for it to be any count.
you should make a similar video but instead of lenovo m720q try minisforum ms-01, it has 2x SFP+ and 2x 2.5Gib rj45, also 2x thunderbolt that could be used for cluster traffic
Well, the swiss to, if you are a customer of Init7, you can get 1 Gbit/s, 10 Gbit/s or even 25 Gbit/s symmetrical fiber. All of that for the cost of 65 CHF (~78 USD) per month, or 111 CHF (~131 USD) for a business contract.
In Italy 1 or 2.5gb fiber is quite readily available. 10g mostly in bigger cities over xgs-pon. I can get a 2.5/0.5 line for roughly 30€/month, 10g/2.5g is usually 60-80 depending on the operator and carrying network. Main problem is most ISP do use PPPoE, which is a bit hard on OPNsense / PFsense. I do have a m720q as well with a 10G nic, waiting for fiber to be installed next month.
Bro. I haven't even watched much of this video and I feel that upload pain you're talking about. I work at a place that is rural, and we pay $600/mo for friggen 20/20mbit..... Do you know how long it takes to initiate a new backup to an S3 bucket over that? At home, Comcast's best plan was 1200/20...... I ended up on Tmobile home internet and get 250/50. SMH At least *that* is a reasonable ratio.
does the m720q 8500T cpu support 64GB RAM officially or there is some kind of tweaks as i think its Up to 32GB DDR4-2666 and from your experience how many VMs it can handel on proxmox in case of kubernests clustering , thanks for the informative video
Always throws me off when I hear your accent with you talking about down south and I'm thinking yeah ok, in the UK. Then you mention AT&T and I always forget you're in USA
In the US you are still required to use the router provided by the ISP? In Germany we have the right to use our own routers. Took us 25 years to fight for.
Brother please close that fibreglass insulation please. Your lungs will thank you. Please take care of yourself
mini dagger stabbing his lungs
There's a reason people don't leave bedrooms and other places people actually live in unfinished.
Also, all that equipment with fans in it is just stirring it up.
I upgraded mine to an i7 9th gen. Lovely machine. I used it as a backup/experimental server, with WOL, so I can trigger it using HA
I have the same Lenovo box, and had already tagged it as the node on my cluster that was going to handle my network stack. Thanks for this!
One really cool thing about these Tiny machines is they are a normal socketed CPU that you can upgrade (there an amazing serve the home listing about these and there specs and generations) so you can slot in little tiny GPUS and have a super power dence little PC
You mentioned that 20% isn't much, and I agree. However, if the task is single-threaded and you're using a 4-core CPU, 25% usage would mean one core is fully utilized. I Learned that the hard way with a 16 Core CPU. It said the CPU is at 6%. Which is nothing but still it was one core fully utilized.
True my virtual pfSense has 2 cores of a amd 7700. You need to tweak some things to use more then one core efficient. I think you bottleneck things when only assign one core of a 8400t.
@@chinesepopsongs00 Yes. I did not wanted to say, that less cores is better. I just wanted to say, that the CPU can be the bottleneck despite having just a 20% load, because a single core is too slow for a single thread task.
You don't really have no idea how cpu utilization logs work.
@@wiziek Sure. Enlighten me then.
@@wiziek it counts the percentage usage in a fixed timeframe. If thing much shorter then that timeframe are not done fast enough because for example your cpu single thread performance is too low. Then you can have a cpu bottleneck with low utilization. I know i had that problem when my pfSense was running on a older hypervisor (i7 3770) my solution was going to 2 cores and i just migrated that over to my new hypervisor. I had about 35% utilization and was limited in troughput on the old hypervisor with a single core assigned. I doubted my isp at first but because i am on 1gbit plan it is very easy to test your own router local by just putting a machine with gbit connection on the wan side and test without your isp. That was proof enough for me the bottleneck was in my setup.
These are my favourite mini pc for a few years now. Reliable, expandable, and can run a decent ish cpu with potential for many gbs of memory. Not many other oems managed to get all these features into a true 1 litre chassis ( can’t think of any really) without making them much larger (HP/Dell extended models namely)
I’ve got 3 nvme/ and a SATA drive in one of mine which runs a really power efficient nas (truenas)- circa 11w idle.
Also got one as a standby router with quad 2.5 gbe should my other more modern 10gb mini pc pfsense router fail.
Waiting for the M90qs to reach my price level then I’ll go to those!
Akin the Lenovo t/e series laptops which I’ve also got a few of - not fashionable designs or fancy but workhorses.
Small correction: That PCIe slot is only x8 not x16. It's only x16 physical.
With custom riser card you can have x16 slot (with x8 lanes from cpu) toghether with 2 x4 nvme slots from chipset. Which adds the possibility to have 4 nvme drives (5 in fact if you use wifi m.2 e-keyed slot too) and 2x10gbit ethernet in this single little PC. Those little PCs (920q/x especially) are great for home LABbing. Even the newer version of them aren't so great.
The M.2 A/E to M-key adapter is a great idea given these M720qs lack the second M.2 slot of the M920qs (edit: M920x, not q)! Thanks for the video!
You can get an M90q, it has 2 M.2 slots on the bottom, 2.5 bay on top and 16x PCIe 3.0 slot.
M920Q doesnt have 2x M.2 slots either. To get that (from factory) you'd need to get a M920X/P330/P340/P350/P360/M90Q.
There is however some people that have figured out that you can solder the "missing" M.2 connector (and some other components) to the M720Q and the M920Q to get the second slot. More info about that in the linked STH thread. You can even hack the bios to support bifurcation on the PCIe slot. (8x to 4x4x)
An easier alternative is to buy one of the community PCIe risers that exist for these boxes. They utilize the extra 4x PCIe lanes available on the chipset to add a extra M.2 slot.
@@KS-wr8ub You're totally right, it was the M920x I was thinking of.
Finally, a decent video that goes over some stuff that I've been looking into for awhile. I prefer a visual presentation along with the reading material I've been gathering. Thank you, thank you a ton.
Super cool setup man! Also: thanks for all the really cool Tailscale videos: I am setting up loads of selfhosted stuff, being helped by my 10 year old kid - and he is practicing his English listening to you + me translating to Finnish as needed. Keep up the good work!
So you left pFsense because open source is better (Opnsense) and now you moved to proprietary Dream machine? I hope you explain such a big shift, on the next video.
Better integration with his Ubiquiti equipment. Pretty easy to understand.
So, better integration with his other proprietary equipment. Got it.
@@Kermit2kvendor lock in
What are you mad at? People change their minds.
Wow, just found this channel, this is first video that I watched. Wow, you've got some great content. I need to build some type of Router / Firewall for home network. I'm just using a cheaper D-Link router that I got when my Xfinity was slower, but now, it's faster so I might need to get something with more horsepower. So, that's a good idea that system you have. Brilliant idea.
Hello and thank you Alex for video. I see that you explain many things in the right and proper way. + some tests and interesting solutions.
Congratulation for such a fast Fiber Internet Speed, up&down! 💪🏻
I am from germany and that is only a Dream here, perhaps only companies have such a fast Fiber Internet…
Great Video, really nice homelab!
We love these little machines! At my MSP, we have these at client sites to remote into for network troubleshooting if need be.
Welcome to the M720q router club :)
I have been running this for a while now too. Part of me wishes I only passed-through one of the SFP+ ports so other VMs and LXCs can share the internal facing port.
Does this tinies support IOMMU to virtualize PCIs passed through, so that other VMs have access to the devices? ( I am just entering the forest of proxmox and co, so that's a legitimate question)
@@pascalabessolo5350 Yes, it was very easy on my m920q and should be similar on the m720q. The latest proxmox versions make it easier than ever.
Cant wait to see the bypass with the WAS-110
Since the Lenovo M720q is running such a critical mission in your home, maybe you should have two? If you had second one in the "rack" next to the first, you could just disconnect the wires from the main system, switch the devices and reconnect the wires. The only question is how to backup the SSD so that you don't miss anything even if the main box totally fries thanks to lightning strike.
I do have the dream machine Pro Max with a 8Gbps symmetrical but with IDS on I'm getting around 5/6Gbps, any idea how much your firewall can handle with Suricata or any kind of IPS? would love opnsense in my Proxmox :D also id love to see your setup after installing the UDM Pro max! (PS LOVE TAILSCAL, have that on most of my LXC /VM now)
Very helpful video.
One thought / question: I noticed at around the 8 minute mark you mentioned about how warm the system is. I wonder if you might have considered (or perhaps tried) undervolting the CPU - since later in the video you mentioned most of the time the cpu was being underutilized - and see if that might help with the heat issue? Especially since it seems to get quite toasty in the basement where all the equipment is housed (I imagine the fiberglass insulation seems to be doing its job rather well of keeping the warmth from going to the upper level(s) 😊)
Which SFP+ card are you using? If you mentioned it in the video I must have missed it. Thanks!
Me gusto ese rack de madera, creo que lo implementare en mi "Home Lab" saludos.
I know from some previous videos that you’re a fan of VLANs. Could you do a video showing how you integrate home assistant across your VLANs?
This is great! Where can I get a baffle and shroud for both M720Q and M90Q Gen 01? I'm in the UK!
What device do you use to measure power thru wifi? (18:36) And, then, how do you measure the temperatures of rooms (19:36)? That is really incredible. I live in Houston, Texas, and plan to upgrade my insulation in attic, because it is very costly monthly with my A/C bills. So, I'd like to monitor (like you do) from outside house to see if my attic tricks lower my electric bills. Also, can you (maybe) make videos on those? I've got cheesy devices from Amazon to monitor temp thru Bluetooth, but it doesn't hook up with PC's or share data over network. Be nice to see how your implementation works. That's really an incredible innovation that you've built.
Hey man, where did you get the cute little 3d printed cover for the network card? I checked the untrusted source site and don't see them there. My intel dial 10gbe nic is just sticking out the back of my Lenovo at the moment.
I'd be curious about the Ethernet SFP module. It could became pretty hot 60+C which would melt the 3d printed bracket pretty soon.
Could you please share a link to the M.2 A/E to M-key adapter?
Not that long ago I Iearned that testing networks in a browser is limited to the browser as a bottleneck.
The speedtest has a win app.
And recently I updated my fiber to 2Gb and disabling the networkcard buffers is at the moment of testing better.
I wash thinking to use a second new zimaboard to use opnsense, what do you think❓🤔💬 as I like these Lenovo very much.
🤩
How do you put the containers (caddy, pihole) on your LAN if the SFP ports are passed through to the opnsense vm? Did you put it on Proxmox's virtual switch? And then plug the 1Gbit port into your physical switch?
Yup! Exactly right.
how did you power the second fan on the m720q? would love the additional cooling for my opnsense box too
Can you please link where you found that 90 degree PCIe adapter? I have an M720q that I'd like to add a 10G NIC to as well and am having a hard time finding one that would fit in the case.
It’s in the description 👍
why not putting the GPON SFP module directly into the lenovo machine or a switch?
Appreciate the content. I admit, I did not watch the entire video. So, I may have missed where it's explained the reasoning behind virtualizing the firewall. I feel this over complicates things while also losing performance. There has been many attempts on my end to virtualize my networks firewall but it always ends creating unnecessary issues. For example, you reboot the host and the pass-through for some reason fails. You will then need to jump through a few hoops to regain access and resolve it. In my experience it's just not worth the headache.
Just to get the maximum info from your setup. Is HA running as another VM on the 720q? Do your other VMs only get access to the 1gig ethernet? Just trying to visualise the full setup. Thanks.
You got it!
Why pci passthrough for rhe 10g nic? Just create 2 Linux Bridges in Proxmox and add those as a network device in OpnSense VM?
What IPMI KVM Switch do you have?
Did you 3D print a low profile io bracket for the 10G dual port NIC, or buy one from someone like untrustedsource?
He sent it to me but it should be an easy print if you can find the file
Can you please do a video on how you set up reverse proxies? The guide on your site never works for me
Oh wow I have one of those. I didn’t know you could put 64 gigs of RAM in there. Which network card is that, does it have to be a certain model or any Intel or similar.
@@markstanchin1692 Any model will do basically. It’s just PCIe, but it has to fit physically. Take a look at Supermicro AOC-STGN-i2S. They are both half height and half length dual SFP+ cards. It leaves room in the Tiny’s for even a shucked SATA SSD.
have a look at the reference thread mentioned to get some recommendations for NICs.
Since I’m thinking about adding a SFP+ card myself I can give you one more thing to consider: power consumption & heat. From what I’ve seen Intel X710 based cards might support ASPM with high C states (C7 according to what I’ve read), while many other NICs either don’t support ASPM at all or only reach C3). Might or might not be an issue for you…
I bought several HP EliteDesk 800 G4 Mini's for a proxmox cluster - dual M.2 NVMe 2280 slots (and SATA), but sadly no exposed PCIe slot like the M720q. The M720q and M920q both seem to be in same price range on ebay as the HP, but it's a real shame that Lenovo didn't offer a 2nd M.2 slot in these models. As someone posted below, the M920x provides the 2nd slot, but then the price goes way up.
Recently picked up a m920q. Can't decide for my 10gbe NIC if I want to go with RJ45 or SFP+. I don't have fiber like this lucky guy. I think maybe RJ45 will have to do for the moment with my lousy 1gbps/down 150mbps/up ISP plan
Well i guess i have to be that guy. How did you get the "neo" art in your shell?
oh. it is figurine
Yup! Figurine. I made a video about it a few months ago. Enjoy being fancy!
Why not run the the ONT directly into the 720q or request an XPON from them?
See the next video after this one about bypassing the ATT gateway.
I recently got at&t fiber but only did the 1gb up/down because I got older ubiquity hardware so can't go faster than that on my network. Time to upgrade to 10G!
Still confused why the i5 has no hyperthreading and if this is a huge issue with promox/containers/VMs. Anyway, CPU should be upgradeable.
just seeing your channel, check ecoflow grid inverters so you can pop on Solar panel to greatly off-set during the day your hardware consumption! Plug-and-play literally just plugs into wall socket.
May i ask why you use the AT&T Router and not just run the fiber directly intro the M720Q SFP+ nic?
To be supported by AT&T in case of failure I guess.
You need an ONT to convert the WAN side fiber and get an IP from ATT.
Does this specific model have pci slot or certain models have it ?
Check the servethehome thread for more info.
Do all Lenovo M720qs come with that angle/riser adapter?
Don’t believe so. See the description link for where to get one (no affiliation)
What 2x10gbps SFP+ NIC did you use ? About the Box getting quite Hot (besides the obvious ... it's PACKED :D), another possibility is that you are using a NIC that does NOT support ASPM, preventing the CPU from entering higher sleep / lower-power C-states (basically anything higher than C2/C3). Unless you have an Intel X710 and to a lower extent a Mellanox ConnectX-4, ASPM will NOT be supported at all (and Mellanox ConnectX-4 is only "Partial" ASPM, while the Intel X710 can achieve up to C7 Power States apparently).
EDIT: I don't think that the Intel 82599ES supports ASPM (even though some Documentation say it should). No direct Experience from my Side, but lots of Research basically only point to the Intel X710 (Full ASPM) and Mellanox ConnectX-4 (Partial ASPM). The Intel X520 and X540 do NOT support ASPM apparently. I'll be testing soon the former 2 plus the Intel XXV710 (25gbps) as well ...
Is there any intel 1 or 10Gbps nic with unproblematic ASPM support? I got lost. Even the x550 have problems with ASPM, depending on the firmware
@@ep_dimi Intel X710-DA2 worked pretty much out of the Box even in CPU-connected PCIe slots, getting a Intel Xeon E3-1230 v3 in a Supermicro X10SLL-F Motherboard down to Package PC6.
Mellanox ConnectX-4 I still gave it a Try Yesterday, but it's not working at all in the CPU-connected PCIe Slot. Whereas you could get away in a PCH slot especially for 10gbps and a Supermicro X11SSL-F & Intel Xeon E3 v5/v6, because the DMI link is PCIe 3.0 x4 (NOT PCIe 2.0 x4 like in the other Platform I tested).
@@silverstone7778 thank you for the info!
@@ep_dimi No worries. For 1gbps I had to "Force" the i350-T4 (because Linux complains about PCIe 1.1 Device or something) but it does appear to work.
Note however that I didn't have any Cable plugged in while I was running the Test.
You furthermore will/might have to struggle with Linux Configuration Quite a bit. I put some stuff into Several GitHub Repositiories about how to Patch, Troubleshoot, what to look for. Although, to be honest, on some Systems it still will NOT work, no matter how hard I try :(.
Maybe I missed you explaining it in the video but wouldn't it be possible to plug the SFP straight into the Lenovo box instead of using the AT&T provided media "modem"?
No those are basic sfp bidi fiber modules The pon functions are all handled by the gateway.
I have an “ont on a stick” on the way. But for now I need to use the ATT gateway to handle that.
pon.wiki/category/att/ yes sir there is ;)
I have the AMD Ryzen pro, and it has been very useful, I run jelly fin on it and it's my prefered "daily driver" I have added 12TB of storage and upped the ram to 24GB, I must say it runs like a champ. I don't game, so no worries there, photoshop runs fine. I like it, saves on electricity and space. my towers just sit to the side!
I use a M910x with Gen9 i9 and a RTX A2000 with a 1 Slot heatsink.
I like this type of PC 😁
You can fit an a2000 in there?!?
@@ktzsystems Google "RTX a2000 one Slot cooler diy". 😉 You need also undervolt it. Its Tricky But Work.
For Gen9 Intels you need Coffeetime to add Microcode for the Gen9 CPU.
@@ktzsystems also you can DL a 3D STL File to make a Case that Suite the A2000 without cooler Mod. A4000 ADA SFF Work also. Main drawback is the 75W Powerlimit of the PCI-E Slot and also PSU. Without GPU you can use also Non K and KF CPU's with High Performance cooler.
What is the Nic that you are using?
the speed test you ran the equipment was connected to the Unifi?
The UniFi is still in the box
Does at&t allow you to put their xgs-pon sfp+ straight into your own router instead of going throught their box?
Nevermind, I've read in an older comment you have an ont-sfp in the works...
It’s on the way from the 8311 discord group buy 👍
I have a Ryzen 2400GE M715Q I would love to do this with but it doesn't have a full PCI-e slot :(
Please give some details about the temperatures running this nic
Warm! Put the fans on “performance mode” and it’s acceptable.
Why not mirror the NVME and A/E SSDs?
Maby I missed it but is that 8400T in the Lenovo fast enough to handle 10gbit routing internally at full speed?
Seems to be. I’ve managed 5gig iperf tests to a buddy with 5gig just fine. I can’t speak to faster than that though.
@@ktzsystems ok. Good to know. Was that with tailscale?
@@djvincon yup
@@ktzsystems awesome thanks!
I inserted a GPU and drilled many small holes so that the GPU fan gets fresh cold air to blow over the heatsink. You sould du similar for that custom blower
hi , i am from india (an architect) knows nothing about networking . i want to setup the firewall and learn by doing so . can the chat please guide me to any ref thats tells step by step process ?thank you
Would love a video on how and why you immigrated to the US. Any hurdle or obstacles you faced. Considering it myself but don't know where to start
Being born in the US helped a bit ;)
very nice setup, most people could only dream about getting 5gig or better fiber.
There’s a guide to add the parts for the second m2 slot. It only supports SATA SSD though. Not NVMe.
@@ROFLMAOwithExtraCheese SATA M.2 on the M720Q and NVMe M.2 on the M920Q. 👍
As always, great content Alex! I assume your WiFI network name and password are not the ones shown on your router label, right? 😃
Of course not! 😍
Hey Alex (great name, by the way :D) Why virtualize the firewall? Why not run it natively on the hardware? My PFSense box is running on a Xeon E3 with 8 gigs of RAM, including Suricata, Tailscale, dual WAN (1 x 1G fiber and 1 x 400mbit cable) and it's very very stable and quick while running below 40 Watts in total.
@@NTVN-Alex what motherboard?
Flexibility I suppose.
Being able to run it as a VM has no real downsides and means I can make the hardware sweat a bit harder.
@@rajilsaraswat9763 Supermicro X9-SCM-F
@@ktzsystems it may be less responsive as the hypervisor needs to balance all workloads and schedule all vCPU requests. So ping times may be impacted. Also, throughput may be less. I notice that when I run OpenSpeedTest on my virtual environment, I get a max throughput of 8 gigabit on a 10 gig line. So it may have limiting factors.
get spray foam insulation or enclose the ceilings - neither option is a huge project and you're a handy fellow ;)
the pink stuff only degrades over time and gets everywhere and is just awful lol
I’ll add it to the list.
Nice video, I suddenly consider my 4/4 gbit to be very cheap for €67,50 a month. (I checked what 5/5 costs at AT&T, $245) Intel 82599ES is getting really old, though a good solution, I went for a i5 MS-01 because of the X710 chipset instead of the really old 82599 you'll find on afforable cards. Later on I found some cheap X710 card though. Good thing you don't have nasty PPPoE, really costs a lot of CPU.
Hey Alex, could you share the link for the SFP+ card you use in this video?
I'm not sure it matters too much but the one in here is a Fujitsi card of some type. Sorry I don't have the exact details.
if you look when he does the lspci command (12:57) it's an intel 82599, which is very similar to the x520-da2. Both are pcie gen2 x8 cards, with no ASPM support, from 2009. If I were to build a machine today with a dual sfp+ nic I would go with a little higher tier card, hopefully with aspm support, like a connect-x 4 LX.
Thanks!
@@ktzsystemsthe card shows it does 1g/10g link speeds. It’s able to negotiate with that switching adapter?
@@Natebur Yep!
5:31 Not much thermal paste in place there 😦
How do you scale this to get linespeed of 5gbps?
I have seen a max of 4.5gbps routing through here with no issue. The on device speedtests from the modem don’t go higher than that so I’m satisfied that I I’m not bottlenecked.
@@ktzsystems is that with suricata switched on?
I didn’t test that yet
You should be able to put the ONT directly into the Dream Machine so no extra (AT&T) modem is needed.
Should have watched your video from yesterday lol
9:57
I see that it has a wifi config, isn't this double NAT?
If that's the case, could it be set up as a router and just buy a cheapo 10gbe switch?
The ATT gateway has an IP pass through mode so I get the WAN IP on OPNsense. No double NAT.
You need the ATT box as that is doing ONT duties.
I just bought one of these for the same reason
Plastic sheet (visqueen) stapled to the joist will keep it out of your hair and eyes.
Tailscale plugin for OPNsense when? :)
Get in line!
your grasteful to have fiber... we had DSL for many many years until recent with 56k speeds.... thanks AT&T. I gave up and called AT&T business and am getting dedicated internet (ADI) this week. We tried using a hotspot but we are to far away from any useable towers for it to be any count.
That fiber glass is going to coat the inside of your electronics. It’s constantly breaking down into glass particles.
Probably. But these systems have been down there for 5 years and counting and are fine so far 👍
Good video sir ! them 720's make good firewalls !! Opnsense with Zenarmor is very powerfull.
you should make a similar video but instead of lenovo m720q try minisforum ms-01, it has 2x SFP+ and 2x 2.5Gib rj45, also 2x thunderbolt that could be used for cluster traffic
It’s a sick box but spendy
My dude, 5000Mbps?! it's USA! we don't even have that in EU, probably only in Japan. How did you get it? :D also, it must be extremely pricey!
Gulp $240 pm
Well, the swiss to, if you are a customer of Init7, you can get 1 Gbit/s, 10 Gbit/s or even 25 Gbit/s symmetrical fiber.
All of that for the cost of 65 CHF (~78 USD) per month, or 111 CHF (~131 USD) for a business contract.
We’ve had 10Gbps available here in Sweden for quite some time now. It’s mostly in the cities though.
In Italy 1 or 2.5gb fiber is quite readily available. 10g mostly in bigger cities over xgs-pon. I can get a 2.5/0.5 line for roughly 30€/month, 10g/2.5g is usually 60-80 depending on the operator and carrying network. Main problem is most ISP do use PPPoE, which is a bit hard on OPNsense / PFsense. I do have a m720q as well with a 10G nic, waiting for fiber to be installed next month.
I have 10 gbit Symetrical in Switzerland. You can get this for as low as 45 CHF or somtimes even lower but not everywere. i have to pay 79 CHF .
Bro. I haven't even watched much of this video and I feel that upload pain you're talking about. I work at a place that is rural, and we pay $600/mo for friggen 20/20mbit..... Do you know how long it takes to initiate a new backup to an S3 bucket over that? At home, Comcast's best plan was 1200/20......
I ended up on Tmobile home internet and get 250/50. SMH At least *that* is a reasonable ratio.
does the m720q 8500T cpu support 64GB RAM officially or there is some kind of tweaks as i think its Up to 32GB DDR4-2666 and from your experience how many VMs it can handel on proxmox in case of kubernests clustering , thanks for the informative video
If u worry about temperature - just open it
Laptop usb cooler below it to cool it
Always throws me off when I hear your accent with you talking about down south and I'm thinking yeah ok, in the UK. Then you mention AT&T and I always forget you're in USA
Looks more like a crawl space than a basement.
Pretty much
WOW 男人的天堂
👀
10Gbps routing easy, 10Gbps IPS/IDS thats the challenge....
If its humit there why is your equipment there then?!
in germany deal with 50k upload 🙈
That’s dial up?!
@@ktzsystemsprobably some shitty asynchronous DSL connect. There are still ISP that love and sell this.
In the US you are still required to use the router provided by the ISP? In Germany we have the right to use our own routers. Took us 25 years to fight for.
Crawl space more like it. Moisture and heat for a server room is not a good mix.
Why is a pc a firewall is that just a computer
Firewall with no wall
20% idle means your cpu is 80% busy… pfSense usually performs better on proxmox