7:26: Just to note: the eBay search results shown there are for the Dell rNDC card, not the stand-alone PCIe version. Unless you have a Dell server with a compatible onboard networking port, don't buy the parts shown on screen here.
We have 20Mbs/10Mbs cable connection for around 25-30$ for last at least ten years, before it was 10/1 for the same price for ten years. THANK GOD FOR O2 !!! They slowed my life so much
Some things to add, the X550 does better at longer cable runs than the aquantia chipset. Also there are router OSes made specifically with low latency in mind like VyOS or tnsr (from netgate the makers of pfsense).
@@mitch7918 vyos is targeted at enterprises and carriers. Those have staff that are comfortable with CLI. The CLI on vyos is pretty easy to work with once you understand the syntaxes and configuration modes. It does struggle with hairpin NAT, so use split DNS for that.
Regarding the AT&T BGW320-500/505 we have found a way around not using it at all on AT&T's network. It involves using an external XGS-PON ONT which has a 10G ethernet port. Availability is abysmal and it's been a community effort figuring the device out (big ups to the few people who got deep into it and figured it out), the manufacture either won't or doesn't respond to emails thus far. Thanks for the video Wendell.
Definitely going to be hunting for that on the forums, currently still have the BGW210 and getting it sorted to bypass at least that unit, but being able to pass the ONT too would be awesome if the same approach works, though admittedly less necessary as the BGW210 is the main source of my issues. Thanks for letting us know the options exist :D
Jeez, keep it up. The tidbit about pcore and ecore on linux vs freebsd was a good nugget of info. Thanks to you and the commenters. The comments about these horrible att routers was great too. Led me down a path. Maybe I can join and figure it out. I do not know how youtube bullseyed this one but I'm here.
7:30 I had no idea there was a Firmware update to the X550, that “unlocks” 2 & 5gbe. I’ve been hunting for the 700series that they made that supports this. Makes a great bargain, even better. 👍
I'm rockin an old Dell PowerEdge R210 II running pfsense. I slapped in a Realtek quad 2.5G network card (I know eww realtek) but it actually works fine if you install the proper drivers. May be upgrading to a R330 soonish. Love your videos, you literally do things I find interesting and I may even go to school for networking next year
Been using a ryzen 3200g in a router box since late 2019. Works fine with 3gbs fiber. For anything faster than 3gbs, you likely need openwrt as I don't think the freebsd can do 10gbs yet - unless they fixed it.
2 months ago, and that specific motherboard is nowhere to be found...any w680 board is $500 I think? that's what my searches have found. What alternatives do you have for that board? how budget are we talking here?
Sliger is a really fantastic company to work with. I had the opportunity to work with Kahlin Sliger in designing a chassis for a medical instrument. The company was so great to work with commercially, that I only ever recommend them personally.
No SR-IOV on Aquantia. :( RE: P-cores and E-cores, I think the way to go is pinning VMs to cores and using isolcpus or cpusets to keep the hypervisor from using them.
I've been working on a home router project over the last year, got a supermicro 1u 5019D-4C-FN8TP with xeon D-2123IT. Setup Proxmox and created a virtualized PFsense. My biggest issues were the 2.5/5/10gb negotiations. My modem has 2.5gb but had trouble getting things to work, so tried passing through the NICs to Pfsense but found out BSD has trouble with realtek drivers. Got things sorted out now using an intel i225 nic but virtualized and passed to the VM using virtio. My connection is 1.2gb but I can get a consistent 1400mbs out of it and iperf gives me stable 10gb between my homelab servers. I have the t540 nic in my desktop, if I had known about the t550 and firmware to get 2.5/5 that would have made my life alot easier. Thanks for the video!
Also have the 540. Didn't realize until this video it doesn't negotiate 2.5/5. Had I paid attention to that detail a couple of years ago I would have saved myself a whole lot of stress. Looking into 550 options now, that 5 second comment in the video just explained so much for the problems I've been having.
@@beepboopbeepboop190 I ended up getting a TEG-S762 switch for my office to connect to my desktop. Has 2x10gb ports so I don't have to worry about negotiation. 10gb to desktop then 10gb uplink to my homelab, and it has 4x2.5gb ports as well.
@@krj15489 That's a good call. Mine is installed in my firewall/router so one is connected to a 10G switch on the lan side but the other port is connected to my ISP's terminal. I'm 99% sure that's not 10G but I think it might be more than 1G. The nic on the fw's motherboard is 2.5 though so I may just swap to that for the wan.
Gotta be nice to have an Internet connection to do your router justice. :) My own network uses 1000 base T/ Gigabit ethernet. The bottleneck is the cable modem. For the router, I use the Linksys WRT1200AC with the OpenWRT on it. If you can upgrade a consumer router with OpenWRT you won't be disappointed. You can then use NFS and have it also be a light show machine - AND a router!
I’m from Chile, here I have 2 isp connections, one at 800 Mbps and another at 600 Mbps. I can upgrade one of the isp connetion to 2 Gbps, and for that reason I’m building a custom router with one of those super micro motherboard with 6x10G ports that you can find on ebay.
I have a Dell R420 w/ Dual E5-2430 w/ 32GB ECC RAM and X520 2xSFP+ card booting pfsense off a SATA SSD. It has no problem with 2Gbit internet and even tested 10Gbit with it (using another machine acting as my WAN connection). I'm into it for maybe $300 and it is absolutely solid as a rock. I'll trust enterprise hardware a lot more than consumer grade hardware. Am I missing something?
I get about ~2.5 Gbps L3 performance using OPNsense with 4 cores of a 2680 v4 under ESXi. I tried with 8 cores and it scaled linearly so I'm not convinced you need hardware as good as what's on display for consumer 2.5 or 5 Gbps services unless you're going to run something like Wireguard.
this forbidden router makes so much sense in a lot of homelab setups, it's simple, a proper machine running a VM that contains your software router, and has proper hardware to connect to your switch. You can add dns, proxy, nginx, vpn, and everything network on the server, and even Firewall rules! If you don't need 10G you can get a 4 port 2.5G ethernet card for cheap and maybe even ditch the expensive managed switch all together! Assigning vlans for each port and rules to connect everything together. All at a low price and with the benefits of a VM!
Wait, that's a novel idea (to me). My switching needs are minor for non poe. Could I get 2 or 3 2.5g NICs to work like this? Assuming chassis supported it.
@@badharrow as long as the motherboard supports giving the PCIe lanes to the VM that's possible! I do something very similar and works wonders, no need for expensive switches here.
I wouldn't mind going in on a group buy for the motherboard. I've got quite a few alder lake cpus from XOC I was planning on using to replace my current homelab system with (3970x, 256gb ram). These would fit the bill quite well.
Great video, Wendell, it really got me thinking about a new router build down the road. AT&T Fiber sounds... less than ideal. We have CenturyLink here (1Gb) and I just hooked up my homebrew opnsense box in place of their Zyxel C4000 modem/router and we were in business. Having to work around subpar ISP hardware is never fun. Cheers!
Just put in 8 Gig internet. I have the equipment to handle it, but I have opposite problems with some of my machines. Download and uploads are not symmetrical, both ways in different machines. I do need to look into building a router, since I have an Asus RT-AX89X that does handle 10 gig, but maybe not well? Your videos are the best!
Please do the kickstarter. As I commented last time you did a video on this board, I would buy one right now for almost any cost. It seems like a much better choice then starting a platform on the ASROCK rack x570 boards. Gigabyte needs to direct sell these things to kick up competition in this space.
Hey Wendell, thanks for the great video! I was wondering how would a mikrotik routeros compare to the pfsense/opnsense, since there was a debate that pfsense could have problems pushing 10gbit. rOS level 4 license is around $45 and it comes with everything a home router would need, so kinda curious how would it stack against pfsense in some higher speed routing.
I know this is old, but your video is relevant. You can pickup tons of power edge servers right now for around $500 with quad 10gbe a raid controller drac etc and quad xeons. I just picked one up with quad v3's for $300 on Marketplace.
A lot of useful information here as always. I would be interested in hearing what other homelab users are doing with > 1Gbe ISP connections (webscraping? actual commercial/public facing websites? etc). For my work almost everything over >1Gbe is constrained to my local network(NAS related). That said it would be interesting to see either a bifurcated or low-idle power solution that could be left on 24x7(e.g. I don't want a 100-300 watt router running a 10pm at night just to watch netflix or browser the web)
If most of your traffic is GSO/GRO friendly (which is usually the case if you are not like a ISP or data center) even a Atom C3xxx will do 8+Gbps *per core* with routing, firewalling and NAT under plain Linux networking (at least on bare metal.) Of course if you need to do something requiring traffic inspection such weak cores could become very painful very quick, or if you are routing a metric crap-ton of non-TCP traffic (no GRO/GSO, but there was some work to fix that recently at least for UDP, wonder if it has fully landed and enabled for routing yet) Still, perf/$ is prob way better on a W680/Alder lake platform, at least the more sensible SKUs, the headroom can come in handy, and the idle wattage - where its probably going to spend most of its life anyway - does seem pretty decent - go for it :)
Good cheap case option for all purpose home server is definitely has to be the node 804. $100 bucks and you can slap 8 3.5 hdd's and a bunch of ssd's. Supports micro-atx boards. virtualize your nas, router, servers do it all in one box.
Hey Wendell, Attempted to go out and find these components and had trouble finding where to buy the motherboard. It appears no one sells it standalone. I went through the gigabyte sites resellers and distributors list and found no motherboards with any of them.
This is my current build project at the moment. Never realized how underpowered consumer routers were. Always went with custom firmware on mine and got the functionality I wanted. Then I learned of how much more potential there is building one yourself. I also plan to build a plex streaming server. Would it be advisable to put in a separate machine or would it be ok combined with the router? I already have it on my NAS server (readynas) and perf sucks when transcoding is needed.
I had plex running on my Synology NAS and it had a tough time transcoding big files as well. Build a media server with a AMD 2700x and it's been a champ. I'm about to drop. 5900x in that box and spin up another VM for this project :) But yah, plex running on a PC is flawless compared to a consumer NAS. Still love my Synology though.
The "forbidden" router: running the router software as a VM while a hypervisor like XCPng actually runs on the bare metal. This would allow you to create another VM for your Plex. However, this configuration is "forbidden" because it has lots of potential for weirdness when the routing software is virtualized.
I'd love to see some comparison between sth like Vyos and Opnsense/pfsense. Would love to know how big the routing difference really is nowadays since it has been pretty big in the past, especially in virtualized environments
I am using a i3 10100 k cpu and pfsense. Might upgrade my nic down the line to a 10gug. But right now I get like 226mb down and 10mb up. Very efficient. Just need to figure out what to do w my wifi router or just switch to ap mode or buy a dedicated AOm
Would a new M2 Mac Mini with 10GIG and a few Thunderbolt to 10Gig adapters / SFP adapters work better? Seems like that's how I would go with it. Its small. Sips power. It does run UNIX flavor. Seems Like that's how I would go with it. Its also SMALL.
I wouldn’t bother, macOS is not made to be a router or firewall. And Linux on M2 is likely not there yet, especially with Thunderbolt support. If you want less tinkering, the Minisforum MS-01 is probably the better fit. And it’s around the same price as the Mac Mini M2, it also has built in SFP+ ports (Intel X710).
I have 2 kids and a wife all using our home router and I'm trying to use my Flex remote Ham radio. Would this help? Beacuse I'm bored of packet loss on my radios audio stream..
Interesting video, inspiring for our Home Labs. How does it handle VPN? Is there an Intel QuickAssist card you can recommend to improve performance? Would you be interested in doing a shootout between a Linux based routing solution versus BSD? For example IPFire vs pfSense; both bare-metal and via Proxmox.
I'm planning my first storage server right now mostly focused on expandability since I don't really know how much I have to store. Thanks for your videos on the topic. I'm still worried I'm missing basic things. DIY router is a very interesting topic. I'm wondering if it would make sense to build a forbidden router into the storage server? I was thinking it would be nice to be able to support 40gbps eventually like thunderbolt in various places. Making the server at least as good as direct storage in that regard. Is that practical? I've been thinking used epyc so far but that may not work well for high speed routing.
Can you set the bios to have this boot on power loss? You want your router auto rebooting always. It would suck to have to go hit the power button or log in to ipmi, or worse not know you lost power because you’re away and have to wait until you come home to power up your network!
I would love a couple of these to throw in my home lab at home here is Australia!! I would go in for the kickstarter group buy that Wendell suggested!!! Does anyone know where i could buy this in Australia??
I bought this same case, the only issue I had with it is the front water cooling IS NOT compatible with custom water loop blocks. It says it supports water cooling but it has to be an AIO that doesn't have the bottom outputs for tubing. My corsair XR7 only fit one way and I had to take out the front USB panel. I don't care since I used a pikvm to control it. And of course you don't need to water cool your router.
When they say the x540 can not do 2.5 or 5gbe it is technically true but if connected to another 10gbe component depending on drive speed it will do all of the range from .1 to 10gbe
Regarding the X550-T2. Intel's product page claims that 2gb and 5gb are only supported in Linux. Since pfsense is freebsd based doesn't that mean x550 would only negotiate at 1gb or 10gb in pfsense?
How necessary is ECC memory for a RAID NAS? I'd like to build a new machine to replace my current one, and wondered if its worth the money to get a motherboard like this one so I have error correction.
Was there anything done in the Tunables to get greater than 10Gb throuput? I just built a host with the 12400 running OPNsense bare metal. Iperf running on the LAN interface. Only getting 7.8 Gb test results without running multiple streams.
I’m just waiting for my first 1gbps Wi-Fi install because I’ve only had 40mbps down and 5mbps upload speed for 20 or so years since I was a kid here in UK.. Wi-Fi cuts out so much and when I eventually build a computer I’m going to invest in a better router for all my streaming box’s and any other stuff I get.. Would you recommend buying a after market router or build one for 1gbps?
I just got a internet upgrade from 1gbps to Fiberlink 2.5 Gbps. My motherboard has a 2.5G LAN port and they gave me a Zyxel AX7501 router.If I pay a one time fee of 100€ I get 10 gbps internet for the same monthly subscription. What do you think,I have a 5800x3d on B550 TOMAHAWK and the only motherboard I found with a 10g port is the X570 creation but used..I don't want to change the processor or the platform... is it worth changing the motherboard or getting a separate network card with 10g network port?
While it's good to see 2.5G ports becoming the norm now, 10GbE ports in motherboards need to really catch up, especially high-end ITX boards. I tried to find an x570 motherboard with 10GbE onboard and the only one I considered was the $1000 Gigabyte board, which I did not get. For now I'd recommend to just buy a plug-in 10GbE card. I have the Asus XG-C100C that I got for $100 USD that works well in my X570 board.
i have laying around a 3200g , B550 2x 4gb 2400 ddr4 .... and 2x Intel PRO 1000 Quad Port which i want to trade it fo Low Profile but 2 or more years ago none of the Router OS Supported WiFi 6 and i didnt have a Low Profile Quad port Ethernet
This is basically how I started my internet company in the 1990's, T-1, T-3, and serial multiport cards... Voila, internet company, initially started on a 486 dx2 back then... LOL 😂
What do you think of the mikrotik ccr2004-something? Basically a router with 2 25g sfp interfaces stuck on a pcie card that exposes another 2 interfaces over said pcie. Stick it in your NAS, hook it up to a switch and job done?
How does the hypervisor handle the E-cores? I'd think the smartest thing it could do is given each VM a perportional share of the P-core and E-cores access according the number of VCPUs assigned. I doubt any are that smart though. Last thing you'd want is VMs randomly getting all E-cores.
+1, I actually emailed their sales channel (through the MW34-SP0 page on their site), but didn't get a response. (To be fair, I'm not representing an enterprise or a system integrator, so it doesn't surprise me too much, but I was hoping....)
One thing I'm considering getting as a router + VM host is a Deciso DEC740. It's a GPU-less AMD Ryzen Embedded V1500B, in a small passive chassis. How does the CPU horsepower in one of those compare to the Alder Lake, for network tasks? I do have my eye on W680 stuff for desktop, though.
If the case is the least expensive component, why is the msrp of that motherboard well over 600 bucks. Also that motherboard is discontinued, and I cannot find one less than 900 bucks? Budget build?? I can afford the cpu at 160 bucks, but holy shnikies????
@level1techs, Is Intel VPro feature a recommended thing to have for pfsense/opnsense build or not? I searched and saw some people like to have it and others wanted to disable or totally avoid it.
Can you disable the E-cores on the Supermicro BIOS settings? I'd like to use an I7-12700K CPU with that motherboard, but don't want hassles with mixing E and P cores.
What if I were to use 7th Gen Intel Core or Pentium N CPUs? I found some MSI ECO motherboards that support them, and they boast some unique efficiency features, with second-gen micro-ATX boards using as little as 14W idle and as much as 90+ at full load, both with efficiency settings toggled.
I love these types of build your own solutions but obtaining a good OS with the features I desire has always shifted me away. After trying pre-build enterprise solutions I shifted to building with Sophos XG, PFsense, Open Sense and others I landed on just purchasing a Firewalla Gold. The price, hardware and most of all the software experience all with my current needs. I hope you review it one day.
I second this I have a Firewalla purple and just ordered the new gold plus after running pfsense for a few years and trying opnsense for half a year I think Firewalla is the perfect balance of being able to tinker with low energy use hardware and solid stability.
Anyone has a updated version or link to a more "new/modern" DIY router? The Mainboard from this video is not available anymore and on ebay it starts at 530€.. Thanks!
![Lp. Сердце Вселенной #60 РОЖДЕНИЕ ЛОЛОЛОШКИ [Финал] • Майнкрафт](http://i.ytimg.com/vi/YoR0pAV9FVQ/mqdefault.jpg)
7:26: Just to note: the eBay search results shown there are for the Dell rNDC card, not the stand-alone PCIe version. Unless you have a Dell server with a compatible onboard networking port, don't buy the parts shown on screen here.
we have 25 Gbit/s Residential in switzerland for 65$ a month but no routers to handle it 🤣🤣
Meanwhile, I’m on about 24mbps/8mbps in Australia in a city 😂
wow, this is stupidity fast for home use! can DIY a x86 router to chug this bandwidth
vyos
We have 20Mbs/10Mbs cable connection for around 25-30$ for last at least ten years, before it was 10/1 for the same price for ten years.
THANK GOD FOR O2 !!! They slowed my life so much
Fiber 7? right
Some things to add, the X550 does better at longer cable runs than the aquantia chipset. Also there are router OSes made specifically with low latency in mind like VyOS or tnsr (from netgate the makers of pfsense).
Interesting. The aquantia has been fairly flakey in my experience. Maybe I should try a shorter cable run.
Wish VyOS had a web gui, sticking with OPNSense
@@mitch7918 vyos is targeted at enterprises and carriers. Those have staff that are comfortable with CLI.
The CLI on vyos is pretty easy to work with once you understand the syntaxes and configuration modes. It does struggle with hairpin NAT, so use split DNS for that.
What are you talking about VyOS and low-latency? I fail to see what VyOS has that OpenWRT/OPNsense don't.
@@mitch7918 I'd kill for a GUI for VyOS
Regarding the AT&T BGW320-500/505 we have found a way around not using it at all on AT&T's network. It involves using an external XGS-PON ONT which has a 10G ethernet port. Availability is abysmal and it's been a community effort figuring the device out (big ups to the few people who got deep into it and figured it out), the manufacture either won't or doesn't respond to emails thus far. Thanks for the video Wendell.
where is this being tracked? just signed onto ATT Fiber 1gig
Deets and link to buy? Lol
@@Level1Techs I attempted to reply but I'm not sure if the comment got ate by UA-cam or deleted, didn't post a link but described where it was.
Don't see it even in held. Email me at Wendell at level1techs dot com and I'll make a forum post for it
Definitely going to be hunting for that on the forums, currently still have the BGW210 and getting it sorted to bypass at least that unit, but being able to pass the ONT too would be awesome if the same approach works, though admittedly less necessary as the BGW210 is the main source of my issues. Thanks for letting us know the options exist :D
Jeez, keep it up. The tidbit about pcore and ecore on linux vs freebsd was a good nugget of info. Thanks to you and the commenters. The comments about these horrible att routers was great too. Led me down a path. Maybe I can join and figure it out. I do not know how youtube bullseyed this one but I'm here.
Omg omg omg omg! I’ve been waiting for this. In my city they are upgrading from 10Gbps to 25Gbps on SFP28
what city is this?
THIS is the content we need more of. Thank you so much!
A group buy is an interesting idea. I'd do it.
7:30 I had no idea there was a Firmware update to the X550, that “unlocks” 2 & 5gbe. I’ve been hunting for the 700series that they made that supports this. Makes a great bargain, even better. 👍
I just updated the firmware on my x550 and if I plug it into a 2.5g switch port it drops to 1g.
@@grizredford8407 I wonder if there are sequential firmware updates. Perhaps still needs to be manually negotiated.
Wendell and team are like the deep dive tech friends I never had! Thanks for being you!
I've found the Wiitek SFP+ to RJ45 copper modules will negotiate 1/2.5/5/10 for cheaper SFP+ network cards and things like the UDM Pro.
I'm rockin an old Dell PowerEdge R210 II running pfsense. I slapped in a Realtek quad 2.5G network card (I know eww realtek) but it actually works fine if you install the proper drivers. May be upgrading to a R330 soonish. Love your videos, you literally do things I find interesting and I may even go to school for networking next year
Been using a ryzen 3200g in a router box since late 2019. Works fine with 3gbs fiber. For anything faster than 3gbs, you likely need openwrt as I don't think the freebsd can do 10gbs yet - unless they fixed it.
2 months ago, and that specific motherboard is nowhere to be found...any w680 board is $500 I think? that's what my searches have found. What alternatives do you have for that board? how budget are we talking here?
Sliger is a really fantastic company to work with. I had the opportunity to work with Kahlin Sliger in designing a chassis for a medical instrument. The company was so great to work with commercially, that I only ever recommend them personally.
Definitely would be interested in a group buy on this setup.
No SR-IOV on Aquantia. :(
RE: P-cores and E-cores, I think the way to go is pinning VMs to cores and using isolcpus or cpusets to keep the hypervisor from using them.
So it is forbidden after all
I've been working on a home router project over the last year, got a supermicro 1u 5019D-4C-FN8TP with xeon D-2123IT. Setup Proxmox and created a virtualized PFsense. My biggest issues were the 2.5/5/10gb negotiations. My modem has 2.5gb but had trouble getting things to work, so tried passing through the NICs to Pfsense but found out BSD has trouble with realtek drivers. Got things sorted out now using an intel i225 nic but virtualized and passed to the VM using virtio. My connection is 1.2gb but I can get a consistent 1400mbs out of it and iperf gives me stable 10gb between my homelab servers. I have the t540 nic in my desktop, if I had known about the t550 and firmware to get 2.5/5 that would have made my life alot easier. Thanks for the video!
Also have the 540. Didn't realize until this video it doesn't negotiate 2.5/5. Had I paid attention to that detail a couple of years ago I would have saved myself a whole lot of stress. Looking into 550 options now, that 5 second comment in the video just explained so much for the problems I've been having.
@@beepboopbeepboop190 I ended up getting a TEG-S762 switch for my office to connect to my desktop. Has 2x10gb ports so I don't have to worry about negotiation. 10gb to desktop then 10gb uplink to my homelab, and it has 4x2.5gb ports as well.
@@krj15489 That's a good call. Mine is installed in my firewall/router so one is connected to a 10G switch on the lan side but the other port is connected to my ISP's terminal. I'm 99% sure that's not 10G but I think it might be more than 1G. The nic on the fw's motherboard is 2.5 though so I may just swap to that for the wan.
Gotta be nice to have an Internet connection to do your router justice. :) My own network uses 1000 base T/ Gigabit ethernet. The bottleneck is the cable modem. For the router, I use the Linksys WRT1200AC with the OpenWRT on it. If you can upgrade a consumer router with OpenWRT you won't be disappointed. You can then use NFS and have it also be a light show machine - AND a router!
Great video,. especially the coverage of the nuances of the 10GbE card.
You should look into VPP (w/Linux Control Plane plugin) + DPDK + FRR.
I’m from Chile, here I have 2 isp connections, one at 800 Mbps and another at 600 Mbps. I can upgrade one of the isp connetion to 2 Gbps, and for that reason I’m building a custom router with one of those super micro motherboard with 6x10G ports that you can find on ebay.
I have a Dell R420 w/ Dual E5-2430 w/ 32GB ECC RAM and X520 2xSFP+ card booting pfsense off a SATA SSD. It has no problem with 2Gbit internet and even tested 10Gbit with it (using another machine acting as my WAN connection). I'm into it for maybe $300 and it is absolutely solid as a rock. I'll trust enterprise hardware a lot more than consumer grade hardware. Am I missing something?
I get about ~2.5 Gbps L3 performance using OPNsense with 4 cores of a 2680 v4 under ESXi. I tried with 8 cores and it scaled linearly so I'm not convinced you need hardware as good as what's on display for consumer 2.5 or 5 Gbps services unless you're going to run something like Wireguard.
this forbidden router makes so much sense in a lot of homelab setups, it's simple, a proper machine running a VM that contains your software router, and has proper hardware to connect to your switch. You can add dns, proxy, nginx, vpn, and everything network on the server, and even Firewall rules! If you don't need 10G you can get a 4 port 2.5G ethernet card for cheap and maybe even ditch the expensive managed switch all together! Assigning vlans for each port and rules to connect everything together. All at a low price and with the benefits of a VM!
Wait, that's a novel idea (to me). My switching needs are minor for non poe. Could I get 2 or 3 2.5g NICs to work like this? Assuming chassis supported it.
@@badharrow as long as the motherboard supports giving the PCIe lanes to the VM that's possible! I do something very similar and works wonders, no need for expensive switches here.
Just got an Intel X520-T2 and it pushes the two 10G connections just fine.
I wouldn't mind going in on a group buy for the motherboard. I've got quite a few alder lake cpus from XOC I was planning on using to replace my current homelab system with (3970x, 256gb ram). These would fit the bill quite well.
I don't know where your intro music comes from, but it is consistently among the best for any YT channel I know.
Great video, Wendell, it really got me thinking about a new router build down the road. AT&T Fiber sounds... less than ideal. We have CenturyLink here (1Gb) and I just hooked up my homebrew opnsense box in place of their Zyxel C4000 modem/router and we were in business. Having to work around subpar ISP hardware is never fun. Cheers!
I use the X540-AT2 in my main home workstation and home servers. I use Trendnet TEG-10GECTX and Asus XG-C100C in my other rigs
Just put in 8 Gig internet. I have the equipment to handle it, but I have opposite problems with some of my machines. Download and uploads are not symmetrical, both ways in different machines. I do need to look into building a router, since I have an Asus RT-AX89X that does handle 10 gig, but maybe not well? Your videos are the best!
Group buy for the MW34-SP0 would be great!
Please do the kickstarter. As I commented last time you did a video on this board, I would buy one right now for almost any cost. It seems like a much better choice then starting a platform on the ASROCK rack x570 boards. Gigabyte needs to direct sell these things to kick up competition in this space.
It seems AsRock Rack are also getting in W680 boards now. But I don't think they're selling them yet, they're listed as "preliminary"?
I like the 4x NVMe and 8x SATA ports on that board. I may actually buy one as an "everything" box.
Imagine in the next 2-3 years when ALL consumer motherboards will have this level of IO.
So exciting, it's like a glimpse of the future :D
The group buy for the motherboard is a good idea. I’d consider buying a new router for my University’s ACM chapter if it went through.
Hey Wendell, thanks for the great video! I was wondering how would a mikrotik routeros compare to the pfsense/opnsense, since there was a debate that pfsense could have problems pushing 10gbit. rOS level 4 license is around $45 and it comes with everything a home router would need, so kinda curious how would it stack against pfsense in some higher speed routing.
I know this is old, but your video is relevant. You can pickup tons of power edge servers right now for around $500 with quad 10gbe a raid controller drac etc and quad xeons. I just picked one up with quad v3's for $300 on Marketplace.
A lot of useful information here as always. I would be interested in hearing what other homelab users are doing with > 1Gbe ISP connections (webscraping? actual commercial/public facing websites? etc). For my work almost everything over >1Gbe is constrained to my local network(NAS related). That said it would be interesting to see either a bifurcated or low-idle power solution that could be left on 24x7(e.g. I don't want a 100-300 watt router running a 10pm at night just to watch netflix or browser the web)
I just have it because I can. 5gbps fiber is cheaper from my provider than 100mbps from Spectrum at $60/month.
If most of your traffic is GSO/GRO friendly (which is usually the case if you are not like a ISP or data center) even a Atom C3xxx will do 8+Gbps *per core* with routing, firewalling and NAT under plain Linux networking (at least on bare metal.) Of course if you need to do something requiring traffic inspection such weak cores could become very painful very quick, or if you are routing a metric crap-ton of non-TCP traffic (no GRO/GSO, but there was some work to fix that recently at least for UDP, wonder if it has fully landed and enabled for routing yet)
Still, perf/$ is prob way better on a W680/Alder lake platform, at least the more sensible SKUs, the headroom can come in handy, and the idle wattage - where its probably going to spend most of its life anyway - does seem pretty decent - go for it :)
Good cheap case option for all purpose home server is definitely has to be the node 804. $100 bucks and you can slap 8 3.5 hdd's and a bunch of ssd's. Supports micro-atx boards. virtualize your nas, router, servers do it all in one box.
Hey Wendell, Attempted to go out and find these components and had trouble finding where to buy the motherboard. It appears no one sells it standalone. I went through the gigabyte sites resellers and distributors list and found no motherboards with any of them.
Ive been waiting for a video like this, some areas in the UK have access to 3Gbps down
This is my current build project at the moment. Never realized how underpowered consumer routers were. Always went with custom firmware on mine and got the functionality I wanted. Then I learned of how much more potential there is building one yourself.
I also plan to build a plex streaming server. Would it be advisable to put in a separate machine or would it be ok combined with the router? I already have it on my NAS server (readynas) and perf sucks when transcoding is needed.
I had plex running on my Synology NAS and it had a tough time transcoding big files as well. Build a media server with a AMD 2700x and it's been a champ. I'm about to drop. 5900x in that box and spin up another VM for this project :)
But yah, plex running on a PC is flawless compared to a consumer NAS. Still love my Synology though.
The "forbidden" router: running the router software as a VM while a hypervisor like XCPng actually runs on the bare metal. This would allow you to create another VM for your Plex. However, this configuration is "forbidden" because it has lots of potential for weirdness when the routing software is virtualized.
I'd love to see some comparison between sth like Vyos and Opnsense/pfsense. Would love to know how big the routing difference really is nowadays since it has been pretty big in the past, especially in virtualized environments
I am using a i3 10100 k cpu and pfsense. Might upgrade my nic down the line to a 10gug. But right now I get like 226mb down and 10mb up. Very efficient. Just need to figure out what to do w my wifi router or just switch to ap mode or buy a dedicated AOm
what is the software at 14:51 ? looks like some sort of NAS OS from the early 2000's. This is something I would be interested in for sure.
Would a new M2 Mac Mini with 10GIG and a few Thunderbolt to 10Gig adapters / SFP adapters work better? Seems like that's how I would go with it. Its small. Sips power. It does run UNIX flavor. Seems Like that's how I would go with it. Its also SMALL.
I wouldn’t bother, macOS is not made to be a router or firewall. And Linux on M2 is likely not there yet, especially with Thunderbolt support.
If you want less tinkering, the Minisforum MS-01 is probably the better fit. And it’s around the same price as the Mac Mini M2, it also has built in SFP+ ports (Intel X710).
I have 2 kids and a wife all using our home router and I'm trying to use my Flex remote Ham radio. Would this help? Beacuse I'm bored of packet loss on my radios audio stream..
@14:46 is this GUI from a Synology NAS or something else running the same desktop environment?
I went for a Topton N305 mini pc, passive cooling, 32GB ram, 2TB nvme, 4 x 2.5Gbit ports, low idle power as seen on STH ;) (obviously no 10Gb though)
i really do like these builds just DIY everything
Yeah I have that exact spectrum router that I have to use for my static ip’s, so it’s modem -> spectrum router -> pfsense
have you done some openwrt?
I think the only usecase for E cores is to help with an IPS/IDS like suricata
That’s scary… maybe less demanding stuff like dhcp, dns services
Interesting video, inspiring for our Home Labs.
How does it handle VPN? Is there an Intel QuickAssist card you can recommend to improve performance?
Would you be interested in doing a shootout between a Linux based routing solution versus BSD? For example IPFire vs pfSense; both bare-metal and via Proxmox.
I'm planning my first storage server right now mostly focused on expandability since I don't really know how much I have to store. Thanks for your videos on the topic. I'm still worried I'm missing basic things. DIY router is a very interesting topic. I'm wondering if it would make sense to build a forbidden router into the storage server? I was thinking it would be nice to be able to support 40gbps eventually like thunderbolt in various places. Making the server at least as good as direct storage in that regard. Is that practical? I've been thinking used epyc so far but that may not work well for high speed routing.
My 50mbit connection would like to have a word with this 2,5gbit internet connection out of dreamland.
Can you set the bios to have this boot on power loss? You want your router auto rebooting always. It would suck to have to go hit the power button or log in to ipmi, or worse not know you lost power because you’re away and have to wait until you come home to power up your network!
L1Techs: Saturates 5gbE from ISP
American ISP Technician: "Ive never seen anything so fast"
All of the US: 😭
I am officially impressed!
Looking at the MB+12500 with the nas sliger case for a nas build. Just a shame the MB isnt really out yet.
Now that the Minisforum MS-01 is out. The MS-01 might be the better appliance for high speed (> 10 Gbe) routing and firewall.
It would be cool to have a short or some short video updating this video.
I would love a couple of these to throw in my home lab at home here is Australia!! I would go in for the kickstarter group buy that Wendell suggested!!!
Does anyone know where i could buy this in Australia??
Dudeeeee, the 13900 on this motherboard would be a nutty home server! VMs for days!
I bought this same case, the only issue I had with it is the front water cooling IS NOT compatible with custom water loop blocks. It says it supports water cooling but it has to be an AIO that doesn't have the bottom outputs for tubing. My corsair XR7 only fit one way and I had to take out the front USB panel. I don't care since I used a pikvm to control it. And of course you don't need to water cool your router.
When they say the x540 can not do 2.5 or 5gbe it is technically true but if connected to another 10gbe component depending on drive speed it will do all of the range from .1 to 10gbe
Regarding the X550-T2. Intel's product page claims that 2gb and 5gb are only supported in Linux. Since pfsense is freebsd based doesn't that mean x550 would only negotiate at 1gb or 10gb in pfsense?
How necessary is ECC memory for a RAID NAS? I'd like to build a new machine to replace my current one, and wondered if its worth the money to get a motherboard like this one so I have error correction.
I wonder do you need this much performance for a over 1 Gigabit router?
Was there anything done in the Tunables to get greater than 10Gb throuput? I just built a host with the 12400 running OPNsense bare metal. Iperf running on the LAN interface. Only getting 7.8 Gb test results without running multiple streams.
I’m just waiting for my first 1gbps Wi-Fi install because I’ve only had 40mbps down and 5mbps upload speed for 20 or so years since I was a kid here in UK.. Wi-Fi cuts out so much and when I eventually build a computer I’m going to invest in a better router for all my streaming box’s and any other stuff I get..
Would you recommend buying a after market router or build one for 1gbps?
Would you say the the processor and ram are more responsible for your performance and if so what other motherboard would you recommend.
Intel ark does not show ECC support for i5-12400. Is it working even though it is not listed?
I just got a internet upgrade from 1gbps to Fiberlink 2.5 Gbps. My motherboard has a 2.5G LAN port and they gave me a Zyxel AX7501 router.If I pay a one time fee of 100€ I get 10 gbps internet for the same monthly subscription. What do you think,I have a 5800x3d on B550 TOMAHAWK and the only motherboard I found with a 10g port is the X570 creation but used..I don't want to change the processor or the platform... is it worth changing the motherboard or getting a separate network card with 10g network port?
While it's good to see 2.5G ports becoming the norm now, 10GbE ports in motherboards need to really catch up, especially high-end ITX boards. I tried to find an x570 motherboard with 10GbE onboard and the only one I considered was the $1000 Gigabyte board, which I did not get.
For now I'd recommend to just buy a plug-in 10GbE card. I have the Asus XG-C100C that I got for $100 USD that works well in my X570 board.
i have laying around a 3200g , B550 2x 4gb 2400 ddr4 .... and 2x Intel PRO 1000 Quad Port which i want to trade it fo Low Profile but 2 or more years ago none of the Router OS Supported WiFi 6 and i didnt have a Low Profile Quad port Ethernet
Wendell did you do any VPN Client testing for through put speeds?
This is basically how I started my internet company in the 1990's, T-1, T-3, and serial multiport cards... Voila, internet company, initially started on a 486 dx2 back then... LOL 😂
What do you think of the mikrotik ccr2004-something? Basically a router with 2 25g sfp interfaces stuck on a pcie card that exposes another 2 interfaces over said pcie. Stick it in your NAS, hook it up to a switch and job done?
11:55 what video is this?
How does the hypervisor handle the E-cores? I'd think the smartest thing it could do is given each VM a perportional share of the P-core and E-cores access according the number of VCPUs assigned. I doubt any are that smart though. Last thing you'd want is VMs randomly getting all E-cores.
I'd really like to get that Gigabyte MW34-SPO shown here. I guess GIgabyte isn't into selling these as I can't find one anywhere.
+1, I actually emailed their sales channel (through the MW34-SP0 page on their site), but didn't get a response. (To be fair, I'm not representing an enterprise or a system integrator, so it doesn't surprise me too much, but I was hoping....)
Same, it was first announced all the way back in the spring and... nothing.
@Level1Techs Would a 5600g/5700g be able to route as well as the 12400 in this scenario?
Question in regards to OS to use. I noticed Microtik has an x68 version of RouterOS. is that any good???
Would love to see the forbidden grandchild, some of us don't have more than 1gig but also don't want the isp modem/ont
Is there any alternative to the Motherboard? Because where I live there are literally none available.
One thing I'm considering getting as a router + VM host is a Deciso DEC740. It's a GPU-less AMD Ryzen Embedded V1500B, in a small passive chassis. How does the CPU horsepower in one of those compare to the Alder Lake, for network tasks?
I do have my eye on W680 stuff for desktop, though.
If the case is the least expensive component, why is the msrp of that motherboard well over 600 bucks. Also that motherboard is discontinued, and I cannot find one less than 900 bucks? Budget build?? I can afford the cpu at 160 bucks, but holy shnikies????
@level1techs, Is Intel VPro feature a recommended thing to have for pfsense/opnsense build or not? I searched and saw some people like to have it and others wanted to disable or totally avoid it.
No sign of MW34-SPO in UK so have to use raspberry pi instead
Can you disable the E-cores on the Supermicro BIOS settings? I'd like to use an I7-12700K CPU with that motherboard, but don't want hassles with mixing E and P cores.
What if I were to use 7th Gen Intel Core or Pentium N CPUs? I found some MSI ECO motherboards that support them, and they boast some unique efficiency features, with second-gen micro-ATX boards using as little as 14W idle and as much as 90+ at full load, both with efficiency settings toggled.
Once again recommended the 12500 because it actually supports ECC and is only a few bucks more.
I love these types of build your own solutions but obtaining a good OS with the features I desire has always shifted me away. After trying pre-build enterprise solutions I shifted to building with Sophos XG, PFsense, Open Sense and others I landed on just purchasing a Firewalla Gold. The price, hardware and most of all the software experience all with my current needs. I hope you review it one day.
I second this I have a Firewalla purple and just ordered the new gold plus after running pfsense for a few years and trying opnsense for half a year I think Firewalla is the perfect balance of being able to tinker with low energy use hardware and solid stability.
I have a question? You know those cards that give you 4 M.2s from a 16x slot. Will that NIC work on that card? What about the Sata port M.2s?
would the asus Pro WS W680-ACE IPMI be an acceptable motherboard to use instead of the gigabyte used?
How much power does it consume on average?
lets us know if you do the group buy of the gigabyte intel mobo
Anyone has a updated version or link to a more "new/modern" DIY router? The Mainboard from this video is not available anymore and on ebay it starts at 530€..
Thanks!
Would I follow the same process if I didn't need a router but wanted to build my own managed network switch?