I would almost be more interested in a fan chiller plate OR maybe a little raspberry pi mounting solution... Still with your branding/ design, but with a more practical use... MAYBE even one with a small patch panel insert if we can find one for cheap enough.
Any product with dimensionality or that requires manufacturing likely won't come out of my shop for quite a while. I'm a one-man-band with decent tools, but I'm also very expensive per hour ;-)
There's something for everyone in here. Low TDP VM hosts that cost less than $400. A 64-Core Epyc build that was $20K when fully built. High end NAS, low end NAS, 1Gb PoE, 100Gb Fiber...
@@CraftComputing Your videos served as base for when I graduated as a network engineer from Uni and started working, and after a career change and opening my own business, I still reference to your content for upgrading and modifying my network.
May I suggest you try an air purifier with a particle filter? It could substantially reduce the amount of dust, at least it works really well for me! I use a relatively cheap one IKEA sells.
It's a 600sqft garage that houses wood working and laser engraving tools. It's going to be a bit dusty. Easier and more cost efficient to blow out the servers once every 6 months.
@@CraftComputing Every six months? My thought was seeing that dust, that it might keep you busy with every 2-3 months. But hey, if 6 is good enough, I'm fine with it. Considering you busy doing other dusty works there, seeing it in this particular time, it's not that bad.
@@CraftComputing No, no. I didn't mentioned anything about being offended, or anything like that. I just told you what thought was in my mind. It was a perception on periods of dust cleaning. You answered as no performance hit on it. Great. No balloons popped in your hand, and I am lucky, I don't have a rack sized infrastructure. :P Dust still a factor, haha.
For your UPS batteries, you might look into 12V LiFePO4 cells. They have integrated BMS's that work with lead-acid chargers and whatnot while still having longer runtimes and much-much better deep-cycle tolerance than lead-acid batteries. They're a bit more expensive than lead-acid packs, but you don't need to replace the whole UPS unit to benefit from the improved chemistry. I replaced the 4 lead-acid packs in my Eaton 9130 rack UPS, and they've been rock-solid for several years, despite frequent power blips and outages in my area.
I considered it, but the UPS is only there to prevent brown-outs and surges, not really for any kind of uptime insurance. For $240, I should get another 5-6 years out of the Pb battery packs, and that's plenty.
I did this with my APC SMT1500rm2u, and eventually upgraded to a SMT2200. The only thing to be aware of with APCs is they have an unusually high float voltage, and it will shorten the life of the LiFePO4 cells. Luckily with the SMT and older APC units you can enter a programming mode with an FTDI serial adapter and adjust the float voltage to within spec for the new batteries.
You're incredibly lucky! I pay between $0.32/kwH and $0.36/kW, so I have to be really picky about what equipment I need to keep running and what I have to shut down right when I'm done using it.
Here in Seattle I had the choice between Comcast "1Gig" and Centurylink 1G Fiber. Centurylink 1G Fiber is the winner hands down with 1Gbps up and down. Real world speeds are 800 Mbps down and 900 Mbps up. I attribute the 900 Mbps up to most people not uploading much of anything but comes in handy when transfering backups to S3.
Good to see your rigs, interesting Aircon for the rack . I have my rack in the basement. I use it to dry clothes. The temp doesn't get above 20c. Interesting power load. I'm at 400w with 4 servers (2 xcp-ng, soon to be proxmox, pfsense and truenas scale 24 drives) The power cost is 35p (UK price) something like $0.04 So I built a Victron system. That can power the house if there's a blackout. But the main point was to access cheap electricity overnight at 7p. So I charge up the 82kWh battery (3 phase 230v) then run the house off grid for the day. It is also the whole house UPS system.
Nice Rack Jeff, needs a little cleaning, but considering its in your garage it looks positively pristine. You are motivating me to get this sort of thing for my attic (don't have a garage or basement available) and get to learning things.
This 12:00 is the reason why I decided to go with just a regular 19 inch monitor + keyboard + mouse and then rack mounted it. It takes up more space in my server rack. I am looking for a swinging monitor rack mount that would accommodate a regular monitor but can fold up and stick inside the rack. No luck so far.
Man, i love this. It's down and dirty with money savings where needed and maybe a real nice thing in there to keep the dream alive! Cheers with a Sierra Nevada from the beer county capital of the world, North County San Diego!
I loved your lab tour. I live in the northeast where electricity definitely does not cost $0.07/KWh, more like $0.20. I have FiOS “gigabit” which provides 950 down / 880 up. I’ve had it for three years and it has not once failed, though our power here is extremely reliable. Take from that what you will :)
Currently have to pay the equivalent of US$0.47/KWh here in the UK. Really makes me want to replace my home server with something more power efficient 😅
Great to see how things are evolving with the home lab! I managed to set up 100Gb networking at home too and agree it’s pointless. I can get around 50Gb real-world file transfers using ksmbd (the kernel-side implementation of SMB which supports RDMA using RoCE) on Linux for the file server and using Windows 11 Pro for Workstations which supports client-side RDMA using RoCE (all using PCIe Gen 4 x4 NVMe SSDs on each end with fast file systems like XFS on the server rather than ZFS and NTFS on Windows). Would recommend looking into this as I get much better speeds than using regular Samba and lower CPU usage. Configuration isn’t too hard as well - ksmbd can basically just be enabled on most modern Linux flavours and is essentially a drop-in replacement for Samba. And on the Windows side (as long as it is the Workstations version or some other enterprise versions I think work as well) it all should just work with default settings (at least with Mellanox cards) if things are configured correctly on the server side. Because this uses RoCE you can just use regular Ethernet networking without needing to play around with Infiniband. But, as you say, you basically never need these sorts of speeds with anything you do in the real world as there are usually lots of other bottlenecks. Would therefore only recommend max 25Gb or 40Gb if using older cheap Mellanox ConnectX-3 cards, for home use.
Your single year worth of server rack costs is the same as 2 months of my ENTIRE home's usage in the UK. Mental how cheap your power is. We're sitting at around 27p (.43c)/kwh
Jeff is probably the leading cause of my buying more rack gear. now that my 12U is full of networking gear, UPS, and 12 thin clients in a cluster, I find myself looking at 22U cabinets and full size full depth racks, disk shelves, and used epyc and threadripper stuff on ebay. and those erying boards are quite tempting for playground usage.
Love that you did the proxmox builds with those erying boards. I ironically ordered two boards from AliExpress a couple weeks prior to you posting the video on the low power update and was building mine right when you released the video. Mine have been running flawless and they are both backed up with a stand alone pbs server
I have two racks in my house. The NAS is using 1kW and my other one has almost all my networking hardware at close to 200W. 350W with everything sounds insane!
My parents just got Starlink. Previously they had a 0.7mbps connection. They only stream tv/movies and surf the web. It has been an amazing improvement for them. The other option sucked so, so much that even with the occasional blip, it's still worlds better.
For the UPS I believe that you can just replace the batteries individually saving money or purchasing better quality or longer lasting batteries then perform a load-runtime calibration. Thats what I do with my UPSs, BatteriesPlus has Duracell batteries with a larger rating for the same size battery. Sure can notice the difference when it comes to runtime.
Helpful tip...on your monitor, press the right bottom button, then the - middle button three times to auto configuration the bottom button again and 75% of the time it works 100% of the time to auto adjust the screen.
I'm so happy with my 10gb network, it's well overkill for my needs, I have to basically stop anything else on the server to reach the limits, but boy does it feel good to know it's there, and also look at.
That t-shirt is very, very, very distracting jesus. I love it. I'll buy one just to enter meetings with it and people to only look at it and reading it over and over and over again which i have been doing 🤣🤣
I just got my first ever server rack : an old Dell Poweredge R710 dual Xeon E5649 that is actually running on Proxmox 8.1.4. I must admit that it as been flawless and more than enough for my needs. I mainly use it as a game server host (running multiples game servers at the same times) for me and my friends. I love it! I think I'm gonna build a server rack with a nas. I mean, once you taste it, you want more I guess... I also plan to max it out before starting to spend money on something newer. I want to put dual x5690 (which they are very cheap on ebay), 144GB of RAM (yes, I will use it fully) and fill up the 8 slot SAS with nas hdd for backups. (The VMs are actually running on a nvme plugged on a PCI-E adaptor, even tho it's old pci2.0 it still reach up to a 1gb/s wich is great for my heavily modded fabric minecraft server). But in fact, I'm gonna fill up the PCI ports with even more nvme that I'm gonna software raid to reach even more.
that 4 port 100g switch is only for backhaul between storage nodes. it's not meant for anything else, and it is a meme to use it in a homelab elsewise. you can utilize 100g in a practical manner in your homelab by using an n9k-c93180yc-ex as your core switch (or preferably 2 in a vPC domain). 260w each though.
While the dust triggered me, I still realize it's probably all fine. We had an HP Gen6 DL380 across the hall from an automotive salt spray test lab with shared plenum space. Server ran for 12yrs without issue despite being fully oxidized inside from salt exposure. We distributed the hard drives among the team as momentos.
@@CraftComputing Oh yeah garage dust is fine but I regularly go into industrial paint applicators where the dust is like printer toner levels of bad for stuff. I have "plant only" clothes. My home network is "suburban dad" simple since I don't want to manage it when I get home. PiHole is the most complicated thing on it. Actually now that I think about it you're probably fine since your target audience is buying stuff off eBay and putting it in used racks or something from StarTech. WifeArgro will naturally limit the noise/heat.
love these vids, good job el jeffe. I'd like to be confident enough on camera to begin a channel. right now my "extravagant" homelab goes unnoticed outside of myself and my kids. maybe one- day sir. keep it up, love the videos.
i just changed out 85 batteries at the school where i work including one of those little 1500VA ones, a few 5000VA ones, and mostly the 2k-3k APC units...all in all i had to install a chevrolet caprice classic worth of lead....i then put in a zabbix server for monitoring them as its just sooooo much better than the Ecostruxure stuff imo and watches our printers, servers, switches, etc., now also
Good video Those little TigerLake boards are cheeky I actually religated a Poweredge R720xd based server with one of those. The performance wasnt as good but the drop in power and noise at that performance was dramatic.
I still cannot believe the idle power consumption, unless most of what we just saw is powered off 99% of the time. That is considering my R730xd with dual xeonv4 is already consuming around 200W alone with all the hard drives installed. That is without networking equipment, without AC, without console, just the server alone. Having that said, electricity is around 35cent/kWh here. Fortunately I do have a big solar farm, so summers are not a problem. During the winters however I'm hurting and still.trying to figure out a way to at least be able to use the heat the rack produces as heating for the house without also importing the noise level, which failed so far.
I'm pretty sure you know this, but as the switch does not support RDMA, the best you can do to maximize the use of the 100G links are jumbo frames for now.
So you're limited to 1000/35. In Australia, NBN Co has in the last 12 months or so released 1000/50 plans and has recently dropped the price for it too. Having such high speeds are just amazing. Also, Yes, Starlink is definitely great for backup purposes or even a primary over a 4G connection in rural and remote areas.
It would be cool to see how you're doing horizontal and vertical cable management behind those devices as I've been failing for years with oem and hacky solutions, but you always seem to have some clever idea I haven't seen before.
thanks for this tour! One question - how do you manage your Mikrotik devices? My understanding is that there is no centralised management dashboard, like Unify (or TP-Link/Omada) has. Configuring each device separately, of are they support some form of stacking to be managed e.g. through Winbox? (I'm looking at building my home network with CRS328-24P-4S+RM and possibly CRS310-8G+2S+IN, but their AP range is not looking great)
Thanks for the video and the update on your Home Lab. Quick question: Do you think that I can put the Erying 11980K Motherboard on a single U case instead of a 2U Case? I am not sure what low profile that is low noise and has the right height can be used on a 1U rack mount cage (i.e., IN-WIN IW-RF100-S315)?
Jeff, if you were to ever upgrade/ replace those 1U Erying TigerLake 11800H servers, what would you go with? My question is geared towards homelab use.
CC, so did you take my advice and replace the ram on those ES boards and they no longer have the rebooting problem, or do they still have a reboot problem but you put them in your rack anyway? Nothing goes in my rack that is hardware defective so I'm curious what the resolution was.
do you have more information on the drive bay inserts for that those black 2U servers..... i just picked up one of those boxes recently but it has a faceplate on it. would be nice to have access there
Hi Jeff. Would be great to know the idle power consumption of these systems, especially of the Erying boards. As Homelabs are usually at Idle the most time this Number would be more Interesting for me.
Where I am in Canada I have a cable connection that is 1Gb/30Mb. I used to have a fiber connection that was fiber direct from the ISP site to the modem that was 3Gb/3Gb but I went back to cable because they kept screwing us over on the bill. I will say that do miss the 100% uptime of fiber though.
I wish someone would make a super shallow (maybe 9" deep) rack mounted NAS for use in 12" deep wall mounted racks. Being 5-6 rack spaces tall would be fine as long as it was shallow.
Silly question... would NFSv4 push more data from the Storinator to your workstations? Windows doesn't like it, of course, but what about Linuxy or BSDy systems?
Question: How is your wear on the consumer Silicon Power SSDs, given that they're in a ZFS array? Also - I think you have the definitions of homelab and home prod environment mixed up, lol.
Hi Jeff. You mentioned ZFS mirror on consumer NVMe SSDs. I've had really terrible performance with them, as in, terrible random write speeds when using them as VM/CT disks. What are your secrets to setting up a performant ZFS filesystem on them?
Great tour, while while I live the ability to Tinker, my wife prefers stability and unfortunately our power is .40/kWh so currently trying shift to a more efficient system.
If it makes you feel any better, my power company is a small customer-owned non-profit. The big power company that dominates the rest of the area is still $0.22/kWh.
our provider uses a smart meter to change what you are billed based on time of day. from 7PM to 7AM its 8.7ct (CAD), then depending on the time of year the other mid peak and on peak rates are 12.2ct and 18.2ct respectively
You mentioned that windows/SMB shares transfer dont come close to using 100Gb links; have you tested with iSCSI or some other block storage ? I am curious to see that
Should have just used that Brocade switch if 100 gigabit is a waste. I'm running two of the ICX6610s. One with 24 PoE+ ports and one with 24 SFP ports. 40 gig between them and one gets a 40 gig connection to my server. The one with PoE+ is used for my home, like WiFi and cameras. Connected to my UDM-Pro with SFP+, and connected to my desktop with SFP+. The switch with all the SFP ports is for my neighbors who pay me for internet. I live in a campground and their WiFi doesn't work well (campers are almost like Faraday cages). Everyone gets an OM3 line and a media converter than also injects PoE to a Ubiquiti access point. This spring I will be up to 11 (maybe 12) customers. 23:42 I feel the same way about Comcast. I'm on 600/35 from them. It's getting really really old. It's asinine how their network is mostly XGSPON but that last few hundred feet HAS to be coax, unless you're willing to pay thousands. And it's stupid how their business fiber is. They only offer crazy plans for that. A dedicated line, static IP, SLA that makes them fix an outage in a certain amount of time. I'd be fine with PON, DHCP, and a more residential SLA.
Hi could you do a Video. On how to run mostly Docker Containers. There are a lot of possibilities and I'm a little bit lost on what is the best way. For example. Do i build a powerfull server instal Linux and then just run them? Do i install Proxmox and then run them in a Container? Do I use multiple Containers? Should i use Unraid for the other benifits and only the minimal lost in performance? Should i run a kubernetis cluster with multiple small Servers? My example would be plex / wordpress / gaming server.
That is what I'm going. I'm running Unraid on an old PC of mine. But now I'm thinking about what should my setup be when I want to upgrade. What dedecated Hardware and usecase should i look into. For example I have a lot of issues with my VM for Veeam on Unraid. Do I run a seperate Server for that or do i run proxmox. But this all depends on my biggest usecase Docker and the optimale Setup i want to grow into. :) Thanks for the reply.
Would you be interested in custom (Designed and Made In House!) Rackmount Vanity Plates? Let me know here, and I'll look at adding them to the store.
Definitely
I would almost be more interested in a fan chiller plate OR maybe a little raspberry pi mounting solution... Still with your branding/ design, but with a more practical use... MAYBE even one with a small patch panel insert if we can find one for cheap enough.
Any product with dimensionality or that requires manufacturing likely won't come out of my shop for quite a while. I'm a one-man-band with decent tools, but I'm also very expensive per hour ;-)
I'd by a 2u plate like the one in your video totally.
I would love a few simple designs like the one in your video. Basically just something funny to show friends while I’m flexing my servers on them 😎
I love it, an attainable, reasonable yet at the same time completely unreasonable home lab... As it should be...
There's something for everyone in here. Low TDP VM hosts that cost less than $400. A 64-Core Epyc build that was $20K when fully built. High end NAS, low end NAS, 1Gb PoE, 100Gb Fiber...
@@CraftComputing Your videos served as base for when I graduated as a network engineer from Uni and started working, and after a career change and opening my own business, I still reference to your content for upgrading and modifying my network.
ah yes a full size rack with air conditioner very reasonable 😂
🤣😂🤣😂🤣@@michealhoward719
I always love seeing Jeff's Rack!
👈😉
Same, I think a tech UA-camr should probably plan to do this every year.
May I suggest you try an air purifier with a particle filter? It could substantially reduce the amount of dust, at least it works really well for me! I use a relatively cheap one IKEA sells.
It's a 600sqft garage that houses wood working and laser engraving tools. It's going to be a bit dusty. Easier and more cost efficient to blow out the servers once every 6 months.
@@CraftComputing Every six months? My thought was seeing that dust, that it might keep you busy with every 2-3 months. But hey, if 6 is good enough, I'm fine with it. Considering you busy doing other dusty works there, seeing it in this particular time, it's not that bad.
A thin layer of dust doesn't affect performance. If it offends a couple people on the internet, they can go home and clean their own racks.
yeah man, that had some dust
@@CraftComputing No, no. I didn't mentioned anything about being offended, or anything like that. I just told you what thought was in my mind. It was a perception on periods of dust cleaning. You answered as no performance hit on it. Great.
No balloons popped in your hand, and I am lucky, I don't have a rack sized infrastructure. :P Dust still a factor, haha.
You've got a repeated clip in the video at 19:46 where you talk about the 100G core switch.
Also that shirt is amazing.
For your UPS batteries, you might look into 12V LiFePO4 cells. They have integrated BMS's that work with lead-acid chargers and whatnot while still having longer runtimes and much-much better deep-cycle tolerance than lead-acid batteries. They're a bit more expensive than lead-acid packs, but you don't need to replace the whole UPS unit to benefit from the improved chemistry.
I replaced the 4 lead-acid packs in my Eaton 9130 rack UPS, and they've been rock-solid for several years, despite frequent power blips and outages in my area.
I considered it, but the UPS is only there to prevent brown-outs and surges, not really for any kind of uptime insurance. For $240, I should get another 5-6 years out of the Pb battery packs, and that's plenty.
I did this with my APC SMT1500rm2u, and eventually upgraded to a SMT2200. The only thing to be aware of with APCs is they have an unusually high float voltage, and it will shorten the life of the LiFePO4 cells. Luckily with the SMT and older APC units you can enter a programming mode with an FTDI serial adapter and adjust the float voltage to within spec for the new batteries.
@@xDownSetx Very interesting. Least we know it's not a drop in replacement for newer APCs. Good to know there are hacks to make it work.
Jeff, Happy New. Enjoyed the honest breakdown of your rack gear. Home Lab Sweet Home Lab :) 😎
@23:35 - T-Mobile Home Internet... no download cap, great coverage, fast upload. Much better as a main or a backup ISP.
You're incredibly lucky! I pay between $0.32/kwH and $0.36/kW, so I have to be really picky about what equipment I need to keep running and what I have to shut down right when I'm done using it.
Yikes. That sounds brutal. I believe mine is $0.08/kwH and I'm still running around the house turning off lights all day.
Love the dust spreading in the air when you open the rack door...
Just scrolling through the comments. No one mentioned anything about the shirt? The shirt shows the proper way to make Ethernet cables :D
I noticed right away. I always struggled remembering that order
Here in Seattle I had the choice between Comcast "1Gig" and Centurylink 1G Fiber. Centurylink 1G Fiber is the winner hands down with 1Gbps up and down. Real world speeds are 800 Mbps down and 900 Mbps up. I attribute the 900 Mbps up to most people not uploading much of anything but comes in handy when transfering backups to S3.
Good to see your rigs, interesting Aircon for the rack . I have my rack in the basement. I use it to dry clothes. The temp doesn't get above 20c.
Interesting power load. I'm at 400w with 4 servers (2 xcp-ng, soon to be proxmox, pfsense and truenas scale 24 drives)
The power cost is 35p (UK price) something like $0.04
So I built a Victron system. That can power the house if there's a blackout.
But the main point was to access cheap electricity overnight at 7p. So I charge up the 82kWh battery (3 phase 230v) then run the house off grid for the day.
It is also the whole house UPS system.
Nice Rack Jeff, needs a little cleaning, but considering its in your garage it looks positively pristine. You are motivating me to get this sort of thing for my attic (don't have a garage or basement available) and get to learning things.
Great video. Agreed it’s hard to make fiber look clean could always use a horizontal manager but then you waste a 1u
I am sitting here in Britain, in winter, my mind blown that you'd want to throw heat away! :-)
Very nice lab! Thanks for all the dust. 😊
This 12:00 is the reason why I decided to go with just a regular 19 inch monitor + keyboard + mouse and then rack mounted it. It takes up more space in my server rack. I am looking for a swinging monitor rack mount that would accommodate a regular monitor but can fold up and stick inside the rack. No luck so far.
I've been nerding out on homelab stuff the past few months and I'm excited the new year brings new home lab tours/ideas 😅🎉
Man, i love this. It's down and dirty with money savings where needed and maybe a real nice thing in there to keep the dream alive! Cheers with a Sierra Nevada from the beer county capital of the world, North County San Diego!
I loved your lab tour. I live in the northeast where electricity definitely does not cost $0.07/KWh, more like $0.20. I have FiOS “gigabit” which provides 950 down / 880 up. I’ve had it for three years and it has not once failed, though our power here is extremely reliable. Take from that what you will :)
I've only lost power once in the last 4 years at this house, and it was during a generational ice storm. Power was restored in 4 hours.
Currently have to pay the equivalent of US$0.47/KWh here in the UK. Really makes me want to replace my home server with something more power efficient 😅
I see the Bat'leth is always ready to grab in your server room. Being truly prepared to defend you HomeCloud is very important! 😇🤪😬
Qapla’ !
Great to see how things are evolving with the home lab! I managed to set up 100Gb networking at home too and agree it’s pointless. I can get around 50Gb real-world file transfers using ksmbd (the kernel-side implementation of SMB which supports RDMA using RoCE) on Linux for the file server and using Windows 11 Pro for Workstations which supports client-side RDMA using RoCE (all using PCIe Gen 4 x4 NVMe SSDs on each end with fast file systems like XFS on the server rather than ZFS and NTFS on Windows). Would recommend looking into this as I get much better speeds than using regular Samba and lower CPU usage. Configuration isn’t too hard as well - ksmbd can basically just be enabled on most modern Linux flavours and is essentially a drop-in replacement for Samba. And on the Windows side (as long as it is the Workstations version or some other enterprise versions I think work as well) it all should just work with default settings (at least with Mellanox cards) if things are configured correctly on the server side. Because this uses RoCE you can just use regular Ethernet networking without needing to play around with Infiniband. But, as you say, you basically never need these sorts of speeds with anything you do in the real world as there are usually lots of other bottlenecks. Would therefore only recommend max 25Gb or 40Gb if using older cheap Mellanox ConnectX-3 cards, for home use.
17:38... check out SFP DAC cables. They come in shorter lengths, works a treat!
Your single year worth of server rack costs is the same as 2 months of my ENTIRE home's usage in the UK. Mental how cheap your power is. We're sitting at around 27p (.43c)/kwh
Jeff is probably the leading cause of my buying more rack gear.
now that my 12U is full of networking gear, UPS, and 12 thin clients in a cluster, I find myself looking at 22U cabinets and full size full depth racks, disk shelves, and used epyc and threadripper stuff on ebay. and those erying boards are quite tempting for playground usage.
Love that T568B shirt from Veronica Explains
Yeah comcast upload is brutal. So glad Ziply Fiber made it to my area. 1Gig symm now is like a dream come true!!
Love that you did the proxmox builds with those erying boards. I ironically ordered two boards from AliExpress a couple weeks prior to you posting the video on the low power update and was building mine right when you released the video. Mine have been running flawless and they are both backed up with a stand alone pbs server
What I like the most about the rack is the Bat'leth on the side...
Glory to you, and your homelab!
I have two racks in my house. The NAS is using 1kW and my other one has almost all my networking hardware at close to 200W. 350W with everything sounds insane!
I really like the HL15!
My parents just got Starlink. Previously they had a 0.7mbps connection. They only stream tv/movies and surf the web. It has been an amazing improvement for them. The other option sucked so, so much that even with the occasional blip, it's still worlds better.
Yeah, if the alternative is HughesNet, Starlink is SOOOO MUCH BETTER! But it can't replace traditional wired broadband.
For the UPS I believe that you can just replace the batteries individually saving money or purchasing better quality or longer lasting batteries then perform a load-runtime calibration. Thats what I do with my UPSs, BatteriesPlus has Duracell batteries with a larger rating for the same size battery. Sure can notice the difference when it comes to runtime.
Helpful tip...on your monitor, press the right bottom button, then the - middle button three times to auto configuration the bottom button again and 75% of the time it works 100% of the time to auto adjust the screen.
I usually auto-config it, and half the time it still messes up.
I'm so happy with my 10gb network, it's well overkill for my needs, I have to basically stop anything else on the server to reach the limits, but boy does it feel good to know it's there, and also look at.
Badass shirt!! Only 5 seconds in! Ok, now ill watch the rest of the video! Hopefully, I can buy that shirt!
Link is in the description ;-)
Thanks man! Just bought one!@@CraftComputing
That t-shirt is very, very, very distracting jesus. I love it. I'll buy one just to enter meetings with it and people to only look at it and reading it over and over and over again which i have been doing 🤣🤣
Shirt by @VeronicaExplains :-)
vkc.sh/product-tag/t568b-cheat-sheet/
I just got my first ever server rack : an old Dell Poweredge R710 dual Xeon E5649 that is actually running on Proxmox 8.1.4. I must admit that it as been flawless and more than enough for my needs. I mainly use it as a game server host (running multiples game servers at the same times) for me and my friends. I love it! I think I'm gonna build a server rack with a nas. I mean, once you taste it, you want more I guess... I also plan to max it out before starting to spend money on something newer. I want to put dual x5690 (which they are very cheap on ebay), 144GB of RAM (yes, I will use it fully) and fill up the 8 slot SAS with nas hdd for backups. (The VMs are actually running on a nvme plugged on a PCI-E adaptor, even tho it's old pci2.0 it still reach up to a 1gb/s wich is great for my heavily modded fabric minecraft server). But in fact, I'm gonna fill up the PCI ports with even more nvme that I'm gonna software raid to reach even more.
that 4 port 100g switch is only for backhaul between storage nodes. it's not meant for anything else, and it is a meme to use it in a homelab elsewise. you can utilize 100g in a practical manner in your homelab by using an n9k-c93180yc-ex as your core switch (or preferably 2 in a vPC domain). 260w each though.
Wow that shirt is so nerdy, i love it!
I love the shirt man!
While the dust triggered me, I still realize it's probably all fine. We had an HP Gen6 DL380 across the hall from an automotive salt spray test lab with shared plenum space. Server ran for 12yrs without issue despite being fully oxidized inside from salt exposure. We distributed the hard drives among the team as momentos.
There's dust, sure, but it's not clogging the servers are affecting performance.
@@CraftComputing Oh yeah garage dust is fine but I regularly go into industrial paint applicators where the dust is like printer toner levels of bad for stuff. I have "plant only" clothes. My home network is "suburban dad" simple since I don't want to manage it when I get home. PiHole is the most complicated thing on it. Actually now that I think about it you're probably fine since your target audience is buying stuff off eBay and putting it in used racks or something from StarTech. WifeArgro will naturally limit the noise/heat.
love these vids, good job el jeffe. I'd like to be confident enough on camera to begin a channel. right now my "extravagant" homelab goes unnoticed outside of myself and my kids. maybe one- day sir. keep it up, love the videos.
You'll never get the skill until you try. Point a camera at yourself and give it a shot! That's how I started.
@@CraftComputing truer words. that first step is always the biggest.
i just changed out 85 batteries at the school where i work including one of those little 1500VA ones, a few 5000VA ones, and mostly the 2k-3k APC units...all in all i had to install a chevrolet caprice classic worth of lead....i then put in a zabbix server for monitoring them as its just sooooo much better than the Ecostruxure stuff imo and watches our printers, servers, switches, etc., now also
Impressed with the power draw, or lack of.
I love the Bat'leth security system :)
Impressive system Jeff! Very nice!
happy year craft computing
Good video
Those little TigerLake boards are cheeky I actually religated a Poweredge R720xd based server with one of those.
The performance wasnt as good but the drop in power and noise at that performance was dramatic.
Great tour, thanks! Are you tempted to test out the new UniFi cable modem? 👀
Hmmm.... hadn't seen that yet. I'll have to give it a look :-)
As a person who works in a steel fabrication factory, I can too easily understand your pain at the end
Love the shirt. I always say brown-white, brown so if I were to get this shirt I'd need an edited version
I need a Bat'leth on the side of my server rack now. Damn you!
Qapla’ !
Excellent stuff as usual!
Perfect amount to geeking out.
Thanks for sharing! 🎉
I still cannot believe the idle power consumption, unless most of what we just saw is powered off 99% of the time.
That is considering my R730xd with dual xeonv4 is already consuming around 200W alone with all the hard drives installed. That is without networking equipment, without AC, without console, just the server alone.
Having that said, electricity is around 35cent/kWh here. Fortunately I do have a big solar farm, so summers are not a problem. During the winters however I'm hurting and still.trying to figure out a way to at least be able to use the heat the rack produces as heating for the house without also importing the noise level, which failed so far.
I'm pretty sure you know this, but as the switch does not support RDMA, the best you can do to maximize the use of the 100G links are jumbo frames for now.
So you're limited to 1000/35. In Australia, NBN Co has in the last 12 months or so released 1000/50 plans and has recently dropped the price for it too. Having such high speeds are just amazing.
Also, Yes, Starlink is definitely great for backup purposes or even a primary over a 4G connection in rural and remote areas.
Hey thanks for sharing your home lab is something that I could inspire to having in the future. Would love to have a Vanity plate for my rack
It would be cool to see how you're doing horizontal and vertical cable management behind those devices as I've been failing for years with oem and hacky solutions, but you always seem to have some clever idea I haven't seen before.
For super clean you could probably do something with patchbox
thanks for this tour!
One question - how do you manage your Mikrotik devices? My understanding is that there is no centralised management dashboard, like Unify (or TP-Link/Omada) has. Configuring each device separately, of are they support some form of stacking to be managed e.g. through Winbox? (I'm looking at building my home network with CRS328-24P-4S+RM and possibly CRS310-8G+2S+IN, but their AP range is not looking great)
I like the TIA-568B-shirt 😀
Thanks for the video and the update on your Home Lab. Quick question: Do you think that I can put the Erying 11980K Motherboard on a single U case instead of a 2U Case? I am not sure what low profile that is low noise and has the right height can be used on a 1U rack mount cage (i.e., IN-WIN IW-RF100-S315)?
Jeff, if you were to ever upgrade/ replace those 1U Erying TigerLake 11800H servers, what would you go with? My question is geared towards homelab use.
That shirt is GOAT. I need one.
Shirt by @VeronicaExplains :-)
vkc.sh/product-tag/t568b-cheat-sheet/
@@CraftComputing Thanks!!
Loved this video.
Jeff, what about backup of your data. What solution do you use on your Homelab?
CC, so did you take my advice and replace the ram on those ES boards and they no longer have the rebooting problem, or do they still have a reboot problem but you put them in your rack anyway? Nothing goes in my rack that is hardware defective so I'm curious what the resolution was.
Did you make a video or blog about the modem rack mount? Want to do something similar so would love to see your approach.
do you have more information on the drive bay inserts for that those black 2U servers..... i just picked up one of those boxes recently but it has a faceplate on it. would be nice to have access there
What brand and how deep is your netowrk / server cabinet?
Hi Jeff. Would be great to know the idle power consumption of these systems, especially of the Erying boards. As Homelabs are usually at Idle the most time this Number would be more Interesting for me.
Is it just me or is the intro to the 100GB switch doubled at ~19:47 and ~20:15. Cool vid tho, I enjoyed watching it!
I'd buy one of the suite home lab blank plates. Just a little something to spice up the home rack.
Where I am in Canada I have a cable connection that is 1Gb/30Mb. I used to have a fiber connection that was fiber direct from the ISP site to the modem that was 3Gb/3Gb but I went back to cable because they kept screwing us over on the bill. I will say that do miss the 100% uptime of fiber though.
Why is the air condition unit not at the top where the heat doesn't affect any server equipment?
6:16 The UPS power itself?
In (phun) reference to 2:39
I wish someone would make a super shallow (maybe 9" deep) rack mounted NAS for use in 12" deep wall mounted racks. Being 5-6 rack spaces tall would be fine as long as it was shallow.
love the t658b shirt
I also use 25Gb. My 112 SSD NAS can handle it, but the link is barely getting over 10Gb. Either the fiber is bad or something else is wrong.
Silly question... would NFSv4 push more data from the Storinator to your workstations? Windows doesn't like it, of course, but what about Linuxy or BSDy systems?
Question: How is your wear on the consumer Silicon Power SSDs, given that they're in a ZFS array?
Also - I think you have the definitions of homelab and home prod environment mixed up, lol.
You might get more endurance from equal spec LiFePO batteries. A lot of people who replace consumer UPS batteries recommend them.
Love that shirt.
Hi Jeff. You mentioned ZFS mirror on consumer NVMe SSDs. I've had really terrible performance with them, as in, terrible random write speeds when using them as VM/CT disks. What are your secrets to setting up a performant ZFS filesystem on them?
Amazing. In the UK I pay the equivalent of 45c/kwh and its 100% renewable...
Great tour, while while I live the ability to Tinker, my wife prefers stability and unfortunately our power is .40/kWh so currently trying shift to a more efficient system.
Great Video! Thank you very much!
I envy your 7ct per kWh. I pay about 27ct (Euro) here in NL and that’s considered a good price nowadays.
If it makes you feel any better, my power company is a small customer-owned non-profit. The big power company that dominates the rest of the area is still $0.22/kWh.
Yep, same country and paying 37ct... My 30W NAS is costing me €100 a year to keep on.
our provider uses a smart meter to change what you are billed based on time of day. from 7PM to 7AM its 8.7ct (CAD), then depending on the time of year the other mid peak and on peak rates are 12.2ct and 18.2ct respectively
@@TheAnoniemo I am with Zonneplan. Per-hour pricing but it averaged 27ct for 2023.
You mentioned that windows/SMB shares transfer dont come close to using 100Gb links; have you tested with iSCSI or some other block storage ? I am curious to see that
Should have just used that Brocade switch if 100 gigabit is a waste. I'm running two of the ICX6610s. One with 24 PoE+ ports and one with 24 SFP ports. 40 gig between them and one gets a 40 gig connection to my server. The one with PoE+ is used for my home, like WiFi and cameras. Connected to my UDM-Pro with SFP+, and connected to my desktop with SFP+. The switch with all the SFP ports is for my neighbors who pay me for internet. I live in a campground and their WiFi doesn't work well (campers are almost like Faraday cages). Everyone gets an OM3 line and a media converter than also injects PoE to a Ubiquiti access point. This spring I will be up to 11 (maybe 12) customers.
23:42 I feel the same way about Comcast. I'm on 600/35 from them. It's getting really really old. It's asinine how their network is mostly XGSPON but that last few hundred feet HAS to be coax, unless you're willing to pay thousands. And it's stupid how their business fiber is. They only offer crazy plans for that. A dedicated line, static IP, SLA that makes them fix an outage in a certain amount of time. I'd be fine with PON, DHCP, and a more residential SLA.
I wish Comcast would lay out a roadmap for when and where they are going to roll out the mid-splits. I could deal with 200 up.
Hi could you do a Video. On how to run mostly Docker Containers.
There are a lot of possibilities and I'm a little bit lost on what is the best way.
For example. Do i build a powerfull server instal Linux and then just run them?
Do i install Proxmox and then run them in a Container? Do I use multiple Containers?
Should i use Unraid for the other benifits and only the minimal lost in performance?
Should i run a kubernetis cluster with multiple small Servers?
My example would be plex / wordpress / gaming server.
Start simple and work from there. I'm running Ubuntu on an old Dell optiplex with docker and portainer. Works great.
That is what I'm going. I'm running Unraid on an old PC of mine. But now I'm thinking about what should my setup be when I want to upgrade. What dedecated Hardware and usecase should i look into.
For example I have a lot of issues with my VM for Veeam on Unraid. Do I run a seperate Server for that or do i run proxmox. But this all depends on my biggest usecase Docker and the optimale Setup i want to grow into. :) Thanks for the reply.
Where did you get the T-shirt?
Link in the description.
I've been having fun replacing stock fans in 1 and 2 u equipment with noctua fans.
Great t-shirt!
I have found the exact same 25G limitation on the 100G switch. I pulled hair out for ages trying to get past this to no avail.
soooo, home lab is new age term for server rack? got it thanks for clearing that up