Це відео не доступне.
Перепрошуємо.
40 Gigabit Server Rack Upgrade FAIL
Вставка
- Опубліковано 11 вер 2022
- Thanks to Lexar for sponsoring today's video! Don't like our puns? Frankly my dear, I don't give a NAND. Get your RAM on here: amzn.to/3ycLgoJ
In the quest for a 40 gigabit switch in my server rack, sometimes you come across a deal you just can't pass up. For $175, I was able to get my hands on a brand new Brocade ICX 6610 40 gigabit switch upgrade. And it works... for only $3,000 extra....
BUT FIRST.... What am I drinking?
You all know I love Sticky Hands Double IPA from Block 15. Well this is "Brewer's Cut" Sticky Hands with a unique blend of 6 DIFFERENT hop varieties. It did not disappoint. But after this video.... I might need another. 8.1% ABV
Links to items below may be affiliate links for which I may be compensated
Check out today's parts -
Brocade 6610 Switch: ebay.us/ZpI6pp
HP Infiniband Network Cards: ebay.us/vIk9aE
And check out all my network gear in my rack -
- Network -
UniFi UDM-Pro: amzn.to/2RV7gPg
UniFi G3 FLEX Camera: amzn.to/2EsGFpr
UniFi G4 PRO Camera: amzn.to/32W9RhU
UniFi USW-24-POE Gen2: amzn.to/3iZcCEK
Mikrotik CRS317-1G-16S+ on Amazon: amzn.to/3p5k5Gp
Mikrotik Desktop 5-port 10Gb Switch: amzn.to/2TvVezP
Mikrotik CRS328-24P-4S+ 24-Port PoE + 10Gb Switch: amzn.to/3fHpGjf
Ubiquiti UniFi UDM-Pro Router/Firewall: amzn.to/3fA3Spm
10Gb DAC cables: amzn.to/3wLrWeJ
HP NC523SFP Dual 10-Gigabit PCIe adapter: amzn.to/3fCjEjB
Asus XG-C100F 10Gbps PCIe 3.0 Network Card: amzn.to/3g1Ulqj
Grab yourself a Pint Glass or Hoodie at craftcomputing...
Follow me on Twitter @CraftComputing
Support me on Patreon or Floatplane and get access to my exclusive Discord server. Chat with myself and the other hosts on Talking Heads all week long.
/ craftcomputing
www.floatplane...
Bossa Antigua by Kevin MacLeod
Link: incompetech.fi...
License: filmmusic.io/s...
It's been fifteen years since I first got bit by hardware licenses and they're still the dumbest thing I've ever encountered in computing. You already own the hardware! It literally costs them zero more dollars to just let you use the hardware you already bought! I will continue shaking my fist at the sky over this for the foreseeable future.
Had that happen to me with a juniper switch, it’s EOL and the web interface won’t open without a license, I am now stuck to CLI instructions.
It's horrible. You can't even operate most hardware now without license, it's dead paper weight, such a waste of materials.
brocade are pieces of shit this hurts small businesses with this type of nonsense
@@c.m.7037 yep we own nothing in todays time this is how they milk people dry we cant do this at all unless were a big company otherwise they dont give a damn about us alot of this will become ewaste eventually
Some modern vehicles even won't heat your butt if you haven't bought a license. The heater is there and the money for the hardware was spent but it just won't work.
I hate to say it but after I saw the title and then the box my instant thought was licensing fail. When I shared the link with my boss I said without watching the video just the thumbnail and the model number "what was the failure" he came back with license or unsupported SFPs. Brocade is notorious for their licensing. I have had many times when C level management or the procurement people have said I can get this for 10-100K cheaper with this other vendor and we have to then explain to them why the quote they got was for an unlicensed or base license unit. Sorry you got burned on this.
I thought the same thing
Absolutely. Brocade is atrocious about their licensing. Like, Cisco does licensing for certain feature sets, but I don't think I've encountered switches from Cisco that literally turned off half the ports if you didn't buy the license for them.
The licenses for this model are freely available with a 2 second Google search in his case, and don't affect the rear qsfp ports
@@fohdeesha Do you have to pay extra for the licences?
@@SirHackaL0t. no, hence the word "freely"
I love when you show the fails. It truly helps the rest of us avoid the pitfalls that would be expensive to find out for ourselves.
There is a very popular STH forums post about Brocade switches and licensing that might be of help to you.
It’s called: Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
Replying to try and bump this to the top. You can get yourself a license for much less than $3200.
Fohdeesha ftw
Agree'd i have the 24 port version of this currently working in my rack, only one set of 40gb ports work as 40gb though, the other set are for breakout cables only!
Thanks for posting.
@@TheTuxprogrammer Fohdeesha indeed. There are so many ICX switches going for pennies on eBay because they're not licensed. I've been buying them up and licensing them myself for my lab. They're great bits of kit when you don't need to worry about the licenses anymore :)
I swear the Aries LTT Store sight gag is my absolute favorite and I'm going to be sad when that sponsorship is over 😂...well played guys...well played
OH, finally somewhere I can chime in!
A gentleman by the name of Fohdeesha has flooded the ServeTheHome forums with their extensive knowledge of Brocade switches, and, by Googling their name, you can easily find a Setup and Licensing guide for Brocade ICX series switches :D
There is a servethehome thread called "Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)". It contains some very useful information
Licensing is getting out of hand, a license for the speed of the connection has to be up there with heated seat subscriptions
This is normal for Brocade, they've been doing port licensing since the 90s.
@@btraker Most vendors have tbf, they all suck. Microtik are unusual in them not doing it. Good guy microtik
You are limited to 55mph... please tap your NFC enabled debt or credit card against the screen if you wish to unlock higher speeds for the duration of this travel.
*stop for fuel after driving on interstate for a few hours*
You are limited to 55mph... please tap your NFC enabled debt or credit card against the screen if you wish to unlock higher speeds for the duration of this travel.
Check out Serve the Home's forum post on the ICX series, there is a way to license these now that they are all EOL. A few people I know run this switch with the config found there and it seems to be working alright from what I hear.
I got the PoE model last year. Bought it due to the STH forum page. I've left a comment but it might not be visible due to it containing a web link. I've tweeted it too.
I think he should checkout the Celestica DX010 next ;)
I'm hoping to use my Dell S5248F as part of my home lab... though I need to register it to sort out the licensing situation with that. And some solar panels as the angry pixies sure are becoming expensive in the UK
I went with a Brocade 6610 poe switch based off what I learned from STH for cheap. I love having 40g for my Truenas and 10g to a few clients. Using Connectx-3 3 40g cards
Spot on Tux..... I did that exact thing for the Brocade/Ruckus ICX7150 switches. It works fine. Check out ServetheHome's forum post. It covers how to open up the licenses step by step.
Also: I’m strongly considering starting an open source project to reprogram Fiber Transceivers. It’s silly that it’s come to that. FS has a box that does it, but it’s tied to an online portal, making it a weird proposition. I want the hardware and software to be fully open source.
I have had the same situation very recently with Cisco. I bought two Cisco Aironet 3602i AP from ebay. I thought buying used enterprise level hardware to basicly reuse was not only a good thing but also cheap as hell and a good challenge for me. Oh how wrong I was. After I got the APs I realised I need new Firmware for them as I don't have a controller. Cisco though only gives this firmware out to company's with a service license. So after back and forth with Cisco I found myself with two perfectly fine APs which I cannot use, because Cisco wouldn't give me the firmware. Not to mention these APs were already out of service but still have their autonomous ios software locked behind a giant paywall. If these APs were still being sold I could understand. But Cisco didn't sell these anymore for several months and it doesn't make sense for any companys to use this. As private Consumers cannot reuse them as well Cisco effectively creates tons of unnecessary ewaste
You're starting to figure out that these companies couldn't care less about environmental concerns beyond the ESG scam. Reduce, reuse, recycle is real environmentally sound policy/business practice yet how many of the F500 truly practice any of it?
If you try again, be on the lookout for Aruba APs that support "unified" the AP-345 and newer I think. The new ones do require tracking down their proprietary serial cable though.
But it does make *cents* for them to not allow it to be used... other customers would need to buy new hardware (with a chance it could be their products) instead of making use of old stuff. All we can do is when there are alternatives, make sure to purchase from different companies that don't restrict products like this. Radio Control (RC) hobby is a good example. A lot of RC Radio/Transmitter companies limit software features based on what product you buy or require license upgrades to enable them. FrSky is really getting popular because they don't do that. The hardware they sell have all software features enable for all models of product. Pricing is less than alternatives to for the same feature and hardware.
They sell them online because most sysadmins know how to get the firmware free online.
I have seen sites with cisco bin files kicking around btw
Thanks for sharing, another company to add to my list to the ones that I will NEVER buy any kind of hardware.
share the list pls
@@CatalystReaction Cisco charge a ton for unlocking features too on their newer routers and switches.
Has anybody made any custom firmware for these switches?
@@jamescollins6085 Nope. Really hard because vendors like using various processors (CPU) as the core of a group of switch-chips in each different switch. They of course just recompile the source code, and presto, but you and I?---NOPE (big nope). Sad, I know.
@@Doesntcompute2k Ah, I see. Thank you for explaining.
I always tell people the hardest part of IT is failing the saving throws. Sometimes you just hit a wall due to licensing, hardware configs, no longer supported firmware and just greedy vendor issues.
The good ol, "ye got a license for that m8?" pitfall
loicense*
Just a heads up, there was an odd buzz under the audio while you're in the office. Not something I've heard on previous videos, so I figured I'd mention it in case it was missed.
Love the content itself, and definitely good to know just how predatory licenses can be on this kind of thing.
I heard it when he was in front of the Cabinet. I presume It's the "noise filter" to remove all the server noise.
It sounds almost like a broken cable or a ground loop
@@deusqain No, the hum was actually gone when he was in front of the server rack. It's at the beginning and at the end of the video.
Yeah there is a definite hum when he is in his office. There is a static white noise in front of the rack as others have said but it is different than the hum as mentioned.
That buzz is very annoying
Just a heads up, I hear there's a group that frequent the STH forums that can obtain the requisite licenses for you, at least for the 10G ports. I've been looking at switching to this switch also to get a flew more 10G ports from my FlexFabric 5800 HP.
They released everything now since they are EOL. I bought a 6610-48p for $49 and everything is unlocked now.
Jeff, take a look at Fohdeesha's Brocade guide - it covers license unlocking for this exact switch.
Congratulations on being the only person in the comments spelling my nick right
@@fohdeesha ha, thanks. I have a sneaking suspicion that this was an intentional failure to make it a multi video series, but we shall see.
Quick tip with heavy rack stuff: Temporary install something in the unit below, so you won’t have to carry all the weight whilst installing the new gear.
yeah, those old HP G4 DL360's empty cases are great weight holders :+
the septup.exe from patchbox seems like a great tool for that type of scenario. Haven't used it myself though.
Sometimes, I would actually test stuff out on a bench first to make sure that the system/unit powers up and is able to do what it is supposed to do in order to look for potentially damage from shipping, especially when ordering from eBay.
setup.exe is AWESOME. I use it everyday.
this is why easy return laws need to be a thing, you can always miss something about a product and only notice it once you have it
I knew the fail after reading the title and hearing Brocade. "I bet it's licensing!" I have yet to light a brocade up before. We had one in house for a review that was sent to trial along with a 3par 3200 SAN. The fun part was the Brocade was sent all 10g copper and we had 10g MMF on the 3par. We never got around to even lighting up the Brocade. We just shipped it back. We just got a pair of sfp+ adapter plates for our Summit Extremes and ran with 10g MMF sfp+.
anybody else hear a buzzing in the main video but not the intro? Clears up after 4(ish) minutes
Yeah I'm getting that too
comes back in the outro as well. Good content as always though!
Yep same
Unfortunately that's the NSA surveillance 5g waves. Not much one can do.
No idea what happened there. We didn't notice it in editing, but it's definitely there. That's my bad....
When you said new in box the first thing that came to mind was Licensing, its why UI and Mikrotik are making great inroads for this sort of environment
As soon as I heard the words Brocade - I understood immediately why this video was labelled as a fail. Brocade even licenses each port on a Fibre switch individually and you have to buy THEIR SFP's for each port.
The opening line is awesome!
I went "40G" on the backbone of my home lab a while ago using a Mellanox sx6036 from ebay for about $200. It has 36x 40/56G ports and can work for ethernet and or infiniband - i set mine to straight up ethernet only 56G mode. 56G only works with compatible Mellanox cables/cards, but these are also pretty available on ebay (56G Compatible ConnectX3's can be had for about $50-75). It also supports a breakout mode for some ports that splits a single 40G qsfp into 4x 10G sfp+ connections - this is good for systems that only have 10G. There is a ton of great support for this model on the STH forums.
Those sx6036's went through the roof price wise when the chip shortage hit. Not sure if they have came back down to a reasonable level. IIRC that switch is nice as the power consumption is not insane like some of the others.
I have the SX6012, which wasn't working, I think I need to use the FAE command to enable off-brand DACs
Wow, an excellent price.
Yeah, I snagged one of those a month or so ago on ebay for $135 or somewhere thereabouts. It didn't include the ethernet license and after failing to find any "free" solutions I pulled the trigger on another ebay auction for the license ($50) so I still got a working switch for less than $200. So far it's working like a champ as my homelab backbone network.
@@ajhieb There are TONS of links to help unlock all those features. Shame you spent the money already. :(
Wonder if there is a crack for the license like there is for the Supermicro IPMI stuff.
yes there is, there is a site called "fohdesha" where a dude has written some info to do that
There shouldn't be a crack. There should be universal license keys for EOL equipment with a disclaimer that the manufacture no longer supports it so use it at your own risk and is for non-business use. Most of us would have been fine with it.
“Get in losers” best opener ever 😂
I think the clip at 13:50 would have made a great opener :)
Yep, even synology charges extra licenses on its home NAS systems for the surveilance station app above the "free" two cameras.
Unlike brocade, the Synology licenses are transferable
Mikrotik CRS354-48P-4S+2Q+RM could replace all your switches.
A great addition would be either the MikroTik CRS354 or CRS326 to add QSFP+ to your SFP+ setup
I love MikroTik gear. I have several at home and been flawless for me. I too eventually want to upgrade to 40 gig but for now 10 gig networking is fine. I agree the hardware licensing sucks. Especially in the used market when the gear could have had a second life only to be e-waste'd. It's really a waste.
I have two of these and they are awesome! (Well, I have the non-PoeE version) …moral of the story is to keep buying MikroTik :)
Love Microtik, it's basically enterprise gear without the licensing
I had a microtik airfiber thing in a rural town
Had a project at work with dual 20 Gbit to Cisco Fabric Interconnect. The best part of the build was finishing the DRS cluster in VMware with dual 20 Gbit VMotion ports. VMotions were insanely fast, as VMware uses both paths without any need for LAGs. It opportunistically uses paths similar to SMB multi channel.
The funny part was how the customer’s SANs couldn’t even come close to saturating a single 10 Gbit iSCSI path. Moral of the story is that spinning rust isn’t useful above 10 Gbit unless you have dozens of hard drives.
Just bought one of these now, I've done my research on it before purchase and found that there's work arounds to the issue you had, mine was pretty cheap so I'll see soon if changing the file and ID codes work. Thanks for the video
There's a guide on how to do it! Sent it via twitter dm! I've been about to pull the trigger on the switch myself and found the guide recently.
I got an R710 for free from a friend and man it's been a hell of a ride. Knew nothing about servers a month ago and now know just a tiny bit more than that. Really neat stuff to get into
This is unfortunate, but I appreciate you making and releasing the video. I wasn't aware of this even being a thing, and I'm grateful for the warning.
On the 10 gig ICX switches, you can just type a cli command to 'install' the 10 gig license and it enables the ports. It might also work for the 40 gig.
the 40gb don't require a license, but it has to be in switch mode not router, requiring the sw firmware.
To add only 2 ports are 40gb the other 2 are 10gb x 4 breakout type.
@@Oyashiro_Chama the 40gb ports work just fine with the full router firmware
@@fohdeesha Not sure if the 6610 is full 40gbps routing capable though, should work great for 40gbps switching though, I use one as my main switch and it works great and I have dozens are spare parts due to my job decommisioning all our Brocade equipment.
@@Oyashiro_Chama the 40gbE ports will route at full line speed regardless of features enabled like the rest of the ports
Considering you already use Mikrotik hardware, you might consider their CRS354-48G-4S+2Q+RM switch. Alternatively, I personally like the EX4300 series switches from Juniper.
Well, welcome to the world of QSFP :D I had a similar issue with mellanox IS5025
It’s going from ridiculous to criminal that most companies now brick their hardware unless we pay them for the privilege of using the products we buy them .so, I buy a used car from some guy and after that in order to it to work, I need to go to The automaker company and pay them also for it to start moving , and even 10 times more that i payed the original owner .
This is the primary thing killing the used market for kit such as this which in turn drives more sales so don't expect it to change any time soon.
@@davidmcken You spelled EXTORTION wrong, bruh....
im sorry to hear about that, but thank you soo much for pointing this out, i had a bid in for a Brocade unit, glad ive been outbid, will definitely never look for another one again
10:40 EE here.
Just wanna point out: 9600 is your baud rate. Meaning symbols/second. You are encoding 8 databits into one symbol meaning your effective bitrate is at 9600*8 = 76.8 kBit/s
I believe each symbol only represents one bit in RS232. And given 8b/10b encoding it only yields 960Byte/s or 7.68kbit/s
@@LucienneGainsborough You are right... It's a simple NRZ code and one might say OOK/2-ASK. So yes, each symbol represents one bit. 8 info bits, 10 code bits (consisting of start bit, data bits and one stop bit). Your calculation checks out!
As a FSE for a big company, all hardware I’ve noticed uses licenses for everything.
I was just recently introduced into an IT position and my eyes were opened when I started learning about all the equipment and the things about them.
I hope one day I get to meet you, I myself am in Wilsonville OR
I can relate - I have not a license issue I can see, but a no-firmware-access-from-the-vendor issue and thus I sit with a 100Gbps switching fabric, but no switching connections. The HPE for instance does not hand out much of the firmware if you are not having a contract with them, so before any buying one shall get hands on the required firmwares-software.
Have a read of the STH forum epic thread on these, there's solutions for your DAC and licensing issues. Probably should have had a good read before starting!
Hopefully he sees this. That thread helped me a ton
Thanks for the video mate. No wonder they are so cheap used. They basically just a 1gbe switch.
Enterprise licenses are absolutely the worst thing - as someone that has been in the industry for 16 years it has only gotten worse somehow :(
Late last year I purchased the new Ubiquiti Pro Aggregate switch to replace my HEAVY, LOUD, POWER HUNGRY enterprise Arista Networks 10G switch. The Ubiquiti, for $850 plus the cost of GBICs, is 28 SFP+ ports and four SPF28 ports capable of 25GbE. Using 10Gtek modules for 1Gb, 10Gb and 25Gb fibre, I rewired my network with slim Cat 6A cables from Monoprice, popped four Mellanox 25G cards in two big Synology NASes and two Dell PowerEdge servers and... never looked back. Silent, cool running, decently fast, and not ridiculously expensive.
I was not expecting an hilarious ad... I got one. Thank you good sir!
Arista switches are hard to get firmware updates for, but licensing isn't a problem, and it's easy to find a key to unlock optics from other vendors. 7050QX would scratch that 40gig itch.
If you take another stab at 40Gb, I'd recommend the FS S5860-20SQ. Its an FS-rebranded Broadcom switch with 20x 10Gb, 4x 25Gb, and 2x 40Gb. And 2 or more can be stacked which is nice so you can do MLAG/VPC, for that completely necessary redundancy in a home lab. I havent had any transceiver compatibility issues with a wide variety of models.
Or theres that new Mikrotik CRS504-4XQ-IN that has 4x 100Gb QSFP that supports breakout cables. Kind of wish I had waited for that when I moved my lab to 25/40Gb because its much cheaper.
I owned this S5860 and it's an amazing switch for the price. And best of all you don't need any licences to use all the speed available
You should follow that guide and get that switch running, and do a follow-up video. I was wary until I saw comments about how to get them working. But I bought one and followed the guide, and it works fine.
I do that with brackets all the time or you can unbend the top and *"retrofit"* the card. 🤓😎
Ok the RAM Joke at the beginning was Great 🤣
Time to go Office Space on that switch.
Switch Mafia: "That's a nice looking switch you've got there, it'd be a real shame if you didn't have a license to use all the ports on it..."
Dude. Overkill with the box knife lol
fodeesha on the STH forums has a full guide for unlocking every single feature on these...
Not $175, but less than the $3000 license, I work at an ITAD and we just recently listed up some Dell Networking S6100-ON modular top-of-rack switches with a single 40G module, for $770. If you want to do the work, there''s also Celestica switches for a similar price for 40G. They're all ONIE switches so you have a choice in what software to run.
Just once instead of "as always, I'm Jeff" I would love to hear, "but today, I'm Francis."
Years ago Alcatel-Lucent introduced 'port-sensitive pricing' where you had to license each port in the switch for it to work at all.
The icx 6610 is a mint bit of kit. But yes there is a license component. Hopefully someone before me has sent you to the serve the home forum on how to get the 8x10g ports going. As well as for some good info on what ports are real 40g and which are breakout only. If not I suggest hurrying on over to their forums as there is a nice lengthy thread about the icx switches
Look at the ServeTheHome forums about the ICX 6XXX series licensing. Also, the 2/2-5 and 2/2/6-9 ports are 4x 10GbE links each, so they support breakout cables, but I’m not sure if they support 40G but as long as the 40G card on the other side can support 4x 10G links and can be bonded, then you should be fine.
I run the ICX 6610-48 at home as my core switch with a MikroTik router
Thanks for the video, you taught me a major "gotcha" in planning homelab hardware.
Wow this is rage worthy!
The coil whine ground loop non isolated noise was dank.... Like that beer
I know the feeling. I just finished building a VPN for a client. Got everything done only to find out that the clients of my client would not install the VPN client software on their machines. So I configured the Web Portal of the VPN so that they could connect Via their browser. Because of some specific network issues that are out of my control I had to utilize the HTML5 clients for the users to connect into their remote desktop gateway. That part went smooth until more than 2 people tried to connect to remote desktops at the same time. The HTML5 client for remote desktop is licensed separately from the rest of the VPN. And those cost almost 10 times what the actual VPN cost initially.
Ah A teaching moment for all you home-labbers. Great video.
Reminded of an Old King of Queens episode, "The money is in the licenses!"
I say get skip 40 gig and get the Mikrotik CRS504-4XQ-IN - you can run 100 gig to the storage and editing rig and then break out the other 2 ports to 4 x 25 gig and get the rest of your servers up to 25. Win-win.
I hope Lexar were appreciative. That was amazing 🤣
Thanks for explaining virtIO vs E1000 in the Dave's Garage video! I never knew what the performance difference was! 🙏😁
Love how you said loser at the beginning of the video 😂
"Get in loser, we're going shopping"
i still get a kick out of this Lexar RAM ad, with the linus spoof haha
Welcome to enterprise networking. Cisco is no better. We bought 30k enterprise switches then had to spend another 10k to enable routing protocols. The industry is just going that way unfortunately.
WHAT
I'd expect this from NX-OS, but the code I have doesn't require licenses for routing (that I know of; by the time I get them, a license file is pre-installed and all teh routing Just Works)
(I'm old-school IOS spoiled so have never had to deal with additional licenses just to enable something like BGP or OSPF (or EIGRP, which I expect on a Cisco router for FREE; they made it))
@@junker15 It was NX-OS. You can do some routing, but the amounts you can do are limited and no supported routing protocols without having a license...or none you would want to use in 2022.
Time to get into Mikrotik my man.
Also assume anything from Brocade is going to be licensing hell.Some older gear of theirs we had per port sfp+ licensing on their 48 port sfp+ switches.
“You call that a knife?!”
“This is a knife.”
You can 3d print full height brackets. Very useful when changing and the other bracket, that you definitely saved because you always do, is *somewhere* in your basement.
Welcome to the future, we thought it would be great!
Thank you for identifying a brand that will never get my money. Ever.
I made a different mistake with my 40/56Gbit setup.
In my case, I could get the licenses from Mellanox for free as a home user/student.
The difference with my setup is that it is IMPOSSIBLY loud.
Neighbors coming over to see what the noise is loud.
'I can hear it while wearing headphones, it's on another floor of the house, and my doors are closed' loud.
Unusably loud.
I was in contact with someone at Mellanox to see if I could bypass the fan-protections so I could watercool it. But that project is stagnating.
Oh! and it also draws like...120W while IDLING?!??!! 300ish under full load?
I shelved the switch and just direct-attached my workstation to the VM Host.
The plus side is it motivated me to build a short rack, rackmount my workstation, and move my VM host next to my desk.
Why? Because DAC cables are the "cheap" option. But they aren't THAT cheap.
A single 0.5m QSFP+ DAC cost me more than TWO used ConnectX3 cards...
If he ever gets the 6610 running it's got some pretty beefy noisemakers in it too. There's a whole modding community just to quiet the fans on it.
As soon as you said brocade and eBay I thought “no way all those ports are licensed” tough break.
Licenses are freely available with a 2 second Google search
I ran into the same problem with a Cisco ISR 4321, all the ports were limited to 100mbps without a license.
Could you revisit this project with the Fohdeesha guides for licensing?
If you like the Brocade licensing issues, wait till you're met with their firmware upgrade issues... They're great fun too 😉
Juniper EX4300 is a great 40g option.
I saw it coming but it still hurt to see the typical license road block.
40Gb?!? Cmon Jeff, just jump to PCIe fabric switches already ;)
40gb is for losers
Protip, boxes of paper work great for something not too high as do cardboard shipping boxes for replacement switch parts. Stools with boxes of paper on top also work for stuff that is more than the middle of the rack assuming the weight isn't crazy like a HP C7000 chassis. For leaving a 1U convection space between equipment, the hot swappable removable PSUs from switches or servers are great for spacing things almost perfect. Better to leave a 1U space between equipment when possible to allow for improved cooling. For heavy lifting, worth getting a cheap A/C lift or engine lift from Harbor Freight to move it into place.
ah man. I'm just south of you in LO and can feel your pain brother. All my kit is aruba 3810s and got lucky that I never faced this brick wall. Cheers
Am I correct, licenses for just 2 ports is $3,500?
That is correct for the two 40Gb ports.
@@CraftComputing if you want the ICX6610-48P let me know...I decided to go Juniper after hearing how loud it was and not getting links up. I knew about the STH post mentioned above but I thought I tried the first couple ports which shouldn't need a license.
There's another comment here that mentions this, I'm pretty sure two of the 40gig ports on the rear are only used as channelized ports, so max config is 2x 40gig, 16x 10gig, 48x 1gig.
@@CraftComputing You should have tried your cables back to back with the network cards too. If you have the VPI (Infiniband + Ethernet) version instead of EN (Ethernet only) you might find the interfaces running in Infiniband mode. I believe you can setup the card to default to Ethernet, or you may need to configure your driver...that's easy on Windows, not sure about the specifics on FreeBSD or Linux.
For an added twist, if connected to a switch when first powered up/initialized the VPI card should come up as Ethernet. Later when you want to use the second port or reboot a server while the switch is rebooting or off you will wonder why your Ethernet interfaces are gone.
Something is with Aruba AP. If you update for example, an AP303 to the firmware 10.x, you also need to buy Aruba Central licences, because they removed the local VC on it...
0:53 I'm glad you're still Jeff!
Thanks for sharing the sheer pain that lack of research can cause.
Thanks Jeff! My curiosity is satisfied without putting in hours of work and $300
What's happening with the audio in this video? Buzzing at the start and sounding underwater while going through the rack.
Have a look at the Mikrotik CRS354-48G-4S+2Q+RM or CRS354-48P-4S+2Q+RM
thats the switch we use in our studio. amazing but need to consider cooling and noise, some fans are better than others on the PSU
Might be late to the party, but, you should give a try with Arista's DCS-7050QX switches, they're pretty good and cheap second handed (and quite compatible w/o licences with what I threw at it).
Good video as always, only this time you have a background beep in the video. I'm assuming it's coming from the microphone
For $175, I would make a video of myself throwing it straight into the trash and move on to a different solution
And be sure to plaster Brocade all over and mention it in multiple videos to remind and inform people to avoid them.
Being your already using Mikrotik switches would you not look to replace it with CRS326-24S+2Q+RM. I purchased this switch which has 24 10g and 2 40g ports. If you only need 2 systems at 40g it might fit the bill. I think its about double the price but it would be new and you get a warranty and no license issues.
First off, I love the glass! Fan of them lead me to lots of others I follow, like you.
Secondly, I'm using a mix of Juniper (from ebay) and Netgear (work was throwing them out) switches. I have a reserve Cisco 4948 that used to be my network backbone, but I'm just not a fan of it anymore. All still 1GbE, wanna go faster but, bills come first.