Buying a Supercomputer is Stupid
Вставка
- Опубліковано 8 тра 2024
- lmg.gg/secretlablmgclips
Thanks to Secretlab for being the title sponsor of LMG Clips this year! Check out their ergonomic gaming chairs at the link above. Your back will thank you!
The government is auctioning off the Cheyenne Supercomputer after seven years in service.
Watch the full WAN Show: • I Am Not Buying A Supe...
► GET MERCH: lttstore.com
► GET EXCLUSIVE CONTENT ON FLOATPLANE: lmg.gg/lttfloatplane
► SPONSORS, AFFILIATES, AND PARTNERS: lmg.gg/partners
► OUR WAN PODCAST GEAR: lmg.gg/wanset
FOLLOW US ON SOCIAL
---------------------------------------------------
Twitter: / linustech
Facebook: / linustech
Instagram: / linustech
TikTok: / linustech
TikTok (LMG Clips): www.tiktok.com/@_lmgclips_
Twitch: / linustech - Наука та технологія
Should had bought the Supercomputer and declare it as a tax write-off
Then it would be free!
How old are you?
@@royz_1The bit because we will never let Linus down with that.
Thats... not how that works
@@meech2163 let kids dream about it xD
All I'm hearing is that you just need a bigger building.
And more engineers
And more power.
And like 2 new PSU's
this right here is the proper mindset
and my axe
But Linus this produces enough heat to keep your pool warm in the middle of winter.
His pool?
He can warm his whole house with this, he have watercooled house, just connect that to the super computer
That legit made me laugh out loud.
Now you only need a way to keep your pool cool enough so the cooling works!
Probably the whole neighborhood alongside it
Nonononono, not just keep it warm, vaporise it over a few days (that power draw, if transferred 100% to the pool, would turn 1 metric ton of water from 20c to steam in about 25 minutes).
People for miles around would mistake the steam for smoke and be calling the fire brigade thinking someone's house is burning down!
One Super Computer, One Lan Center
To do scientific calculations. That cost more to maintain hardware than to do calculations with.....
Nope... Recycle it. Lots of metal to be used for new smartphones and medical gear.
I would still love to see the "we built a sueprcomputer"
"We built an abandoned supercomputer" there fixed it for you.
@@AllanWorks I wonder how far back you would have to go to say "we built a gaming pc that rivals a super computer (from 1994)"
And it fall in my pool
Its going to be parted out - somebody did an analysis and came to conclusion that it was worth over 750k in CPUs and memory alone, based on the going rates on Ebay.
until they flood the market.
These computers are sometimes bought by third world nations (like South American countries friendly to the US via a third party) who set them up in one of their government agencies. It's crap for the US, but a cheap and big jump for those nations' computational power.
@@TheNotOverShow There's nothing stopping one from not selling it all at once.
@@somdudewillson they did just sell it all at once,as a whole unit, and bulk sales are less profitable, although in this case everything comes in bulk.
@@TheNotOverShow yeah, AliExpress already has Broadwell gen xeons by the mountain loads. It's gonna be much cheaper than they think it will be. The motherboards will be the only thing that won't tank in value.
“I missed out on the last one, so I made one instead” banger title. could be a deep dive on supercomputing and its history with a fun little tangent about how you were able to beat cheyenne with random stuff in the warehouse.
my man it’s 145,000 cores, there’s no beating that with random stuff in the warehouse
@@Wellington-je9nx I thought they had all those spare CPU pillows laying around tho
@@Wellington-je9nx depends completely on the benchmark, if it's something like 3d rendering or AI stuff, with enough GPUs they could maybe achieve similiar speeds
@@Tgjarman they're only 32bit processors. Cheyenne requires 64
@@Wellington-je9nx You can most definitely beat CPU based supercomputerw with warehouse filled with high end GPUs as long as your workload is e.g. AI interference computation. CPUs really suck for that kind of workload with modern algorithms. And for benchmark numbers looking for max GFLOPS without checking for interconnect speeds, GPUs win hands down, too.
So, we should expect an influx of 8064 Xeon E5-2697 v4 CPUs alongside some DDR4-2400 ECC RAM on ebay.
Probably within the month
we definitely need those rdimm ecc modules for cheap knowing they are ddr4.
It’s NAS building season baybee
Was thinking the same thing.
Shout out to everybody who couldn't finish watching yesterday when the video went private
I was watching the helldiver's video and was stubborn enough to keep hitting the retry/refresh button every couple seconds to finish it. Lol
I was wondering why the comments disappeared and the video started acting weird! I assumed it was UA-cam being 💩, closed it, and forgot about it.
Is that legal?
@@danielpicassomunoz2752 Is... What legal? Privatizing your video after it got uploaded? Yeah, its your video- you own it
I watched it start to finish after seeing it appear on my feed saying uploaded 58 seconds ago. I was so confused seeing this reupload today
HPC system admin here, 4GB of RAM per core will do you for 99% of all HPC workloads. Our current system is 4.5GB per core due to number of DIMM slots and module sizes for max memory bandwidth worked out as. Plenty of bigger high end systems than ours run on less RAM per core. 1.5 or 2GB per core is common place.
Who asked 💀
@@mqccheeze in the video Linus makes a comment about how little RAM each node has. It is a comment made out of ignorance of the HPC space.Linus has little to no experience of HPC. As an HPC administrator for nearly two decades now I was adding context so people watching the video would know that actually it is a generous amount of RAM for the use case.
@jonathanbuzzard1376 And legitimately thank you for the Info, interesting addition to what Linus said.
Out of interest what kind of Workloads do you have in the HPC Space?
@@jonathanbuzzard1376 Yup, this video is full of complete nonsense. I'm management of the world's largest supercomputing grid and we're pretty much the same as yours, the vast majority of our sites are 4GB or less per core. Plenty of other ridiculous statements as well, e.g. the idea that no CPU based supercomputers are still built.
Hire me! @@BlueCosmology
Basically it's like a Race car
It not about the price itself but about cost of running it and cost over time and where to run it .
Damn we missed out on 32K gaming on a decomissioned supercomputer
If i was rich i would buy it just to run the original doom.
I guess the US government officially stopped the Stargate program...
😂
Or they finally just stole a DHD and retired Carter's custom dialing program after one too many failsafes were ignored.
They can not keep up with gregtech new horizons dev team.
I drive past the Cheyenne Mountain Complex often enough. The lights are still on, it appears. Of course after the Ori, who's the next big threat?
I was looking for this comment lol. Maybe the Russians took their gate back and the USA is using the Atlantis gate. Sure as hell more secure with the shield instead of an iris
The recycling/ decommissioning of a super computer sounds quite interesting.
His old tech hoarder mate could build up quite the ram and GPU hoarde from this super computer
Just to give a perspective on power use. Apple's former quartz foundry in AZ was adjacent to 2 dedicated switching power substations w/ a 3rd larger one just down the street. The building is about the size of any Amazon Fulfillment Center and the new Apple "World" Datacenter is now occupying a 3rd of that space. The 69KV transmission feeds for the substations were at both ends bringing transformer voltage to a dozen 4.4KA switching terminals outputting 440V. There's probably provisioning for +5MW if they wanted and more to spare. Siting a supercomputer (no junkers) would take more than LTT could possibly handle. We haven't even covered the cooling infrastructure, let alone the cabling and custom written OS software libraries. 😰
Don't you disrespect that computer
It probably made more stellar drift calculations for gate travel than you uploaded videos
The “supercomputer” does 5.34 petaflops and an RTX 4090 can do 0.083 32 bit petaflops or 0.0013 64 bit petaflops. So you’d need just 65 4090s to match the FP32 performance or 4,140 4090s for FP64
Why does the 4090 have a way worse relative FP64 perfomance?
@@M1szS Because FP64 performance is throttled for consumer products to sell more prosumer products.
@@flintfrommother3gaming oh makes sense
@@flintfrommother3gaming FP64 doesn't make sense when graphics are involved, that's why they save die space to put more useful stuff
There is a serverfarm in a city near me that has 55,000,000 Watts in bachup power generators on the roof. So ~1MW seems reasonable.
So....can it run Cities Skylines 2? 😅
No
That's rough, buddy
if you open windows taks manager, the cpu cores display has enough squares to run Cities skilines 2
Gaming Performance would probably be on par with a Core 2 Duo at best. It just isn't designed for that.
Surprisingly, no. I know, I was super surprised, too! Apparently, kind of counter intuitively, a super computer would struggle with modern gaming due to it not being optimized for it at all. Driver support is one thing, but also the game supporting such a combination of hardware is highly unlikely either-- flashback to the sluggish quad sli days.
"i have to get this stuff from butt-fuck wyoming up to our warehouse" Literally the entire trip is on interstate highways within a rounding error. You're not getting it out of the titanic.
And the entire process would be able to be written off right? Even if he never once powered it up, or filled up the coolant just making B roll of the insides and disassembly of it explaining how everything it tied together would work. And over 2-3 years part it out so you don't tank the prices so hard.
@@matthewsoules7064 no, he would not be able to write it off. All the profits from parting it out would be taxable income.
@@matthewsoules7064tax write offs are not free money. It just means you’re not taxed on the income you spent on it. In other words, if you spend $500 of your income on your business, you don’t “get” $500; You just get a refund for the income tax on that $500, or a few dozen dollars total.
Just moving that whole thing to canada would cost more than its worth
Cheyenne Super Computer? Cheyenne as in Cheyenne Mountain Complex?
They really did just auction the real life WORP😅
Would you like you play a game?
@@johngaltline9933 Love to, how about global thermonuclear war
@@FireStriker_ How about a nice game of chess?
Back when (maybe it's still a thing, I don't know) you could do custom BIOS splash screens on some boards, I had mine as a GIF typing out the conversation with Joshua. Always made me smile when I powered up.
@@johngaltline9933Later, Lets play global thermonuclear war.
That is epic, I think it’s still possible. Im going to need to try that now lol. I love to put war games Easter Eggs in my code
*stargate theme intensifies*
You could milk that for 20 videos in a series. I know we would watch.
yeah, he could probably make more than the computer costs if he tried to
They would make a lot of cool series if only one person had to watch
"especially it has not aged well with the way that the workloads for supercomputers have evolved in the last seven years [...] this thing is a CPU supercomputer, which is not how they build them right now"
What a load of nonsense. CPU supercomputers are absolutely still built, plenty of workloads for supercomputers are CPU based.
Yah
I remember 4 years ago I took a road trip to Wyoming and stumbled across this massive building with a sign that said Super Computer. Had absolutely no clue what it was at the time and spent the whole night doing research into it
8064 Intel E5-2697v4 those alone you should be able to sell at a cheap price and make the money back on it. That doesn't include the ram, racks, cabinets... Even with paying for people to disassemble it and move it and a place to store it while you do that you should come out way ahead.
Sadly, from what I can tell, most used racks just get scrapped or thrown away, because they cost more to ship than they are worth. Finding affordable, used, racks is a game of being at the right place at the right time.
I'm hoping someone does part it out, because those are exactly the model CPUs I've been wanting to pick up to upgrade my current workstation. As I recall, they are the highest core count processor in that generation, more than double my current processors. 8000+ of them flooding the market should bring the price down on them (not that they are very high to begin with). edit: correction, just short of double. Those are 18 core while mine are 10 core.
I really would love to have the empty server racks with the Cheyenne name on it.
Just a mostly empty showpiece and maybe put a few newer servers in there, but definitely not a lot.
I was thinking the racks as a show/setpiece where the most valuable bit myself.
I'm pretty sure I remember Linus recently saying his favourite video was 7 gamers 1 CPU. Imagine what could have been with 8,064 CPUs!
The million dollar computer has probably as much compute power...
65x 4090's would run circles around this thing, keep in mind this super computer is basically all CPU based, whist everything has moved onto GPU & compute power.
@@StuffJason437 that's why I said "compute" the million dollar computer (or rather rack) has a good amount of gpu power in it. All that for a lot less power.
@@thealeas Compute as in A.I accelerator's not traditional CPU compute.
"I bought a old supercomputer, can it run Crysis?" would've been a dope ass video though,
Can it run Crysis ?
No just Lemmings.
no, but it can dial a stargate
So question - given the move to GPU-compute, do you think non-commercial motherboards will bring back something similar to SLI to run dual, tri or whatever GPUs?
9:51 How would you calculate the flops of a cpu? Are we looking at the iGPU?
I remember the IBM Roadrunner HPC, an interesting combination of AMD and PS3 CPUs. Was in service for maybe four years? Like other HPCs used for classified DOE applications, it had to be dismantled with key components destroyed.
I have multiple compute nodes from a hpc server which has the cpus of the same generation and interconnects as the one sold and you can get them sometimes for free you just need to take them (some companies are very happy when people help getting old servers out/ don’t need to pay companies to take the trash out).
I also work at a data center but the servers I got are not from the data center I work at
Having lived in Wyoming, I visited the Cheyenne Supercomputer when it was new. It's shocking how quickly it fell out of date.
7 years is a looooong time in computing.
excited to see the super computer video next week!
I'd still love to see a video about a totally hacked-together super-computer. I love those sorts of mad-scientist-pushing-hardware-to-its-limits videos.
I love seeing how excited and low key disappointed Mudahar was that he was unable to buy it with you guys and now I see the title calling it stupid
You're gonna do my boy dirty like that?
Who's this guy?
@@mathhews95 SomeOrdinaryGamers. Epic YTber, makes Linux and hacking content. Does game emulation, and does research on scams. He has collaborated with Coffeezilla multiple times on some crypto scams. Also does "reaction" content (but not like react to videos, mainly only has his own opinion without or with short clips of the video).
Fr rip mudahar very uncalled for
Through my work I got to use a super computer once. Although it was headless and I ssh'd into it from my office. There IS a super graphics node, but we only used the main node
yeah we all have these crazy purchase oportunities...I have multiple cubic meters of thermionic valves aka radio tubes ....not regretting anyting.
Whoever bought it did it for the CPUs and DIMMs in it. The rest is E-waste
8,064 Intel E5-2697v4 CPUs + 313,344 GB of DDR4-2400 ECC memory at USD 480,085 = good payday for them once they part these out and put these out on the 2nd hand market for old servers to repair/upgrade. The market price of these parts will easily be double
I had a friend who was a technician working on mainframe computers and smaller. He knew that some of them had connections and wires made of gold and would pick these up cheap, pull out all the gold and then recycle the metal parts and trash the rest. He made a lot of money doing this. It was work, but he knew where all the 'good' stuff was.
What happened to the blinkenlights style super-computer that was used on the sets of the original Jurassic Park movie?
I would have like to seen Dawid Does Tech Stuff try to game on it!
Sad they didn't post a cinebench score in the listing
I have always wanted to see a Collaboration where LTT and other Tech UA-camrs get a share space building and turn it into something similar to what Star Trek has done where the tech is built into the walls and you can use voice commands to have an AI computer help you and you all figure out a way to create communicators that can work similar to those in Star Trek.
Like a building of the future type of thing, with things being automated too, like a hydroponics bay and so on.
HPC guy here, for most scientific workloads GPUs don't make sense. That's why these exist.
I remember muta was almost going to buy it too, dam having a supercomputer must be wild.
“$60,000 just to put electricity in its digital veins” sounded cool
HEY! I'm from Butt(expletive) Wyoming! Be nice!
Can we get a cluster build at somepoint? Original PS3's, pi's, hell $40 used dell workstations. People do some cool at home clusters, would love to see them covered a bit more.
Unless you already did, in that case good job..
It might be interesting to rent time on an existing supercomputer to make a video about it.
Cheyenne is the biggest city in Wyoming, it is pretty large tbh. I’m only like 6 hours away or something in southern Montana.
Yeah. Middle of nowhere is like the area north of Rawlins or the Wyoming Black Hills.
i'd definitely still watch the jank supercomputer getting built tho
It looks like it would be a good fit as a technology museum piece.
linus,, you could do the super comparison but use a few select super computers over the years. as a quick how speeds or storage has changed comparison.
What would it cost (energy) to run for a month in Canada? Of course its a Clusterf__k - its a Cluster! All the parts will appear on EBAY next month with the QTY available for each assembly being huge.
For power, you could rent 2 one-megawatt generators. Those would use approximately 140 gallons of fuel each per hour and require renting a dedicated fuel tanker truck to operate them continuously around the clock. It seems like a solid investment versus putting in real infrastructure to run them......
also noticed the change in banner
They're upgrading the SGC computer?
The infiniband switches might be worth more than the servers.
Slow? Sounds like an upgrade to me. My primary server runs DDR3. It's primarily a file server and a plex server. But I also use it to transcode ripped DVD and Blu-ray files for the plex server. I'm happy with it crunching either 3 DVD quality files or 1 Blu-ray at over 300 FPS. At this point "faster" would not really get me much.
I guess ebay will be full of x99 cpus soon, maybe its a great topic for an ltt video to return to them as the market will be full of them
Wait, the one used for the Stargate Program?
People do be hyper-fixiating off "you can buy a super computer" when it's probably 10 ton in weight and you WILL have to uninstall it yourself from the facility and get it across the whatever border they need to go thru. Ain't no one is using it again, things gonna be salvaged and chopped for parts.
what about for digital mining?
Looking at the specs. It from 2016, has Xeon E5-2697v4 Broadwell CPU's
Leaking water cooling... See.... Even supercomputer grade liquid cooling sucks....
Used prices on the E5-2697v4 are about $50 each. That's 400k right there.
3:35 i think he meant to say fuster cluck.
I'm actually hoping that some business on eBay starts parting it out. The processors in it are exactly the model I was looking at to upgrade my current workstation. 8000 of them flooding the market will likely bring the price down on those.
Wonder if it's located under the mountain
I still want to see that "super-computer" video idea. It would be entertaining to compare
Sounds like a nice homelab proxmox cluster 🤣
Linus casually roasting SomeOrdinaryGamers live
You would have to bring in high voltage from the power lines and dedicate your own transformers to
Im glad I was a part of the people that sent Cheyenne info to Linus 😅
Im running ALL the minecraft shaders on that bad boy
How else are you going to run lama 3 400b model
This is why someone needs to invent a portal gun, because this would be no problem to transport with 1 portal gun.
I think it would be fun to see if you could make a super computer type PC for home.
But Linus! You would finally have a use for full room watercooling!
annual power bill is probably at least $480,000. I wonder if that is scrap value.
Still would've been worth it for the cinebench score
but will it run crysis?
Petaflops... I remember when we measured shiz in Drystones and Whetstones to see how "amazing" it were... and someone did the math for the saleprice and the price of selling the CPU's that would give a profit IF they all sold... and that doesn't calculate all the memory, storage, cases etc. prolly sell them 1 at a time for a good price for "nerds" to have a "slice of a supercomputer" and make double his money back
Can it run crysis tho?
Great scott!
It draws more than 1.21 jiggawatts of power?
How much is it gonna cost to ship that?
for the amount of power you'd be needing you would need to built yourself a hydrodam then at least you'd have lots of water for cooling.
all im hearing from the first part is that if you want linus to buy something expensive is to send him a message about it and a dollar
Build a cluster from the parts you have in stock!
Imagine how OP you'd be in BDO with the framerate
Hearing Linus call Cheyenne “buttf*ck” Wyoming, while I am driving through Cheyenne made me laugh way harder than it should’ve.
This might just be me, but I would love to see the diy GPU HPC video.
Holy f***, with this amount of parts, you could build a Metal Gear with it.
I was buying stuff from US e-waste companies... They are very good at ripping things apart and preserving valuable parts in good shape for sale. So I am pretty sure it will sell well and these CPU cores will see some more hours in the years ahead....
yeah yeah linus......you where thinking about it