How Things Work: Super Computers
Вставка
- Опубліковано 13 тра 2019
- A visit to the NCSA Blue Waters supercomputer at the University of Illinois at Urbana-Champaign. The history of the super computing at the University is discussed and the calculating power of such machines are compared. The energy used in this one building is huge, and why it is needed. How the building is one huge air-conditioner. The insides and the racks upon racks of the computer are seen as well as the robot-driven tape storage. Under and over the computer is viewed and what it is used for discussed.
I have recently started watching this channel and I love the content presented here. So before I make my comment let me begin by saying thank you for sharing this!
Now I just want to point out that during the clip at 11:58 you begin to explain what the blade is for the super computer and while most of the information was correct there were a couple huge parts that were not correct. First all of the black thin blade like wafers you pointed out being processors are in fact the RAM not the processors and the heat sinks (the copper finned areas) are in fact where the processors are and this particular server blade has 8 physical CPU cores each with its own heat sink two of which have been removed, showing the mounting bracket below which are the silver squares in between the two copper heat sinks in the middle to the left. The silver heat sinks in the middle are heat sinks for board components and not memory aka RAM as you pointed. That again is the black blades to the left and right of the copper heat sinks.
Again you do fantastic work and I love your content so keep it coming. I just wanted to point that out to avoid confusion and make sure the correct data was presented so people were not confused and mislead.
Thank you!
I agree with all of this. Evidently, even emeritus professors forget their tweezers.
I was about to make essentially the same comment. But I figured it would have already been made as it's not a new video. I would have liked to see a real analysis of that board from Curiousmarc or Gamer Nexus.
Add me to the list. heh. Additionally, on that demo model, they've left out two of the CPUs to show the slots they plug into. I do not know, though, what that is in the middle that he was calling RAM. Just re rest of the motherboard components?
And if I could get my hands on just ONE of those blades, I'd be a happy camper.
at 12:25, those are the memory sticks. the cpus are under the coper heat sinks
@@WoodsyDotOrg Came here to see if anyone else noticed. I couldn't believe what I was watching when I saw him mislabel these components.
@@coldspringhead It's quite sad too considering how good his videos on nuclear energy have been. Was shaking my head so hard at a science professor not even knowing the basic components of a computer.
aiGeis why would you expect a professor of plasma, energy, and radiology to know about computers? The two fields are pretty separate. Also it’s not like it’s a usual commonplace board, it’s the highest end of computer products, if you were able to inspect all the components I’m sure you would get things wrong too unless your a specialist in the field.
@aiGeis re: "It's quite sad too considering how good his videos on nuclear energy have been"
Oh poo - another one who would throw the baby out with the bathwater ...
@@dylanshandley1246 You don't need to be a specialist to recognize standard components
I've actually got a feeling they recorded that scene specifically to trigger youtube viewers.
If I had professors like this one I would have been major on every subjects..such an amazing teacher!!!
that s basically where computers where at their beginning! Cool to be here witnessing this technology ramping up, thanks to EnergyProf to keep us in the loop all around the world!
This guy is amazing, I love his work. Who thought learning could be interesting?
Tricia is awesome. I want a job there.
Thank you! Keep the videos coming. You should be the Secretary of Energy
The center section of the blade identified as RAM, is actually the VRM Voltage Regulation Modules for the CPUs. the RAM flanked the CPUs in DIMM Modules on either side of the CPU sockets.
Cite please? (Link to manual, installation or configuration document.)
@@uploadJ Check out this link www.nersc.gov/assets/Uploads/ShalfXE6ArchitectureSM.pdf It should give you everything you are looking for!
@@uploadJ Source, Engineer in a high performance computing lab. 30+ years experience handling similar equipment. Same way a plumber knows the difference between a abs P trap and a pcv S trap. Its my gig to know.
@@NorthmeadowMedical nice find. with hi res rendering, yes. 100% VRM modules. 1 per CPU. The staggering is so it has wide enough pathways to delver the power to the cpus in that tight area. Smart layout.
@Jeremy Fleming
That's not what I'm looking for, that was not in the original request ("expert testimony").It's not that you can't be trusted, its just that you can still be wrong. W/o consulting the manufacturer's documentation everything is speculation, guesswork, flawed recall involving people's aging memory and the like. Remember, PRIMARY DOCUMENTATION RULES!
Thanks for sharing the video.
The nice things about datafloors is that you can just take out the tiles if you need to add new infrastructure and/or rack units. This floor may look large and it is for just one (collective) machine. But many datacenters (which hold conventional multi-purpose servers) have multiple of these floors which hold a ton more servers and are mostly connected to the ISP network and routed out to their respective paths.
But can it run Dwarf Fortress with two hundred dwarves and max embark size?
Depends. Did any of the dwarves bring their pet cat?
The banter in this video is on point! 1:16 and 6:31 lol
I have been in IT for over 40 years.
I’ve been in my share of data centers that Halon fire suppression. The reason we don’t use Halon anymore is that most of the servers are so cheap that it’s easier to replace than repair.
They are used to protect the data...
Can it run crysis ?
Probably not. Specialized hardware is kind of weird that way.
@@WarrenGarabrandt It's a joke, dude. Back when Crysis came out, it was so absurdly over engineered that most PC's couldn't run it on max settings. It became a meme to ask anyone bragging about their hardware if it could run Crysis.
Haha, that was my first thought as well. Awesome.
It was attempting to run it at Max Settings during this video.
Awesome
That raised floor is beautiful! I wish the one I used to work under was 6ft up.
i first heard about blue waters when i was looking at stuff from leigh orf who was using it to figure out how tornadoes work. he had really cool stuff and it was neat to now also see where the data came from :)
Truly great sciencing
WTF is sciencing? Invented your own word?
I am where I am supposed to be.
I love these videos. You exceptionally brilliant people are beautiful to watch to a partial nerd like me but you need a Babel fish.
When the water cooling is being used in the winter, is the heat transferred to other buildings before going to the storage tanks?
Floor space use, electrical power usage and air-conditioning needs were the very same outlying requirements when ENIAC was king.
How about harvesting the heat from the data center to heat other buildings or generate electricity?
I wonder how updated is this facility in 2021?
Those rows were the ram. Ram doesn't generate much heat, the CPUs do, and thus reside under the copper heatsinks.
The middle of the board appears to be Mosfets so it no doubt makes up the power conversion than provides the roughly 1.2v required for the CPUs.
In my understanding computers generally get expanded in a parallel manner, because their clock speeds are fixed to around 5Ghz as it stands. So that is ok for applications that can be done at the same time, but if new tasks require the output of a former stage, the process stalls. 12:38 the stuff in the middle looks like 2 unpopulated cpu sockets without the heatsinks, the stuff on the left looks like memory, and the stuff on the right looks like voltage regulation.
agree the computer components were incorrectly identified.
LEED certification seems pointless when you consume 24MW.
LEED cert is a joke. You can get the same LEED point for installing bicycle racks as you get for properly installing insulation or not building your walls entirely out of glass.
I believe its about efficiency. For given infrastructure, how much real electricity its using. Also as said in the video, 24MW is the theorectical capacity, not the actual use
@@hyeon-seoyun6940 all certification is a joke, its a system to regulate which companies get all the work.
Cracking a joke by referencing MTG. My people.
The laugh from the lady tell you he hit it right on the head lol
I worry about supercomputers and Urbana, Illinois. After all, the HAL9000 became operational on 12 January 1992 at the HAL Laboratories in Urbana, Illinois...
A clear reference to the superior supercomputer climate at UIUC as the institution has always been the pioneer in supercomputer technologies that started with ILLIAC!
What happened on the 14th of May, 2019?
She seems very intelligent.
"Tap into" "Blue Water" .
Obviously Plumbers Look after the Cooling System and are in Charge of this Facility.
Good Presentation.
When I looked at the thumbnail and the title of the video, somehow I thought It is a scene Nathan Ingram being interviewed about "the machine". Though soon realized there is no such scene in the person of interest show and he's of course not the same person.
Dude, and I was proud of my comp build RIP
What's more impressive: a massive semi truck rumbling down the road with hundreds of horsepower getting 3 MPG, or a small, compact car built just big enough for the job that gets 50 MPG? You didn't build your computer to simulate an atomic bomb or compute the last digit of pi. You built it for a specific purpose that (if it's gaming anyway) this supercomputer probably can't even run. So, don't sell yourself short.
@@WarrenGarabrandt Im an AI engineer. I built my computer to train deep learning models. Yeah, that's not simulating an atomic bomb but google isnt using one single dope desktop build to train their DeepMind models. So Id say id have more use for the this super computer than tryina play minesweeper on it. lol
@@jermeekable yep. They got you a little bit here. Heh.
How would a modern gaming PC compare to an old supercomputer? How far back would you have to go for a modern PC to equal an old supercomputer?
All new stuff since my time in grad school in the late 1980s at UIUC. Completely unfamiliar.
Supercomputes, ah the fun of planning for MW of cooling and power usage. NERSC at LBNL is interesting, the power limit is just how much copper can be shoved into the walls.
back in late 2018 sitting around 13-15Pflops puts you around top 15. GPU power was just starting to show how strong it could be if your workload uses it.
most DC have 3-4ft tall below ground sections which is fine. you really shouldn't be walking around there much.
Someone needs to learn the ABCs of fire. A paper wood, B flammable fuel like gas oil, C electrical fire. Water cannot put out an electrical fire, it's only A rated. Halon is ABC rated and will not harm electrical equipment. Also fire fighter have breathing gear for smoke, which also works for Halon. If you have a small fire, one would use a fire extinguisher not the overhead suppression system. If you use a water fire extinguisher on electrical fire you could die.
or worse, lose data
@@MrMorbo420someones gotta protect all the por-- data...
I guarantee you that lady was wrong. They are not using water as a main fire supression.
Tape storage? I didn't think anyone still used tape back up anymore.
I also wanted to add this link for everyone asking for a manual or guide to the Cray Xe6 blade the professor pointed out. www.nersc.gov/assets/Uploads/ShalfXE6ArchitectureSM.pdf
www.cray.com/sites/default/files/resources/CrayXE6Brochure.pdf
This should answer all of your questions, if it does not please feel free to comment and I will be sure to answer any question you may have!
3:43 Wouldn't some kind of water cooling be more appropriate for this kind of scale? Idk...
edit: 14:43 nvm they have water cooling.
its pretty fast for a super computer...
but what about super duper computers?
Hi, I'm from Harvard. Would you like a scholarship? This is Big Brain thinking.
So anybody can use the super computers ? Does it have a OS like Windows ?
I wish she would stop clapping her hands 😂😂😂
I wish you hadn't drawn my attention to it!!! :)
So, the old movie named, The Forbin Project, this one could win the competition of the top 2 computers, then take over n control us with computer #2 as a teammate
I too choose not to measure myself according to some unimportant ranking procedure. I choose my own rank.
i'm kind of warming up to illinois.
Yeah, but can it run Crysis?
Not a chance.
On ultra
So - can I tap in and farm bitcoin?
Some say it is able to run ARMA 3 at over 60 fps...
1:30 my ego's 1202 alarm just went off.
Interesting that they don’t use haylon or some other gas.
Didn't know there was such a cut throat competition and politics in super computers
lol they should use a nuclear reactor next to that super computer to make the energy cost very low
There’s a nuke power plant maybe 50 miles away near Clinton. I don’t know if the research reactor (near the old building made for ILLIAC IV and used for the Cray X-MP) is still in operation as it was when I had an office a hundred yards from it. Didn’t produce power.
This facility, I see from a bit of looking on line, is south to southwest of most of campus, well away from the Engineering campus and relatively near the campus coal-fired plant, I should think (haven’t been back to UIUC in almost 30 years). Used to walk near there on warmer winter days to clear my mind.
Great topic and interesting video footage. Audio was very disappointing.
what, no free youtube dubstep backtrack? im ok with that
As an aside, I think she's gorgeous as well as informative.
She looks like a carpet muncher.
There's a reason why reaction/responses, (or 2nd camera shots) on video are known as "noddies"; the natural human noises of encouragement don't transfer well to video.
GTA5 would run like hell on this.
GTA won't run at all since it only works on Windows and I can tell you that this system is not Windows.
@@MasthaX It also would only use the resources of one blade, cutting performance to a thousandth of the total
Is this, "Hal's little brother" (Hal being the 2001 movies computer that was an AI computer, if you have never seen the movie)
First year of higher education should be social education. As an introvert, it would've been helpful.
re: "First year of higher education should be social education"
I would have quit; something like that should have been part of - wait for it - HIGH SCHOOL.
... And, just like in the '60s Lunar shot, wait a few more years and ALL of this will fit in a matchbox and be powered by a coin cell.
"A few more years" until you have 13 PFLOP's in your pocket? That's a hard no. But I like your moxy!
Computation is "free", as in, it doesn't cost any energy or generate any heat. Your computer actually only generates heat in the CPU when it has to force a transistor to clear its value, to reset for another computation. This is mostly why your CPU doesn't really use much electricity or generate much heat when it's idle, but can use an order of magnitude more power and produce an order of magnitude more heat when it's actually doing something like playing a game. The evicting of information from memory (including the states of transistors used in computations) removes information from the computer and radiates it out as heat. Information isn't destroyed, it is radiated away as heat, increasing the entropy of the universe.
With that in mind, there is actually a lower limit to how much power and heat a computer can use/release at a minimum for a certain amount of work. Unfortunately, the lower limit for 13 petaflops of computational power is well beyond a coin cell and a matchbox sized heat radiator. But I do get where you're coming from.
We had computers in the 60s that were so amazingly inefficient that with technology advances we now have phones with orders of magnitude more processing power. We will certainly get faster and more efficient computers in the future. I doubt we'll ever get a desktop PC with that much processing power, at least not so long as we are using silicon transistors.
I am surprised that they don't use carbon dioxide or nitrogen to put out any fires
This guy is a chick magnet .... she is all in awe lollllll
So they can afford to run a supercomputer, but they can't afford two microphones? 😆
Next weeks lottery numbers.
Pointing at RAM. "These are the CPU's"…
The CPU's are under the "pink" heatsinks. You can see the socket on one, and the CPU on the other one where there's no heatsink.
"These are the memory." Those are NVMe drives for storage.
You are right there are not ram. But they arent storage either. Voltage Regulation Modules. Each CPU has its own power supply that is a field replaceable unit. Hence the Beefy Heatsinks and being in the hot section. These blades have no storage. Everything is netbooted over the infiniband and storage is handled by master nodes. Source, Im an engineer in a high performance computing lab.
@@MrMorbo420 They looked very similar to heatsinks for M.2
My mistake about those, thought they used scratchdisk.
@@ptilrem reasonable guess. Mutli cpu systems use VRMs that can be replaced in a jiffy in case of failure. They have diagnostic systems that will identify problem parts and send codes to master consoles to generate work orders.
@@ptilrem with massive distributed computing like this there is no local storage. It is all communicated over the 1200gbps infiniband. Local storage would be way too slow.
15:13 "Here we have... Russian cold water..."
Headlines: Extra Extra, Illinois Professor of Energy slips up his hidden pro-communist Soviet love during tour.
It's no coincidence that Chernobyl manages to invade all the neutral topics he might be discussing or even full on Katyusha takeover into teaching Soviet technology.
Those are ram sticks not cpu's, the cpu's are under the copper heatsinks
I worked at an IBM facility and there were huge raised floors like that, but they were only about 2 feet deep which meant you had to pull floor tiles to get where you were going, and it was cramped for space.. 6ft tall and you can walk through them is absolutely wonderful.
Regarding solving 'global warming - climate issues'.. super computers don't do a bit of good when the software, the models that the programmers create are wrong.. Ever computer model made so far has been proven wrong by real world measurements.. All of them err on the side of hyperbole.
ROFL points out the CPU points to the ram... points out the Ram points to the VRM's :p
Minutia* WRT the balance of the video.
* the small, precise, or trivial details of something.
I believe that the way to our Creator is appreciation (child heart wonder) of the capacity humans have to create amazing things. There comes a transcendent moment, when we are sincerely amazed, that we have an opportunity to open our heart to what I believe is the Spirit of God speaking to us.
Dude is f'n RUDE. Continually interrupting that girl.
Mic everyone talking.
13 petaflops really isn't that impressive, and you wouldn't need "millions" of laptops to equal the raw FLOPS of that supercomputer. A teraflop of computing power on a GPU costs about $50 in 2019, factoring in the cost of the rest of the system (RAM, CPU, etc.) and that's 1/1000 of a petaflop. So if you had the energy and space to run it, you could slap together 13 petaflops with $650,000 or so. Real supercomputers cost a lot more because they're a bit more space-efficient than putting together thousands of desktop systems, and there are very expensive, high speed interconnects between the different parts. For embarrassingly parallel algorithms that require little to no communication between the various parts, though, you wouldn't need those expensive interconnects, and could just use a bunch of independent systems.
I have a feeling this might be from 2015.
@@JeBuSv20 the SC their talking about "Blue Waters" was built in 2012.
So sure, your GPU offers several dozen Teraflops of capacity. But definitely not your CPU, which runs at Ghz*Cores*Ops/Clock (8 Double-Precision FLOP/cycle for Ryzen 8, up to 32 on some high-end Intel). So even with a 64 core Ryzen at 4Ghz, you’d need 500 per petaflop to equal that performance.
Also the GPUs often only have those rates when you do single precision (32 bit) floats, when most scientific computing demands double or sometimes even quad precision. And those often get slower exponentially, not linearly.
Hahaha he touched the RAM slots and said it was the CPU. WRONG
I gotta say, these how things work videos don't really match the high standard you set with your excellent whiteboard videos on fission and energy economics. This doesnt really answer any questions about what makes a supercomputer different from a regular computer, or how they can split up a problem to thread it efficiently on thousands of processors, or how the system synchronizes physically quite distant components or moves data between them
the splitting up of problem answer actually makes your synchronize data question redundant. splitting up the problem into manageable slices eliminates the need for all the components to know the answer. the splice of data output each component works on will be uploaded to storage.
nothing about the hardware of a supercomputer is that special other than in some cases networking (theres rare examples like intel xeon phi) the special part is networking and the software that allows thousands or tens of thousands of nodes to work problems.
Tape drives? Who uses tape anymore? Haven't see a tape drive since the '80s.
Yes. Super computers. Good application for their use case. Large storage at a good price
Stop interrupting, professor! It sends a bad message and it's disrespectful. Her patience is amazing
All his videos with other people are like this. I kinda get it, he wants to make it more like an interview. Also I think these videos are in line with a class he teaches so he wants to hit certain points because they'll be on the test.
He's a teacher. You expect him to sit quiet for long?
Play minecraft !!!!!!!!!!!
Dude, thats ram not cpus... someone knows literally nothing about computers...
How about a link to a manual of document where I can verify this?
@@uploadJ you don't need a manual, you can literally see it. You can clearly see near the middle where 2 copper heatsinks have been removed there is an empty CPU socket and a socket with a CPU in it. What he pointed to as the CPU rows are clearly RAM DIMMs and what he pointed at in the middle calling RAM look like slot mounted VRMs.
@Mr. Potato re: " you don't need a manual, you can literally see it. "
>> Stimulus generalization; you think you see what your eyeball reports, when in fact you may not be looking at CPU sockets but rather power supply modules. STILL no documentation to confirm ANYTHING.
@@uploadJ *sigh*
Here STILL no documentation but just 5 seconds of google and even better. ua-cam.com/video/EvBdRL1J7kE/v-deo.html
Points to the AMD G34 sockets which you can see in this video have one empty, one occupied and 6 with a copper heatsink.
Points to the DIMM sockets which you can see are fully occupied with DIMMS in this video.
Points to the connectors for voltage convertors otherwise known as VRMs (voltage regulation modules) in between the two rows of CPU sockets then shows the VRMs which you can see occupying the slots in this video.
Is having it come from the Director of Manufacture Logistics at Cray good enough confirmation?
@@uploadJ Question: can you elaborate on this "stimulus generation" a bit please? I'm confused. I haven't gotten any stimulus recently. Am I in the wrong generation?
They should mine bitcoin with it.
It's a computer for the good of humanity not for greed
She should know about super computers. Does it sound as if she knows about super computers? Why is she giving a lecture on super computer when she doesnt know much about super computer!
Have you noticed this "professor" teaches a lot of children using exclusively only the vocabulary of an elementary school teacher? He doesn't EVER seem to slip up a natural more complex word or 2 ever, anywhere as if he's only been exclusively teaching elementary school his entire life.
He has a PHD.
I bet she's single.
Looks like a muff diver.
@@supercooled What is that?
Bob Lob Law lesbian. Muff is another word for vagina.