The Grid, CERN's Global Supercomputer - Computerphile

Поділитися
Вставка
  • Опубліковано 11 жов 2024

КОМЕНТАРІ • 235

  • @lewisbarnes
    @lewisbarnes 10 років тому +2

    If any of you ever get to visit CERN there is a brilliant display model globe which highlights cities around the world that are part of the Grid project. Similarly there is a cool Google Earth download which may be available somewhere online. Surprising how many institutions help out with the shared processing requirements :-)
    (I used to work there and always enjoyed guiding people and finding their city/university on the map)

  • @TheSpacecraftX
    @TheSpacecraftX 9 років тому +72

    10 Gigabits per second sounds low for such high performance hardware.

  • @guymartin1949
    @guymartin1949 11 років тому +2

    Another great video Brady. Thanks for sharing it with us.

  • @Uristqwerty
    @Uristqwerty 11 років тому +1

    There's also random-access seek time. A tape needs to slow down and then reverse direction to get to a previous part of a file, while a CD or HDD just needs to wait for part of a rotation, so the disc can always spin at full speed.
    Since many uses of a PC tend to require random rather than linear access to storage, HDDs (or SSDs) are more common in general usage, while tapes are mainly limited to specialized purposes, such as large backups. Not much demand for home-user affordable tape drives.

  • @ShaunHusain
    @ShaunHusain 11 років тому +1

    My guess is they're really referring to programs written in a multi-threaded way where each thread can be independently run on a separate processor and then the data can later be recombined (SETI@home). Generally speaking this isn't going to be prone to issues with latency. In games where multiple real time events are being combined latency is an issue, UDP is used to reduce the network traffic/chatter.

  • @CorvaireWind
    @CorvaireWind 11 років тому

    No, folks were discussing durability and types of storage. I was just making the point that in 5 years storage will be handled in extremely unique and powerful ways. Very unlike the last 30 years. Quantum storage utilizing synthetically created 0 energy crystals.

  • @ShaunHusain
    @ShaunHusain 11 років тому

    I do believe that's the idea. I believe it's generally referred to as a thin client architecture. You have client hardware that runs some software which gives you a display and reads your input then everything else gets sent to a server for processing. If cellular technology were improved I could see this happening relatively soon. It can already be done if you setup an AWS account and remote in your effectively running a thin client at that point (using the term loosely).

  • @moondanceff
    @moondanceff 11 років тому

    They are really reliable, they don't crash that often, and they offer a pretty good storage density/retrieval speed. Every main frame uses tape storage, weird, right? It's safer than having something that spins, or could blow up.

  • @Kd8OUR
    @Kd8OUR 11 років тому

    Yes people, data tapes are still used and should NOT be laughed at if you are doing any serious data storage.
    For the record voyager 2 is fitted with an 8 track player and it's still doing alright. on that same note I am sure there are still a few VW vans with 8 track players

  • @JanCRefsgaard
    @JanCRefsgaard 11 років тому

    In Genomics and Proteomics which I have experience with there is a huge discrepancy between raw and processed data, a raw spectrum can be a list of a million numbers, where a processed spectrum can be a list of 100 peaks arriving after signal processing.

  • @CreightonMiller
    @CreightonMiller 11 років тому

    We appreciate your service, Captain Buzzkill.

  • @gds7012
    @gds7012 11 років тому

    They are high capacity, quite reliable when stored correctly and very cheap. They have very slow access time, but I guess they are used for long-term storage and archive, not as a storage for current calculations

  • @jibby626
    @jibby626 11 років тому

    The main reason people use HDD is because they have the read/write hardware with the platters, so all HDD can be used at once. When using tape its like using CDs or DVDs. The storage device is separate to the read,writer, co unless you have multiple tape drives, you can only use one at a time.
    Anther reason is that since the drives cost up to $2000 the home user can't afford the drive itself, even though you can get 1TB for 25 odd dollars.

  • @KarnKaul
    @KarnKaul 11 років тому

    Of course, my home internet line itself is 25 Mbps. Point I was making was: enterprise utilise a lot of WAN optimisation (like WAAS for data and codecs for voice/video) to significantly reduce required bandwidth, since SLA-grade leases are very expensive.

  • @michalkononenko9833
    @michalkononenko9833 11 років тому +2

    Have there been any instances of scientists getting sucked into the grid and entering the Tron universe?

  • @systemvoid287
    @systemvoid287 9 років тому

    10 Jiggabits per second? Wow. That's impressive!

  • @klaxoncow
    @klaxoncow 11 років тому +1

    As measured in June 2013, around 95% of the top 500 supercomputers in the world were running some flavour of Linux.
    Linux rules the roost in the supercomputer world.

  • @chiblast100x
    @chiblast100x 11 років тому +1

    That last part raises a question in my mind... is there a meaningful or fundamental difference on the software level between a multi-core processor of today and the multi-processor computers that were on the enterprise/professional market in the '90s?

  • @voveve
    @voveve 11 років тому

    There will ever be a moment when technology will be so far ahead of us that we will struggle to be able to use it at its best?

  • @Pendoza84
    @Pendoza84 11 років тому

    They are the most reliable source of storage. The lifetime of tapes extends way longer then of hdd's/ssd's.

  • @SamuelTSteiner
    @SamuelTSteiner 9 років тому +3

    ok do NSA next. I wanna hear those numbers.

  • @SEThatered
    @SEThatered 11 років тому

    Agreed. I also often pronounce borrowed words in their original transcription if I know one.
    Language is based on "preferred pronunciations" anyway, so it is not wrong by all means, it's just a matter of choice.
    For example I have several names that mean all the same:
    Eugen, Evgen, Eujeño, Egen, Gene, Jevgenij, Евгений, Женя, Uxio etc.etc.
    I do not think that people are doing something wrong when they call me by their local iteration of my name.
    This topic is just not worth arguing about IMHO.

  • @DoctorDARKSIDE
    @DoctorDARKSIDE 9 років тому +3

    Question: do they run The Grid on a *nix based OS?

  • @stellarfirefly
    @stellarfirefly 11 років тому +1

    This makes me wonder if at least parts of their data set are divisible enough to be processed through one of the public distributed computing solutions, e.g. BOINC. I know that LHC@Home exists, but that's primarily small-scale verification of Grid results, IIRC. I'm wondering if *more* of the core data can be processed in this way, if the time were taken to find out how.

  • @caboosukin
    @caboosukin 11 років тому

    I love her accent... and her head bobbing is mesmerising

  • @DoubleM55
    @DoubleM55 11 років тому

    Because tapes beat any other medium on almost all aspects when it comes to archiving huge amounts of data. Price is the biggest advantage of tapes, when compared to hard disks or flash memory, tapes are virtually free. They have very good data density (Many TB per tape), and are reliable for long term storage, unlike other mediums. Only reason why tapes are not used in PC industry is because they are not practical for random access (They require rewinding).

  • @666Tomato666
    @666Tomato666 11 років тому

    Tapes are much, much cheaper per gigabyte of storage. Remember that not only you need the media itself (which is not much cheaper than your ordinary HDD) but you need a computer that is connected to it. In case of tapes a single computer can mange thousand of tapes, while a single computer can mange at most few dozen HDDs

  • @iSirButters
    @iSirButters 11 років тому

    So true. Man, i got chills just thinking about it.

  • @jpmthemonk
    @jpmthemonk 11 років тому

    The grid can store and handle over 25 petabytes while the human brain (according to the current understanding of neurology) can hold about 2.5 petabytes (of course, degradation, re-allocation and basic chemistry do not allow us to sequentially process even a cent of that). Computationally, I imagine the grid is more powerful, what with quantum computing for certain processes now on the horizon; but the software isn't up to snuff with what our brains come pre-loaded and kickstarted with. Yet.

  • @Dickie6990
    @Dickie6990 11 років тому +1

    I spilled my drink when she said "Higgs boson" @ 03:12.

  • @C0MPUTERPHILE
    @C0MPUTERPHILE 11 років тому

    Thanks! I hoped that was the case (I have annotations turned off by default, and I'm not going to change that when too many ruin the entire video by using spam annotations).

  • @Fottrel
    @Fottrel 11 років тому

    the grid
    the human brain is just better suited to different tasks, but in terms of raw processing power, computers have been winning for at least 35-45 years now.

  • @dannymiller7860
    @dannymiller7860 11 років тому

    Wait Brady were you at CERN recently?! My dentist was at CERN a couple weeks ago! Did you see my dentist? I bet you did.

  • @ThomasAttree
    @ThomasAttree 7 років тому

    I believe they use Ceph for their distributed storage.

  • @klaxoncow
    @klaxoncow 11 років тому

    Well, the point is that other words in the English language that come from that same Greek root - "giant", "gigantic", etc. - have all been translated with a soft "g".
    So, yes, I should rather have said something like "respecting English etymology" there instead. Thank you for the correction.

  • @TactileCoder
    @TactileCoder 11 років тому

    Modern tape drives can last several decades in storage in a climate controlled storage facility. Obviously they're not designed for heavy, every-day use as the seek times on the drives is very, very long, but they're perfect for long term backup and storage.

  • @hanssondaniel
    @hanssondaniel 11 років тому

    Will there be a Grid@Home project? I would gladly donate some computational resources to the progress of science, and I'm sure many others would too!

  • @H3kler
    @H3kler 11 років тому

    Looking at the comments in this video, it might be worth doing a video on Storage. Get people into world of SANs, LUNs, DFS and other fun storage related acronyms.

  • @dxw3895
    @dxw3895 9 років тому +18

  • @Seegalgalguntijak
    @Seegalgalguntijak 11 років тому

    Wikipedia tells me, we're even already at 8.5TB per tape cartridge.
    But modern file systems like ext4 already superseed these limitations by large, in ext4 you have a max file size of 16 TB and a max file system size of 1EB, so we'll have some time for technology to catch up. However, these data centers probably use file systems like XFS or ZFS with max file sizes of 8 or 16 EB and max volume sizes of 16 EB. No comparison to crude consumer level technology like NTFS... ;)

  • @chmis3
    @chmis3 11 років тому

    Wow, it wasn't until I read the description that I realised I'm listening to a man from my own country.

  • @DontMockMySmock
    @DontMockMySmock 11 років тому

    The metric prefixes come to English through French, by people who were probably more familiar with Latin than Greek. The soft G is understandable from that origin, even if it's not true to how the Greeks pronounce Greek.
    Ultimately, though, in English the hard G is probably more correct just because people will look at you weird if you say it with a soft G.

  • @Valsainte
    @Valsainte 11 років тому

    Thanks for all the info on tapes! :-)

  • @Nathan-ji2nd
    @Nathan-ji2nd 11 років тому

    I do believe that the theoretical limit for NTFS is 16 EXAbytes, so I wouldn't say it is unreasonable to be implementing such a thing at this point...

  • @KarnKaul
    @KarnKaul 11 років тому

    My workplace houses about 10,000 employees. Its WAN line is 30 Mbps.
    PS - I work for Cisco.

  • @PaulBunkey
    @PaulBunkey 11 років тому

    Quantum computing today is like talks about AI back in 60's. Much enthusiasm, much interest, but it will take 10 years only to understand how far we are from even trying to do that...

  • @mitchumsport
    @mitchumsport 11 років тому +1

    this was awesome. I wish I knew even more! thanks

  • @nedelkosm
    @nedelkosm 11 років тому

    Is it possible to have extra footage on that grid similarities - grid idea ?

  • @23PowerL
    @23PowerL 11 років тому

    'Modern' Greek pronounces it that way (actually slightly different), but the word is derived from ancient Greek.

  • @onkeldagobert5401
    @onkeldagobert5401 11 років тому

    They simply are cheaper per GB than hard drives.
    So if you store big amounts data the lower cost per GB pays well over time.

  • @noccy80
    @noccy80 11 років тому

    I would guess because you have no seek time when saving the data to a tape. As long as you got enough tape, you can keep recording through the experiment and then transfer it to a seekable media at a later time.

  • @ramuk-
    @ramuk- 11 років тому

    Cheaper, more reliable, They are slower too so they are mainly used for backups or archiving.

  • @Za8riX
    @Za8riX 11 років тому

    The surname in the description is misspelled; is not Pradillo but Pardillo, I think.
    P.S.: is spanish and Im spanish so it would be ankwad if Im wrong

  • @ShaunHusain
    @ShaunHusain 11 років тому

    No the answer I was trying to say is no. Information is intentionally dropped as it goes through the Tier 0 "data per event is around one million bytes (1 Mb), produced at a rate of about 600 million events per second...First, it runs dedicated algorithms to reduce the number of events that CERN physicists are either already familiar with or consider uninteresting. The physicists can focus their analysis on the most important data - that which could bring new physics discoveries."

  • @Eliphas_Leary
    @Eliphas_Leary 11 років тому

    "I want him on the grid until he dies playing!"
    -MCP

  • @ThalesII
    @ThalesII 11 років тому

    How does the Grid deal with the latency that comes with using computers in distant parts of the world?

  • @thenorup
    @thenorup 11 років тому

    Brady made a video about that on Sixty Symbols... though it's kinda confusing.

  • @mkaatr
    @mkaatr 11 років тому

    In terms of hardware I think you need a single a tape drive, and you could increase the storage by adding many tapes. A tape does not need reading heads like HDD, does not need a motor attached to it, which would make tapes cheaper.
    Also tape drives are used for the data that you don't need to change rapidly or access randomly, e.g. Backups unlike HDD where you could rewrite the data again and again.
    I suppose you need an expert to tell is what I said is right or wrong. :)

  • @LordXelous
    @LordXelous 11 років тому

    This is so nerdy, I love it.

  • @trefod
    @trefod 11 років тому

    Yeah well, I learned to ignore that. Few people can distinguish between bits and bytes and an 80Gigabit line sounds weird and implausible. I think she meant a regular ten gigabit line.

  • @Valsainte
    @Valsainte 11 років тому

    Not a computer expert here -- can someone explain why they use tapes for storage? I was surprised they weren't using more "modern" media. Thanks.

  • @MasterEnsis
    @MasterEnsis 11 років тому

    It's pretty common to use tapes as a backup medium. It's because they are more stabil and reliably then other storage devices.. Also they can store a very big amount of data. The tape with the biggest capacity at the moment has a size of 8.5 TB.
    They are mostly used for backup and archiving

  • @ShaunHusain
    @ShaunHusain 11 років тому

    Also I'm sure the underlying servers that are storing the data then at Tier 1/2 are setup with either RAID 5 or RAID 1 to ensure data integrity. I just picked up a few servers setup for a non-profit that had RAID 1 on two servers and one server was a duplicate of the other... talk about redundancy (you'd think they were doing something top secret).

  • @SuperDevastetor
    @SuperDevastetor 11 років тому

    You can create a RamDisk and create a virtual "hard drive" on RAM, which would be way faster than a tape, HDD or even SSD for that matter :)

  • @onkeldagobert5401
    @onkeldagobert5401 11 років тому

    Depends on what you want to do.
    If you want it to play chess, the grid wins big time.
    if you want it to create beautiful music or other pieces of art or just having a descent conversation, i'd go with the brain.
    Computers have a really hard time doing a lot of things that seem so easy for us humans, like having a conversation (Turing test).
    Because in the end they still are glorified beefed up calculators.

  • @TheMohawkNinja
    @TheMohawkNinja 11 років тому

    I am aware of that, but what of the data that actually get's though all of the filtering?

  • @GereonB
    @GereonB 11 років тому

    I know that feeling - we have 10Gbit up and down at my University, but we can't really use it... :(

  • @Gedom666
    @Gedom666 11 років тому

    relaxing cup of café con leche in Plaza Mayor... -_-'

  • @Niki_0001
    @Niki_0001 11 років тому

    I think your school has one (or a few) 1Gb connections that the 2000 pc's are connected to.

  • @ashwith
    @ashwith 11 років тому

    So do they use a entirely custom made OS (and not, say a Linux kernel)? If yes, is there more info I an read up?

  • @spektrum1983
    @spektrum1983 11 років тому

    LHC@home member since 2005 :P

  • @ShaunHusain
    @ShaunHusain 11 років тому

    Much of the data is pared out as it passes through the different tiers in the system. There's something like millions of particle collisions per second so a lot of the data has to be filtered out to avoid storing tons of information that isn't pointing at anything noticeably interesting or known things. My guess is theoretical physicists require lots of repeats to consider something proven.

  • @TheMohawkNinja
    @TheMohawkNinja 11 років тому

    I know it's a lot of data, but is it mirror RAIDed at all? It would be really bad if the drive(s) that held the evidence for the Higgs boson, or any other important discovery were to fail.

  • @ShaunHusain
    @ShaunHusain 11 років тому

    home{.}web{.}cern{.}ch/about/computing/processing-what-record My guess is the data after being slimmed down and processed some then gets stored and is redundant etc. through the Tier 1 and Tier 2 parts of the grid. Researchers then access the data via terminals hooked up at the Tier 3 level of abstraction from the main gathering point.

  • @KarnKaul
    @KarnKaul 11 років тому

    10 Gbps is the DC network bandwidth, not the WAN bandwidth!

  • @MetrotomX5
    @MetrotomX5 11 років тому

    Well, for one thing, many more simultaneous downloads.

  • @JanCRefsgaard
    @JanCRefsgaard 11 років тому

    if you go with 1 kb = 1024 bytes (2^10) then it's more like 3.37 * 10^6 (~12% higher)

  • @PalaeoJoe
    @PalaeoJoe 11 років тому

    Thank you for correcting me.

  • @jibby626
    @jibby626 11 років тому

    SSDs decay with re-writing. The information at the LHC is written once then stored for the next year, or longer, so there would be no real decay.

  • @klaxoncow
    @klaxoncow 11 років тому

    Anyway, in truth, I always say "gigabyte", "gigahertz", "gigabits" with a hard "g", just like everyone else does.
    I was really just defending Doc Brown's pronunciation as historically accurate (in the '50s, "jigga-" was the official pronunciation, according to the US National Bureau of Standards), not suggesting anyone change their pronunciation to a soft "g".
    I'm not changing pronunciation and I don't expect anyone else to either.
    Keep calm. All is well. I accept your rebuke in good spirit.

  • @natheria4933
    @natheria4933 6 років тому +1

    Welcome to the beast.

  • @tacosattack282
    @tacosattack282 11 років тому

    Network bandwidth (mentioned in video) does not equate to network latency. Besides, thalesll wasn't suggesting they had low latency, he was asking what techniques Cern uses to minimise it. Don't be so condescending when you haven't taken the time to read the comment.

  • @rich1051414
    @rich1051414 11 років тому

    Well, to be more elaborate, tabe reading is only done by a few of the computers in the grid. This process is slow, so the whole grid queued up for this would be very wasteful. Only a few computers access the tape, then distribute the data where needed for processing. Each rack holds a single computer that does part of the overall load. Each individual rack would not be very fast, but it doesnt need to be, as the workload is spread out so much.

  • @FranciscoSilva84
    @FranciscoSilva84 9 років тому

    you ask what type of back up use?

  • @jibby626
    @jibby626 11 років тому

    Tape is faster, cheaper, more durable, and has a higher capacity than HDD, unless what you mean by accessing the tape physically putting the tape in the reader.
    Also the speed of a game is determined by the RAM, GPU and CPU.

  • @rhodesy761uk
    @rhodesy761uk 9 років тому

    what make and type of cpu do they use intel i would guess zeon

  • @TactileCoder
    @TactileCoder 11 років тому

    And? Tape storage is the standard long term medium for data centers.

  • @iwouldlikenottouesmyname8622
    @iwouldlikenottouesmyname8622 11 років тому

    Does C.E.R.N store their written documents on .jpegs?

  • @Ludix147
    @Ludix147 11 років тому

    Sorry, messed that up with veriatasium. :)

  • @klaxoncow
    @klaxoncow 11 років тому

    The "giga-" prefix comes from the Greek and, yes, it should really be pronounced "jigga-". It's a soft "g", not a hard "g".
    The mistake in that movie, in fact, is not the "jiggawatts".
    It's Doc Brown saying "one point twenty one" rather than "one point two one".
    A scientist would never say "point twenty one" - the two is in the tenths column, not the tens column. It's not "twenty", it's a fifth.
    But pronouncing "giga-" as "jigga-" is just correctly respecting its Greek origin.

  • @GGanon
    @GGanon 11 років тому +1

    that lady's a bobble head

  • @jpmthemonk
    @jpmthemonk 11 років тому

    SSD's have relatively low lifespans due to rewrite wearing though right? And instead of only a gradual decay, I believe they fail completely, with no warning. Someone correct me.

  • @Nathan-ji2nd
    @Nathan-ji2nd 11 років тому

    I think we may be talking closer to Minecraft at 8K.

  • @CountChocula
    @CountChocula 11 років тому

    How does the LHC produce so much data?

  • @824Icebox
    @824Icebox 8 років тому +10

    but will it run crysis?

  • @natpbs
    @natpbs 11 років тому

    Nice video, and nice accent. Accents are fun!

  • @jpmthemonk
    @jpmthemonk 11 років тому

    How, pray tell, is that different from what I said? I'm not being facetious. And they're not "apples and oranges" - they both perform equal tasks - data storage, manipulation, and processing. Data is quantifiable (or comparable) in both cases. You are thinking in the short term, comparing brains to desktops. May I remind you that computers have beat chess champions? But I'll repeat myself: raw computational power is nothing without software to leverage it.

  • @Mar184
    @Mar184 11 років тому

    Informative video, but nevertheless "the Grid" always connects to Tron first in my mind.

  • @TheMohawkNinja
    @TheMohawkNinja 11 років тому

    So, is the first sentence the answer, which is a yes (while maybe not RAID, it is in someway duplicated)?
    Also, yes theoretical physicists (amongst other sub-fields of physics) need a very high level of certainty. Brady did a video on it. It involves Sigma, whereby 1 Sigma is about 33% certain, and it goes exponentially up to 5 Sigma (if not higher) which is 99.99...% certain.

  • @james4airsoft
    @james4airsoft 11 років тому

    The Mars rover actually continued roving around and sending data from Mars far longer than it was supposed to last up there, by years even, what are you smoking?