DeepMind’s New AI Beats Billion Dollar Systems - For Free!
Вставка
- Опубліковано 14 лют 2024
- ❤️ Check out Lambda here and sign up for their GPU Cloud: lambdalabs.com/papers
📝 The paper "GraphCast: Learning skillful medium-range global weather forecasting" is available here:
arxiv.org/abs/2212.12794
Code + models: github.com/google-deepmind/gr...
Predictions out there in the wild:
charts.ecmwf.int/products/gra...
WMO:
www.flickr.com/photos/worldme...
www.flickr.com/photos/worldme...
📝 My latest paper on simulations that look almost like reality is available for free here:
rdcu.be/cWPfD
Or this is the orig. Nature Physics link with clickable citations:
www.nature.com/articles/s4156...
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Bret Brizzee, Gaston Ingaramo, Gordon Child, Jace O'Brien, John Le, Kyle Davis, Lukas Biewald, Martin, Michael Albrecht, Michael Tedder, Owen Skarpness, Richard Putra Iskandar, Richard Sundvall, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: / twominutepapers
Thumbnail background design: Felícia Zsolnai-Fehér - felicia.hu
Károly Zsolnai-Fehér's research works: cg.tuwien.ac.at/~zsolnai/
Twitter: / twominutepapers
#deepmind - Наука та технологія
I think its important to remember that a good deal of thst billion dollar industry will be involved in the collection of the data that is required for the ai to use. This will all still be needed.
More data more better
The NextGen Weather Program will use data from aircraft. Each aircraft will transmit on ADSB to ground information such as wind direction, velocity, humidity, temperature, vertical winds, air pressure, and other parameters. This is very low cost program, but with greater impact on ships, planes and us humans. All this data information is already accessible from the aircraft navigation systems.
Automatic weather stations need maintenance too. Near me is weather station but it did not got all data because no-one want to do that part time job for pennies (doing work with schedule mean you can not keep vacansy etc bas things)
And just think of the quality of the inputs,especially considering how the weather is such a bugbear for the powers that be....this should be viewed as terrifying rather than revolutionary
exactly, this paper is impressive enough in terms of performance (if we assume that the numbers are right).
It would have been better not to act like they were doing magic, forgetting that the data that was used comes from the international “Billion Dollar System”
One could argue the billion dollar system was necessary to build the training set needed to create the AI.
And one would be objectively corrects
And you would be correct
But once the AI learns to walk, it will be able to take over, and only need refinement thereon.
Not really, it needs lots of maintenance, supervision and new and better data
@@inoscopedjfk8207 just like kids and their parents. But the kid would never have existed without being raised by its parents
My Jaw gets significantly lower each time he says " ... it get's even better".
What a time to be Alive.
Ikr i was holding on tight to my papers. And good thing i did, i would have dropped them otherwise.
Could fit a basketball into your mouth after this video
🎵 my anaconda want some
What a time to be jobless
I think the huge elephant in the room in this video is that their model runs on *a single tpu* (see 2:20 ). That really blew my mind!
I don't think most of us in the comment section understand the gravity of this development. This is a revolution in AI technology. Working on something immensely complex like the planet's weather system which is dependent of tens if not hundreds of variables is mind numbingly difficult.
Moreover such application has a direct tangible impact on lives of people.
Even if the predictions are half good in this first generation paper.. Two more papers down the line, this tech might be able to give accurate predictions most of the times..
Awesome paper... 🤯
Well yes I do agree that accurate and efficient weather prediction is important. The paper is over a year old, so I don't see the crazy new AI part that much.
most of the people won't understand how the gov would use this kind of ttecnology to c.ontr.ol the worlsd
@@DWM864 Hah! Funny, Control what? "It will rain tomorrow, oh no! we lied, it's actually sunny! Muhahah!!"
You think *any* government has any control over the world at al? Weather forecasts are dozens of nations working together, it's not controlled by a single entity. Also they said this program is open source, meaning ANYONE can see **exactly** how it works and operate, does that sound very secret conspiracy to you? This is a channel for science not pseudo-politics.
another skynet dude. bruh@@DWM864
More than hundreds. More than thousands, even. Every weather station in the world is its own set of temperature, pressure, etc. variables, and they are recursive. Weather is time series dependent, so different readings over time from each station would need to be accounted for. Each parameter in the model represents a variable, so we're looking at a problem with hundreds of millions of variables.
In all humanity's history, technological revolutions of this magnitude have been rare.The printing press, steam engine, flight, space flight, computers and now this. And this is likely to be the biggest transformation. What a time to be alive indeed!
I was just thinking about how it was only a matter of time before AI was used to predict the weather.
Some form of AI has been used to predict the weather since it's beginning, this is just another step (a huge one) in it's development
@@roscatres Yes what I meant was modern AI, as in, stuff that runs on GPUs and uses machine learning.
@@roscatresI don’t think we can even call that old technology that used raw calculations and algorithms AI anymore 😅
It makes me concerned about chips with all the rapid progess in AI applications.
It was already a point geopolitical tension and demand for compute has ramped up beyond manufacturers expected projections by a significant amount.
@@memegazer well what's going to happen as well is we are going to develop new kinds of hardware which can do these AI operations thousands of times more efficiently. So not only is the ability to make more chips going to give someone more power, but also, figuring out any sorts of more efficient designs could really turn the tide.
I think it doesn't make sense to have this computed in your pocket if the input needed from all the weather stations needs to be transmitted over the internet anyway, right?
I think so too.
He probably only mentioned it to elucidate how power-efficient this calculation is.
It's a metaphor derp.
It is common for rural properties to have their own equipment, but depend on companies to analyze them. With this, it seems that one person can really do everything locally.
If this was made into an app and the sensors on phones were used for data collection it very well could be done. Perhaps even better than weather stations.
@@atlas_19 the sensors of the phone would have to be Seriously Different (Repeat Seriously Different) from what they are now to be used with this model.
This is awesome. I'd like to see an AI tackle predicting space weather and its effects on Earth's weather as well
Extremely important! Imagine if we could predict solar activity.
@@benisrood that would be.. significantly more difficult than predicting the weather
@@freerobux49magnitudes higher
@@freerobux49 we just don't have the necessary sensors yet. all we need is more data.
but not impossible @@freerobux49
I've been following this channel for the past 3 years. I only see cutting edge science.
Thank you, dr With difficult name :D
Carol's whole knife hair.
Carols Johnny Ferrer
Cara Jolay If I Hail
I hear Dr.Karo johna ifahir.
your enthusiasm is infectious! your channel is the one i trust most for grounded non hyperbolic AI news. thanks for holding on to all these papers!
"Right on the tick. Amazing. Absolutely amazing. Too bad the post office isn't as efficient as the weather service." - Dr. Emmet Brown, Back to the Future 2
That would greatly optimize the harvesting periods, since even minutes are very important for farmers. That's a great breakthrough! Hopefully it will get better in 2 years or so.
Solid work with this paper my man! Keep up the good work
This is so cool! 1km resolution when? Managing to beat the HRES in some bases is amazing
Thank you for bringing these videos to us. Truly, thank you.
I've just started to work with GraphCast in the past week or so, and I'm blown away by this video! What a time to be alive!
This is actually insane. One of the most incredible practical applications of AI in my opinion.
Genuinely unbelievable. So crazy. It would be interesting to probe the sensitivity of the models to the various inputs and learn about the physics of weather (or rather, learn what the model learnt about the physics of weather).
Could a similar network be put to work on long-term climate predictions?
I started off thinking "at least the previous model was necessary to get here". But then I realised the weather is always happening, so other than the sensors, none of the current system was required at all. Fundamentally those Billion dollar systems are now functionally worthless. And if we had never had anything other than Deep Mind and the old measurements, we would still have just skipped right here.
Utterly mind bending.
What a time to be alive.
no you still need another system to compare them and weather models are used to develope clima models, not sure if you can do this with a system like that. time will tell
@@0bloodshot0 You don't need another system to compare them to, because you can choose any historic weather state, input all of the known values, then compare the predictions to the actual readings at the predicted intervals.
The previous model absolutely was necessary to get here. You could theoretically argue that all you need is the data and not the models but that’s just not how it works in practice. The AI are almost always built off the back of previous modelling efforts, both for validation and more granular and wide ranging data assimilation.
@@calorus Much of the billion dollar systems is instrumentation. We'll still need those. It's the supercomputer that's not going to be needed anymore. Except for training up the next-gen weather prediction AI.
As far as using old data for comparison: that's all training data. Using training data for testing is no good.
@@Steamrick If you leave out the last 6 months of historical data from the training set, then that last 6 months is now good testing data, and that's always going to be the case because time is linear and keeps happening.
There's no difference between doing that and giving it all the data for training then waiting 6 months for new data. Data is data. It's good testing data regardless of it being current or past, it's merely whether or not its already in the training set or not.
What a time to be alive!!!
What a time to unalive
@@markus-ks9sf now clean your washing machine lint
This is your first video that's really got me thinking what a time to be alive!
it's all graphs baby
always has been....
Wolfram agrees with you...
I wish your video titles were less clickbaity, so that I’d know what it’s about
i already squeezed my paper to early, so i have to squeeze my desk now.
This is great because I own a rain gutter company and it will really help to have accurate weather forecasts. Now we just need to learn how to run this myself! Hopefully someone will make an easy to use UI for it
Sir, your enthusiasm is an inspiration. Thank you for existing.
I've been waiting for something like this, like surely a deeplearning network could be used to predict the weather better with how good they've gotten the past decade
Only after seeing this video have a realised just how much history I have witnessed in the making thanks to this channel. Thank you so much for giving my lazy ass a peek at so many things that now have already changed the world.
Slight problem of weather modification, but I guess that’ll help expose any deliberate changes like that.
exciting stuff
Just having more accurate rain forecast would improve my quality of life so much. Not even getting into extreme weather here, just knowing when to pack an umbrella and rain boots. I hope we get to enjoy the fruits of this labor soon.
i love the future what a great paper really impressed
I remember there was a paper also by Google about nowcasting I think way back in 2020 I think? and it was also awesome. Machine Learning is really good at this kind of problem - science that is truly complex, in the sense that it can't easily be understood just by decomposing it into small, simple pieces but really does have to be studied in it's whole detail.
I bet that won't stay at just 40 million parameters, somebody's gonna throw AI on the already existing weather predicting super computers and it's gonna be nuts.
Can’t wait to see your video on OpenAI’s Sora that was just announced today
This demonstrates the power of specialized models using current algorithms. Imagine a Mixture of Experts system made up of thousands of experts.
6:22 paired with SORA!! insane day
Where does it say that?
I'd be interested to see it tackle historical data and see how it measures up to something like the farmer's almanac(among others), which in the past, predicted some harsh winters that others missed.
so, those graphs can be pictured like beefed up cellular automata? ca also take the neighborhood into consideration, but those graphs make this much more flexible and extensible i guess?
No, Not in the least. Cellular automatas have absolutely nothing to do with this at all. No similarities whatsoever.
@anteshell not even like neurons in the brain?
@@chrismata1590 Not even what? What are you even asking? That neurons are not like cellular automata, or like neural AI?
If the first, not at all. Neurons in our brains are nothing like cellular automata. If the latter, there are quite a lot more similarities (as in more than zero), but that is not what OP asked about.
@@anteshell maybe my wording was off. sorry for the confusion. if you view every cell in a ca having relationships with their immediate neighbours you could picture this as a 'sort of' graph conceptually. maybe the analogy does not work here
@@anteshell why so butthurt? bashing on a curious individual seem fun to you?
I wonder if someone's already working on an aurora borealis/australis model!! This is potentially revolutionary, what a time to be alive!
I can imagine that a couple papers down the line it'll be used on your phone in real-time (
I think this video is one of the most impressive as of late, it's just non-stop jaw dropping facts
Weather prediction was one of those things that felt like a nobrainer to do with AI. Amazing results!
What a time to be alive!
I remember hearing that phrase talking about that "if tech is so advanced beyond our comprehension it's like magic"...This is so close to it! Imagine 100 years back hearing that we would be able to predict storms during a bright clear day.
Hopefully as this advances, it will become able to predict weather further and further into the future. 20- and 30-day forecasts would be incredibly useful in weather-dependent industries that require planning ahead, like agriculture. If farmers are able to reliably predict first and last frost dates and can see droughts or damaging storms far in advance, it could help improve crop yields/harvest dates/etc.
I'm not sure if it's physically possible. Weather is a chaotic system (just like a double pendulum, for instance), meaning even the slightest change in initial conditions can have a huge impact on the outcome. It means that the precision that you use to represent numbers and even small errors in measurements of model inputs may cause a large error in the end, especially with longer time horizons.
you could use this for ingame weather patterns.
sure but that seems widely overkill lol.
As a atmospheric model computer scientist myself, that's mind blowing!
This method also can work for engineering simulations: air, heat, fluid.
Had to read this paper myself for that AI course I aced
Honestly a great technology
Have graph neural networks like this been applied to video compression? With their accurate next step prediction it seems like a large pretrained model could describe lots of videos succinctly.
I know that there's a paper that uses diffusion models to compress videos.
It is called: 'Extreme Video Compression With Prediction Using Pre-trainded Diffusion Models' it compresses at rates as low as 0.02 bits per pixel.
Holy Mother of Papers!!! 😅 ❤
Great news! Was wondering if the voice for the video is AI generated. It feels that the delay between words is sometimes longer than usual.
This is a great and actually useful use of ai. Makes me happy for our future and what else could be done with ai.
I wonder what would be the performance of those predictions if we use the same architecture of NN like for "text to video" like SORA. Temporal coherence and including lots of "irrelevant" data is key here
Super cool! Deepmind FTW! 🥰
Tack!
Thank you.
Incredible what AI is able to accomplish. I can't wait to see what's next!
THIS... exactly this is the REAL PURPOSES of IA... on this matter and all matter to do best job on our world, to live well... not to TRASH BILLION'S and in the end THE BEST RESULTS... and not only in this specific case. The possibilities are enormous... HUGE. We NEED the help of IA... NEED.
I just waited for this!
Imagine the amount of computational power this will free up and make available for other tasks!
It would have been nice to get an idea how much training data they had, where they got that from and how much processing power went into that.
But in any case it's just amazing how all expertise goes out the window and gets easily replaced by neural nets.
Can it predict unusual weather events, like the formation of hurricanes, so far ahead of time? If it were to simulate a full decade of weather, I'm curious to know how many hurricanes it would predict would occur in that decade. Would it be a reasonable guess? If not, does it have a tendency to underestimate or to overestimate such likelihood?
Wonder if it can detect cloud seeding operations, 'abnormalities' in weather prediction..
So this is in practise linearization of compute cost for complex systems symulations right?
Not quite linearization, but for chaotic systems with inter-element interaction this is a very good start to a generalized solution. Perhaps a GNN architecture that is as versatile as the transformer will emerge.
@@greg7633 Can we up front say how will a problem of factorial compute time scaling change (the scaling) if 'formulated' (I dont know the nomenclature) as generative transformer?
So how long until we can use this to “solve” butterfly effect scenarios and push whether systems around to where it’s needed with minimal effort, such as dialing up or down wind farms?
The butterfly effect scenario is a component of chaos theory and can't be solved, only approximated with non-linearly decreasing accuracy with time. Fortunately dialing up and down wind farms are a resolution issue with a relatively short time horizon (not familiar with wind farms, but I'm guessing you don't need more than a day or two of warning?). With that in mind I think the current model can be used, or at the very most a retraining with finer-resolution or localized inference.
Good thinking! If you're in this industry let me know if I can help.
I have a feeling that the simulated world is way better tban here ❤️❤️❤️❤️❤️❤️❤️🌎👌
will we have weather forecast that is even remotely accurate for as far as the day after tomorrow now? that's game changing.
Stuff like this makes me believe you could make an AI that vastly outperforms fortunetellers on predicting life history from an image of a hand, if you were to get your hands on a labeled dataset representative of general populace. There are probably ethical reasons why we shouldn't, but it would be cool to see if there is a function that could be approximated.
Can this also be used to predict the economy and stock prices maybe?
This sounds like it could be very useful in the oil futures market
Presumably this little AI will improve by leaps and bounds very quickly as well. Having real world models to predict against daily should hopefully supercharge the future learning process?
Wonder what data it's missing that blocks better predictions. Or if it needs higher resolution data for higher quality predictions.
Also wonder if factors external to Earth (like the sun, or gravity from the moon) affect the model's accuracy.
I imagine the moon is very predictable, especially since it only takes a single satellite to track it's trajectory. It's either solar interference, limited understanding of our geomagentic field or dark matter. I've been meaning to look up a documentary on the higgs field theory.
There is just chaos at some level.
However, it's still possible to rather accurately do ensemble forecasts for those situations, like, how, it's not feasible to say where exactly the cloud will initiate, but when it does, it will cause that sort of wind pattern and this approximate rain pattern over the next hour(s).
And to an extend, weather radar for example does already track individual clouds to give rather precise sub-hour forecasts based on how the known clouds can interact with the environment.}
And the ensemble forecasts can be used effectively: you get essentially a set of like 100 timelines for the next hours/day/2-days, u[dated around every 5~60 minutes, with representative scenarios that could happen.
That could be for example that you can schedule one evening that either the next day's afternoon or the day after that has a nice streak of dry weather, so while you can't say which of the next 2 days you can do your outdoor BBQ, you can still go shopping as you know that one of those days will be fine. Then, some around 2-3 hours before the BBQ would start, you have accurate enough predictions to decide to prep the food then or tell the guests that it will be tomorrow.
Or for walking the dog, you can know that it will have a dry hour after an approximately 30 minute rain shower, sometime late afternoon, so you can schedule to wait for the rain to start before you wind down whatever you were doing so you'll be ready to get going once that rain stops.
Considering dolphin-phi with 2.7B parameters can run easily on my midrange phone... This weather prediction is incredibly efficient and maybe can run in small pi system
I went to a talk by Peter Battaglia a year or so ago where he talked about this system, glad to see it came into fruition!
What a time to be alive :)
Maybe weather reports will actually start to be accurate soon! I've been waiting for this.
It is indeed impressive given the size of the network. But it is a bit of question how much of this billion dollar system it is going to replace de-facto. The network of sensors and all the infrastructure and staff for measuring the current state still has to be at place and might represent the much larger portion of expenses for such forecasting.
I love this, research that exponentially empowers other research. Days where you throw more powerful and more expensive supercomputer on a problem to solve it are gone. Every day you can see comparatively cheap research, turn into a free software update. Update that in return improves performance way faster than anything that is possible with better hardware.
But how much data does it need? As input in order to get those outputs?
sounds like the weather in video games is getting quite the upgrade
My first thought is that, it seems the bottleneck is not on the algo but on the data collection. The problem of weather prediction apart from data acquisition requires mostly local information to solve (past data around the region) which is not a hard problem in the line of research of graph NN. But still the computational efficiency is still very amazing.
3:14 i have an idea. Instead of just simulating weather, can we input our technology and difference in growth so it can predict and possibly foretell of new technology or help us find them?
DeepMind is killing it with protein folding, geometry, materials stuff. We are going to probably achieve absolute knowledge before we achieve absolute understanding. Before AGI even gets here. Pretty insane how analog / neural computing just solves reality.
It's digital. Although analog would be far superior albeit at the cost of adaptability. The photonic analogue/digital hybrid processors that are in the R+D stage should be an absolute beast at running neural networks. Might be a decade away though
@@MichaelBarry-gz9xlI believe he means all these advances utilize a neural/analog paradigm of computation which operates on (virtually) continuous, not discrete, inputs, not that their stuff is not running in a digital environment.
@@gabe_ownerI'm not sure what he meant, I'm not sure what you mean either 😂 it's fully 100% digital, nothing continuous about it, if that's an abstraction, the abstraction exists only in the mind of the person, the models themselves are fully digital at every layer of abstraction. One day we will have analogue models, running on analogue chips (probably photonic) and that would be it's "natural home" so to speak, and that should be incredibly efficient and powerful.
if by solving reality you mean proving wrong theorems like the alphageometry does - then why one would need rnns in the first place?😂
@@MichaelBarry-gz9xl give it time and the neural networks will be continuous, and more accurate than any discrete approaches. Photonic isn't necessary (and is in fact inefficient as it converts from electrons to photons). It's all about algorithms. And while deterministic the stochastic ones will always win. Retrospectively it's deterministic, but while running, not so much.
And it gets even better - it can predict all of our strategies trying to counter this AI's global war on humanity several weeks ahead of time in just 0.0001 second
What you're forgetting is airplanes fly into hurricanes and tropical systems and also until these systems even rain systems any system coming one short on to California that has to be sampled until it sampled there's no real accurate information.
While this may be a decent tool to add it's not going to replace anything.
What it will exactly be good for is short to midterm forecast once the systems are on shore.
Without a sampling we can't know anything.
This cannot sample on its own it's going to be taking parameters that are taken by professionals and that input into the system.
Still good to see technology play a factor in something like weather because sometimes flash flooding and stuff like that would be good with a little bit more advanced notice than what is currently given.
But I highly doubt it's going to show where bending for snow is going to end up, that's just Mother Nature nothing can predict it.
It is truly a time to be alive!
Just think what this can do to Stock Market..! 🤯 💥
So weather forcasting is better than ever? I'm coriues to experience it since usually apple weather can't tell me precisly even the current weather
could you simulate realistic weather patterns in a game/other planets using this? or is the use case just for earth? I'm assuming it's just for earth but I'm wondering if you gave it a earth like planet with a different land and sea layout if it could make something realistic, not perfect, but good enough.
speaking of games, you can allready create realistic weather distribution by using randomness. This tech is about predictions pertaining to the real data, in a game that 'fake' real data would be more sensor data than all the other game assets combined. Of course you could take the same dynamic principles and make it simulate weather, but that is already very easy, in terms of just making it seem like real weather. But the realism of weather in a game is more about how it is rendered than how it moves globally.
Wow, I literally have a video on the same method they used in their GNN architecture (on this channel). Made it during university and it was an original idea. Could be a coincidence, but I do wonder if I influenced them at all in making this paper.
WOW... weather forecast has always been the Holy Grail of predictions. My question is if this would make weather forecast more reliable, or if it just makes the same old (unreliable) predictions faster?
Is there a way I can find out wind speed given a latitude and longitude, using the model?
Wow this actually has some pretty great implications! It seems like it could actually improve people’s lives, unlike the pain AI has created for artists.
I love the content, I always have! But an FYI these 2 minute papers are not quite 2 minutes anymore! hahaha but the papers tho..
I really want to know if (and how much) results improve if you go from a 40 million parameter model to a 4 billion parameter model and train that with an international organization's deeper pockets. I mean - 4 billion parameters is still very small in comparison to the average LLM and should be easy to run on a desktop GPU...
Where can I get the most accurate weather forecast?
Can this also be used as a climate model? For example if it also has co2 levels and similar, as input.
Maybe a video on how to use the models in a notebook?
We have tutorials for such here thr focus is on research
where?@@warrenarnold
It's amazing to see what we can achieve now. However this would never have been possible had these billion dollar stations not collected the amount of data for these predictions. So while it is amazing we still need to actively obtain the data about the weather to run these predictions on.