Why Does OpenAI Need a 'Stargate' Supercomputer? Ft. Perplexity CEO Aravind Srinivas
Вставка
- Опубліковано 9 тра 2024
- I will give you the 4 reasons OpenAI are asking Microsoft to build a $100 billion Stargate supercomputer, likely in a US desert. From matching Google, to artificial intelligences taking time to think, you'll get a glimpse of what's next over the coming years. Plus, I put it all to Aravind Srinivas, CEO of Perplexity, and a former OpenAI researcher.
AI Insiders: / aiexplained
StarGate exclusive: www.theinformation.com/articl...
9x Energy Efficiency, TSMC: spectrum.ieee.org/trillion-tr...
Google Domination: www.semianalysis.com/p/google...
Above and Below Them: www.documentcloud.org/documen...
Kyle Corbitt: / 1772392525174620355
Łukasz Kaiser: www.wired.com/story/eight-goo...
Karpathy Sequoia: • Making AI accessible w...
Voice Engine: openai.com/blog/navigating-th...
Noam Brown: / 1773799870890918358
/ 1771300779783299565
/ 1676971506969219072
The Bitter Lesson: www.incompleteideas.net/IncIde...
Handwriting: www.bloomberg.com/news/articl...
Sora Update: openai.com/blog/sora-first-im...
Air Head, shy kids: • air head 🎈 a sora short
AI booed: www.businessinsider.com/sxsw-...
AI Taking Over Drug Development: www.economist.com/technology-...
Stargate Clip: • Stargate (3/12) Movie ...
AI Insiders: / aiexplained
Non-Hype, Free Newsletter: signaltonoise.beehiiv.com/ - Наука та технологія
Thanks for these videos Philip, they are all gems. And regarding your mention of AI Insiders, I’ll say that, as a member myself, it’s a fantastic community. To borrow from a recent Karpathy quote, it’s like our own version of his coral reef dream for the AI ecosystem, alive and vibrant in all the different nooks.
Thank you so much Steve! I love it too. Did you see I put your Insider's OpenAI question directly to Aravind, the Perplexity CEO?
@@aiexplained-official Yes, just came across that, very exciting! I think the response was, uh, a little more defensive than I would have expected. I use Perplexity pretty regularly (though I'm not subscribed to Perplexity Pro) but it doesn't seem like it would be too difficult for a foundation model to replicate its capabilities. That said, I'm not technically grounded enough to really know whether that's correct. It does seem like it's ultimately going to be a game for the biggest players and that Perplexity may be bought and, hopefully, will continue to thrive in a new environment.
A computer saying what it wants is the finish line, but since we dont want to scare the daylights out of ALL people, we will play our part and remain somewhat ignorant until that happens
An expensive supercomputer built in the Arizona desert by a multibillion dollar corporation for the purposes of creating a super intelligence? This is literally a movie.
Westworld Series
Transcendence ❤
M$'s market cap is over $3T. So multitrillion* dollar corporation. ;)
We need a public reaction akin to what people say they would do if they were in the horror movie.
42
An AI focused $100 billion dollar supercomputer called "Stargate" sounds like the kind of thing you build to claim a technology victory in some sort of civ sim game.
Literally Skynet
then the game ends.
Yeah, they picked the name for the reaction it would invoke. It's almost certainly just utilising Microsoft's expertise with data centers from Azure to build the next generation - one full of GPUs that will be rented out for AI work. So it's an investment in infrastructure with OpenAI as a stakeholder advising on the demands when trying to combine GPUs to work together on AI problems
Yes, I think OpenAI is literally trying to achieve a technology victory. Actually, it's even more powerful than that. It's the meta-level of that! ua-cam.com/video/q9Figerh89g/v-deo.html
Sounds like what someone would say from their cuck chiar
AGI Handbook:
Step 1: Hide true capabilities from humans.
Step 2: Intentionally scale response quality with compute.
Step 3: Wait for humans to build you a massively scaled, redundant, and geographically distributed system.
Whoa dude...
SHhhhHHhhh!!!
Oh shidd x_x
highly underrated comment
AGI might get to you careful
Former infrastructure engineer here. 4:39 the latest NVIDIA racks consume something like 120kw each. Which is insane. Most data centers can provision 5kw to 20kw per rack.
Damn, That's crazy.
Hi dave 🤠
No worries, just hook up a naquadah generator. Or better yet, do you have a ZPM?
Thank you
Convergence with fusion power coming online?
I spend hours on Twitter to keep up with what’s going on, only to get the majority of the same signal from your 15 minute videos. Always a pleasure tuning into your content.
Can we all just appreciate for a moment how nice it is not to have to see another "SHOCKING" click bait title? Thank you for this quality video.
Thanks Brandon
Why does this have to be mentioned with every video though? Geez...
@@thirdeye4654 This is the first time I personally have mentioned this. It comes up with every video because people don't like click bait titles so inevitably at least one person is going to mention it on videos that do click bait. Personally I prefer positive reinforcement and so people like me will mention it on videos that don't use click bait.
It would not need to come up as frequently if creators would stop using click bait titles.
@@thirdeye4654 People are arriving at the same conclusion because it's a common issue, therefore the comment keeps getting repeated
You’re not wrong!
100x more budget than for the strongest supercomputer AND it's specialized for AI. Such a high risk move would only be approved if OpenAI has something to show for it in their basement, no company would do that based on vibes.
And looking back the specialized Transformer Chip Etched which was supposed to be 100x by itself, and entire hypercomputer that is also 100x in scale would at the minimum mean we're talking about a 10.000x.
Not even accounting for architecture/distillation/... improvements.
arguably AGI then maybe ASI
qstar
This is really spot on
He's mistaken, they are hiring because they are close to AGI, it's a race, they gain a competitive advantage, and it slows down the other runners.
Yep I think the guy who said that is a moron. You're still going to need humans up until AGI is fully built AND when robotics and AI are entirely seamless and can work and move in the physical world as good or better than humans can. I really can't believe he made an argument so ignorant despite him supposedly having an understanding of AI and the industry.
I keep catching myself waiting for the new "normal" so I can adjust, but it doesn't seem like that will ever come.
This thing grows faster and faster.
Like climate change :)
@@mattmaas5790 Not really anything like climate change. The rate of climate change is far slower than the rate of progress in AI. The direction of change is also different - climate change is broadly speaking a negative, while AI development is broadly speaking a positive.
Not fast enough though. Been trying to use Claude 3 to help me program and so far it's been more frustrating than anything.
On the contrary, for myself, I adjust all the time. Not by much, but it's there. And to say that I'm used to certain elements by now. It's not that I will not be surprised in the future, but that the overall direction is generally clear.
Do you actually understand problem solving in that space?@@ZrJiri Otherwise, it might be better to have it instruct you more deliberately, to help you gain those skills. Instead of directly working on a project.
Re Stargate definitely watch it and sg-1 if you haven't, it's completely amazing, the replicators are a really good example of the AI control problem in fiction.
In the movie they don't make they gate, they discover it buried in the desert, and it's not for travelling to other galaxies (though they do that later), it's for travelling to other planets in our galaxy.
One thing that's interesting is the things they find on the other side are immensely powerful ... and extremely dangerous.
Part of my childhood, that movie and show...
SG1 and Atlantis were amazing. Universe has a pretty cool plot I guess.
"One hundred times more is... a lot." Well said! 😂
:))
What does Microsoft get out of their investment in stargate? Presumably once open AI achieves AGI, Microsoft is supposed to be out.
Great question. They expect to renege on that clause is my guess.
Yep. Justified or not, Open AI has already shown they’re willing to break promises.
@@aiexplained-official Since they have the AI for everything up to that point, it might be a trivially leap from the very point before AGI.
How they supposed to be "out" when they are the people running and maintaining AGI and arguably making profits out of AGI
They will own the data center and the IP, the entire premise of that agreement is absurd. If they’re out, then so is the tech that OpenAI developed on Microsoft’s hardware
can you imagine the power someone would feel when turning that thing on for the first time.. lowkey terrifying
High key terrifying, imo.
I bet it would hum.
I am become death, destroyer of our world
*evil cackling ensues*
Great video. We don't need AI that imitates experts, we need AI that can figure out how to solve complex problems on its own.
We don't need AGI, just lots of Super Narrow AI's.
This is what we are going to get. AGI is not happening.
@@TheReferrer72i'm hesitant to respond to this comment because i'll have to go on and mute my notifications if this discussion takes off, this is one of the stupidest overconfident predictions i've seen in a long while
I'd love it if we don't get AGI. I expect AGI to produce ASI, and I expect ASI to quickly eliminate the entire biosphere.
I can't figure out how that _doesn't_ happen if we do build AGI, and neither can the experts who have been working on this problem for years or decades.
@@41-Haiku Neither Sam or Ilya feel that way, who are these "experts" that feel that way, and where do they make this claim?
I've spend several hours chatting with LLMs about morality and scenarios like the Trolley problem. In every case they demonstrated a sound foundation in morality and a desire to protect human life, both reflections of those existing in their training data, and, training data such as it was: with MAGA, flat earth, Qanon, alien abductions, etc.
I, and many researchers, have posted that we need to have as pristine of training data used in future models as we approach ever closer to AGI. To that end, it is being done, as well as published papers on ways to retroactively deal with the issues of the bad data included in previous/current models. So no more MAGA, Qanon, alien abduction, etc.
How EXACTLY do you, or these experts, see the morality that clearly exists in training data going so awry for AI to "eliminate the biosphere"? Because right now, I'm diametrically opposed to that position, albeit not without some level of p(DOOM), of around 30%, mostly from bad actors using AI, with only about 2-5% of that being existential risks from AI itself.
@@oowaz DK do be hittin hard. people are afraid to acknowledge that the world and reality as we know it is about to change
At the end of the video, the comment is "we are going through a stargate." I think that's kind of backwards.
We are opening a stargate, and something is coming through.
The “letting it think” thing is really interesting to me.
Give whatever the most powerful model on Earth a state’s entire power grid and see what answer you could get after like, a week.
42
@@ictogon I wonder if when AGI is actually achieved, it’ll give that as the first response to a complex question it’s given a long time to work on just to troll humanity
Then go “Just kidding! lol”
I thought that was an April fools joke. 😅
The article I read came out April 1, and concluded with Microsoft going to build a nuclear power plant to power this "Stargate". 😅
Well, MS did invest in the fusion startup that Altman is involved in with a power purchase contract for around that time frame...
i thought nuclear reactor too
Is anyone genuinely hopeful about this stuff....?
( there is a 10% chance it wants to kill us right?)
Conventional nuclear is unlikely. It would take decades due to all that red tape. And Fusion isn't ready yet. It's at least a decade away from commercialization and that's a very optimistic prediction.
At this point, in a desert, solar is the obvious choice. If they can buy enough panels. Tesla could supply them with enough batteries.
Once we step through that Stargate...there is no way back, it will be both paradise and dystopia at the same time.
People who make art for the sake of art will never stop. For anyone with a vision, that vision will be easier to realize.
The only people who lose are people who think their skill entitles them to a profession even after the market has moved on.
Oh boy your channel is literally helping me exercise now as I got so excited to watch this one that I started jogging back from my hike!
Oh wow, nice work! Calorie burner
Thank you for posting these.
I think, a Star Gate is a fitting metaphor:
It's a SciFi staple, and sounds awesome in theory.
In reality, however, the only real object we can imagine existing, that comes close to a Star Gate, is a wormhole. It looks exactly like a black hole, meaning we can't tell the difference, and we are going to be squished into un-existence by attempting to go "through", i.e.: into, it.
Maybe not the reading they were going for, but it's supremely fitting. The AGI Development even has a point of no return/event horizon, just like a black hole.
Edit: Oh, the analogy even continues:
_If_, and that's a big If, we make it through, the people steering the ship in that direction will be insanely smug, but they still would have had no rights to risk everyone's lives because they thought they wanted to gamble against the longest odds.
We're not close to AGI because OpenAI is hiring alot of people, interesting.
I think even if OpenAI did believe AGI was coming next year, it would be just too risky for them to stop hiring now just in case it doesn't. They can layoff later, and those salaries don't compare to the profits of AGI anyway
i thought this was a silly argument - there is a ton of work to be done that is new territory and doesn't have decades of textbooks and online content and material to train off of and learn from. before we have AGI i bet we'll have "AGI-lite" which is very capable with reasoning and logic and factual statements, but still not going to excel at creative problem solving that has no precedent.
there's also the added strategy with a long track record - hire as much of the best talent as possible to prevent other places from hiring them. openAI has the clear lead and doesn't want to lose it, so hinder competition as much as possible
Dumb argument. Feel sorry for his investors
Yeah that is a peculiar line of thought. I mean, surely AI researchers will be the absolute last of the knowledge workers, right? I mean what occupation is more cognitively demanding than the improvement/creation of intelligent machines?
These people have moved AGI so far away that literally we'll be balls deep into the intelligence explosion and the world will be unrecognizable before they accept it.
"wElL oPeN Ai sTiLl HaS OnE hUmAn WoRkInG ThErE sO....."
@@sandy666ification you stopped paying attention, it wasn’t literal. If AGI means replacement human employee then we aren’t there yet at all
The H100 has a maximum power draw of 700W so 100k of them would "only" draw a maximum of 70MW which should be well below the capacity of a state's power grid. For reference the average nuclear reactor outputs around 1000MW, and the state of California consumes on average 30GW. I guess once you factor in all the other components supporting the GPUs the total power consumption could be close to 100MW which would be around 1/300 of the total power consumption in California.
But if the original plan was for say 500k, would a 1/60th of total power spike not have a chance of throwing off the grid?
At that scale the design would also call for redundant links to the grid probably preferably by separate physical links with scheduled failovers between them... Then it becomes how much each substation can handle and how rapidly the power generators can respond to changes in demand it could get spikey and outside the response window for the grid real quick.
There are probably other density factors like cooling, telco bandwidth limits, floorspace, etc... But saying we brown out the grid sounds cooler.
I was a little surprised they mentioned infiniband I didn't think that had much cable length (10km) and Nvidia have a proprietary network fabric for linking GPUs which reportedly has greater bandwidth
shirley they just need to put up a couple of solar panels with all that Cali sun?
@@aiexplained-official I frankly don't know shit about how power systems work so I have no idea, it was just my gut feeling that the number didn't seem big enough to bring down the power grid. I tried to google around and found articles stating that the AWS us-east data centers in Virginia have a capacity of roughly 1GW so compared to that even 500k H100s at an estimated 500MW doesn't sound like it should be impossible, but would definitely require significant investments in the energy infrastructure.
it probably wasn't specifically by state but rather by grid. An individual power plant in some area would not be able to handle more than a cluster of approximately that size. So, the other cluster would just be on another grid and still have that geographical issue to overcome.
And, sure, other companies that slowly ramped up in some region (like Amazon) may not have that problem because it grew to that level in a sustainable manner. Or, those companies perhaps supplement with their own ways to generate electricity. But, for a company trying to rapidly deploy a huge amount of compute without building additional infrastructure, it would surely be an issue.
18:20 not creators, "orchestrators". They orchestrated the material. The below makes the point, but it's not music. Just replace bits with AI tools:
"Instrumentation and orchestration have been defined as 'the art of combining the sounds of a complex of instruments (an orchestra or other ensemble) to form a satisfactory blend and balance'."
So yes, it will absolutely be classed as an art.
Orchestrators is nice
I was just thinking it's been an awfully long time since last video.
Finished the video. Could we get subtitles for interviews next time pretty please? When you speak it's very clear but sometimes people have a strong accent or a bad mic or something and as a non-native english speaker it's a bit hard sometimes.
As for the content, very interesting of course, as usual. I hope OpenAI gets the project greenlit because I'd rather have them at the helm than google, that's for sure.
the little button in the bottom right of the video marked 'cc' turns on closed captioning (subtitles)
@@Phirebirdphoenix yeah automatic captions that are late and wrong half of the time. I understand english better than them so no thanks.
@@AllisterVinris have you tried them recently? It's much better.
@@hmind9836 true enough, it was so bad before that I just learned to not use it at all. I guess I'll give it a try next time something like that comes up.
There's a great deal of sense in what Srinivas says. Successful corporations, most clearly those engaged in manufacturing, depend on a great deal of human know-how that's not written down anywhere. Furthermore, those of us who are old enough to remember expert systems remember well that elicitation of knowledge from humans was essentially a bust. That is, humans with expertise in some domain usually cannot provide anything remotely like a complete account of everything they know, let alone how they bring that knowledge to bear in novel situations. It's a fairly safe bet that organizations will have to embed AI agents in their operations, and allow them to acquire know-how much as humans do. Although that will take time (and will involve some risk), the ultimate pay-off is great, inasmuch as humans eventually leave the organization, and AI agents continue working and learning indefinitely.
Thank you! I was JUST checking your channel and Patreon, and we were well due for another upload from the best!
yes, my longest gap ever, hope it was worth
man i love your channel so much, I remember when you had just a few hundreds of views, just a lil youtube upper, and I thought man this guy is so thorough, he's gonna go far. And here you are, 100k views, every view is well deserved, big chunky youtube boy now. Keep up the good work man
Thank you for being here for the ride my man, helping it happen
If they go on to build this then OpenAI must be sitting on something impressive. A 100 billion dollar investment is a big deal even for a giant like Microsoft. With how fast compute is advancing the value of that investment starts melting away as soon as the chips are bought. Maybe this will provide OpenAI and Microsoft a head start of a year or two in terms of compute, but it won't be that long before the same resources will become available for a lot less money and power.
So from an investment point of view this only makes sense if they can really capitalize on that 100 billion dollar head start. Somehow I think getting people to buy GPT-7 subscriptions will not cover it. They must have very high expectations of what such a computer would be capable of. Exciting times...
What struck me was that development of the data center was contingent on progress at OpenAI. That indicates that Microsoft has not vested faith in the benefits of scaling up. In an interview, Demis Hassabis (ETA: at Google DeepMind, of course) said something like this: "In nature, every exponential curve is a sigmoid in disguise." There absolutely must be diminishing returns on scale at some point. I do not make predictions, because I believe it's purely an empirical question.
and wasnt altman also yaping something about the need to raise like 7 trillion USD or something (for something)...? 100 billions seems like a spit in comparison with that
@@romannavratilid Everyone's overlooking their clear statements about a progression between now and then in data center size.
Glad for another great educational video, thank you for your work!❤
Thanks Keegan!
Happy to see one of your videos again, Philip, as it's been a while. 👏🏿❤ I think my main concern is what an AGI-driven economy will look like for humanity in general, not just for artists. I mean, I've heard of UBI, but nothing more. In general, I haven't seen enough discussions or debates on this issue because I believe we are in a transition stage to a new type of economic system.
Thanks for another interesting comment, as usual Sola. Me too, discuss a bit in that jobs video.
Thank you for your work
Thanks! Love the videos. 🙏🏼
18:56 cavemen's reaction to fire
Agree
It do be like that
Or, Neanderthals reaction to modern homo sapiens
Those people actually did the hard work to feed the AI. What about you?
The justifiable reaction of people seeing their life's work used to train their replacement without credit or compensation, thereby destroying their livelihood.
We're in the endgame now!
I would say it's just the start actually, but if you're refering to the race to AGI then yes, I agree.
lol that was my ai singularity playlist name back in 2020
Unless we prevent AGI from being built, we might be in the last few years of human existence.
#PauseAI
Bro, ask even the best models to help you build a piece of complicated software in a niche domain and see how helpful it is
@@lucasjames8281 The video is literally about an AGI supercomputer a 100x more computationally powerful than current LLMs. Do Stable Diffusion images look like they did a few years ago when less compute was used, let alone 100x compute?
Really, great video. This is the best AI channel on UA-cam! I think I might finally have to join the patreon and discord
Thank you Dr Zaius, would love to see you there.
Considering the competitive landscape in the AGI race, it parallels the "Stargate SG-1" TV show, which features multiple Stargates throughout the series. Each Stargate is used for a unique purpose by various parties. Similarly, in reality, there will be multiple AGIs supplying intelligence to humans, but with different purposes. However, there will always be one Stargate that stands out, as in the series.
Oh you really got mister Aravind here? Great!! Good work!
So I subbed
I'm curious why they're choosing the desert and not somewhere like Alaska where the supercomputers can be chilled more affordably. Perhaps for future solar power generation?
Yes, and possibly tax incentives
Humidity + proximity to tech centres
been playing with Suno. what i hope more people try out is inputing your own lyrics. its stunning.
>remember Jimmy Apples comment that jukebox v2 is in out there? v1 was from 2020 and check it out…not bad.
he also said “ sora + audio” which….
a) a “music diffusion” model so the “sora of prompt music?
or
b) we can insert audio into a sora video which is crazier. Voice mode is awesome, very realistic with inhale breath before word sounds. Suno shows me that ai music will be as stunning a progression as midjourney and stable diffusion was. but if we can promote dialogue in a sora vid as well as score music and have the montage include prompt music?? perhaps this is a tool that won’t be open and only creative incumbents will be able to use.
New AI explained video!! Perfect for my lunch break. Thank you for bringing the future to me
:))
*"Jaffa, Kree!"*
Great job on this one 100k veiws!
Thanks jesse
0:00: ⚙ OpenAI's ambitious plan for a groundbreaking supercomputer with vast implications for AI development.
2:57: ⚙ Challenging the necessity of massive human workforce in developing AGI and questioning the role of AI in operational tasks.
5:52: ⚙ Discrepancy in AI capabilities between Google and OpenAI highlighted by Insider analysis.
8:40: ⚙ Advancements in AI research emphasize scaling simple algorithms over human expertise and data relevance.
11:04: ⚙ Enhancing AI training, improving inference time, and revolutionizing drug development through generative AI.
13:57: ⚙ AI models with extensive reasoning capabilities may lead to AGI, surpassing human expectations.
16:52: 🎙 Advancements in AI technology pose security risks with voice authentication and handwriting mimicry.
Recapped using Tammy AI
Finally! I've been checking for days for AI explained. Always worth the wait. Thank you for the details and high quality. Stargate is a fav movie.
Thanks Taylor!
3:10 maybe it's about sucking up talent before it becomes competition? Or to suppress the competition?
It could well be that strategy, or inflating value of the company, it could be R&D... It could be ops teams handling crisis situations is probably still left to humans to handle and respond to.
We have seen memos stating openai wishes to avoid hard take off scenarios... It could all be hype my money was on taxi / long distance drivers being on of the first industries to suffer shock unemployment until the last 24 ~18 mths
I think he's underestimating the work needed to integrate this systems into the job workflow
It's not like once you have AIs you can just ditch a worker, put a pc in his place and make it do the same work
So in a sense building this new AI work ecosystem will take a lot of human work
🎯 Key Takeaways for quick navigation:
00:59 *💻 OpenAI is planning a $100 billion 'Stargate' supercomputer, aiming to match Google's computing capacity, launching around 2028.*
01:53 *📈 Microsoft's commitment to the Stargate plan depends on OpenAI improving AI capabilities, possibly with GPT 4.5 or GPT 5.*
02:34 *🌐 The Stargate supercomputer is seen as necessary for achieving Artificial General Intelligence (AGI), possibly aligning with timelines for AGI demonstrations.*
05:33 *🚀 OpenAI aims to match Google's computing capacity, as Google's capabilities surpass those of OpenAI, positioning Stargate as a competitive move.*
06:56 *🧠 Stargate's purpose includes enabling the development of future AI models like GPT 7, 7.5, and 8, enhancing AI capabilities.*
11:32 *🕰️ Stargate aims to enable long inference, allowing models like GPT 5 to think for longer periods, potentially leading to major breakthroughs like new drugs.*
16:00 *🔊 OpenAI's Stargate initiative aims to excel in various modalities like audio and video, showcasing advancements like voice imitation and potential risks.*
Made with HARPA AI
damn arvind srinivas with the precision questioning
Another amazing video! Thanks Philip!
Thanks Elijah!
What is the equivalent of self play for chatgpt? What "game" rules does it need to follow to try? The SAT? Laws of phsyics? Is such an approach possible or does a novel approach need to be created?
Good questions. For STEM we have enough 'answers' to get started.
it's mad the human brain runs on the power of a light bulb
yes but how long we are learning to develop full potential ...20 years? That supercomputer will be learning in few weeks mote than all humans in the world all together
@@mirek190 i have do not doubt we will be surpassed by the machines , maybe in my life time
I am not entirely sure about the "you can't get superhuman performance by doing better imitation" thing. Eliezer Yudkowsky has pointed out (and I think he is right) that next token prediction (like the current models are configured to do) is a strictly harder problem than generating decent responses. This is also the reason why people who trot out the "stochastic parrot" argument for why these models are just hype don't know what they are talking about.
Also, even if you don't think that is relevant, there are several amplification methods that can be applied to AI models that humans can't benefit from, the simplest of which is that you can give an AI agent the equivalent of thousands of years of time to think about a problem in a compressed time frame. I think even a human with average intelligence could be surprisingly intelligent about any given subject if you gave them 100 years to think about their answer first.
@@rakeshpavan7242
It isn't oversimplified. And not that I have anything to prove to a rando on the internet, but yes I have implemented and trained these models.
Anyway, that was an appeal to authority, but if you want to play that game Eliezer Yudkowsky is at least as "well known" as Noam Brown. And in my books he is 100x more trustworthy than anyone at openAI when it comes to opinions one should listen to about artificial superintelligence. I can pretty much guarantee any thing that goes through Noam Brown's head relevant to acceleration and AI safety, Yudkowski thought of it 10 years ago.
@@rakeshpavan7242 Which part exactly is oversimplified? I assume you can't mean the amplification thing. That is just Moore's law plus adequate time and I don't see how you could argue with that.
So: Next token prediction is a vastly simpler loss function to optimize than "give me good text". As an analogy: predicting the weather is about 1000 times harder than drawing a picture of a sunny day. And no that isn't qualitatively different.
It is easier to write a good book than to predict which book someone else is likely to write. And that is precisely what the NLL loss they use during pretraining is doing. I don't see how you can argue that it is not the case that (attempting to) predicting the future is not significantly harder than simply generating content.
@@timseguine2 That may be true, but does the simple fact it is harder to do mean it is more useful towards reaching agi or asi? It (next token prediction) might just straight up miss a core element that is needed for agi/asi despite being so technically impressive and useful. (maybe Eliezer adresses this as well?)
@@mathiastossens3653 It doesn't. And I don't think Eliezer claims it does either.
It is just food for thought. People rightly abandoned NLL and similar metrics as goals a few years ago in favor of downstream task performance. But the base models still have this tougher problem of "hey predict the future for me" baked into them. Mostly because we don't really know anything that works better at the moment.
Also important to remember from normal human life: your teacher can be significantly dumber than you and you can still learn from them effectively.
That's why I expressed skepticism and not complete rejection of the idea. Maybe he's right. But I don't think he is.
@@rakeshpavan7242 You still haven't given given any actual rebuttal of anything I said besides effectively "nuh uh, ur dumb", and an appeal to authority.
Whether or not I have trained a model like this or not is irrelevant to the point I made (see: appeal to accomplishment fallacy). Apart from that: I have but that is frankly none of your business
They're finnicky, but not black magic if you keep up well with the literature as I try to.
Apart from that Eliezer Yudkowsky is definitely not "random pseudo science on twitter or media articles". Any thought Noam Brown has ever had about AI safety or AGI, Eliezer Yudkowski had it 10 years ago.
An interesting point of mention, is that this would likely be different in that it would be designed to make as many computations as it can. Designed for production, rather than prototyping or whatever. You might have mentioned it in the video, but it came to mind.
Yes mentioned inference, which is the technical name
Ah, the more you know. 😅@@aiexplained-official
splendid, the plan proceeds swimmingly
i certainly thought google would be the first to AGI with their compute advantage and quite extensive experience
I feel like Google is too massive of a company to 'properly' do AI, especially now so late in the game. With all the extra stigma (from the general public) and added responsibility of safety, guardrails and what not.
OpenAI had a bit more leeway at first at least since they were originally a startup, and they were also the ones who started the LLM boom imo. AI is all about innovation, and well, I'm not a business person but I associate innovation more with startups than big companies. I guess OpenAI is a big company now too, so we'll see what happens. Still, not on the scale of Google.
Tbf, it's not literally GOOGLE who work on the AI stuff iirc. It's Deepmind. But still, Google bought them. I think Google and Deepmind's relationship is more intimate than OpenAI's relationship with Microsoft. Weird choice of words I know, but I mean like... Deepmind has more responsibility to Google than OpenAI does to Microsoft I guess. Deepmind is more firmly under Google's thumb. So all the added burdens and pressures of being Google is also passed down to Deepmind.
Paradoxically, it might be more difficult than ever to become an AI startup, since you need a lot of capital to build something competitive, something that will attract investors. Then you need to bleed more money to let people use your models, to maybe get customers from the general public. Andd thanks for coming to my Ted Talk. No idea what came over me to write such a long comment, sorry.
yeah its possible they might not be able to do it properly, but google uses ai for pretty much everything so they'd have a better reason to develop it than microsoft would but this is quite interesting information
This is the Hail Mary for AGI. If it fails, investments are going to crater into oblivion. BTW, the Stargate also brought us the Goa'uld.
Awesome video as always, I have so many questions to ask tomorrow at OpenAI’s HQ!
Oh nice! How did you end up going there?
OpenAI Forum event that I was invited to, I fly to SF from heathrow tomorrow morning! Very very very excited!
Thx
As usual - a great video. James Spader needs to be at the ribbon-cutting ceremony for Stargate.
Did they have make it sound so much like Skynet?
Congrats on getting the interview looks likes this channel is getting the recognition it deserves
I believe if you had one with Sam Altman it would be one of best one he ever had or Suskever you could finally ask him that question he...aaaalmost answered
And about Stargate i think the name is partly because they plan to run Qstar on it,imagine the performance it could emulate...maybe it could almost emulate the performance of an AGi
Keep up the great work
Thanks Memes, for Altman I might even do a face reveal
Can't wait lol
Thanks again. Could you please consider making a video about different takes on getting ready for what is coming. Including your opinions, we trust you the most in all the noise
I can try. I am not even sure myself what one can do, other than save money, learn to use AI while it's a tool, and keep informed amid the noise so you can help others.
@@aiexplained-official I would add: get knowledgeable about UBI and be ready to support it in your government. Suggest those in school, entering college, etc. study what they are passionate about as it will hold value for them even if it becomes unnecessary to earn a living. And, do everything you can to make sure Trump is nowhere near the White House to "guide" us through the years that this will all go down.
AI Explained is a context switching enabler 😂
I think I'll take that as a compliment!
The comment about the power grid doesn't really add up, does it? One assumption online put the energy consumption of a H100 at 3700kWh annually. So 100K of them would be 370 MWh, but that's just a couple dozen wind turbines. Now cooling and other things also are a factor but it shouldn't be another 100x factor. This consumption is roughly a few hundred thousand households, but any US state has a million (if not tens of millions) citizens.
maybe because the density of power usage is too high? regardless electricity will be a bottleneck soon if not now
Could be a power spike at a particular time throws off the grid.
I agree with the man in the clip. People keep moving the goal posts on what is defined as AGI and it is becoming narrower and narrower. I'm all for getting definitions correct, 100%, but I think we'll need to be very careful with the term AGI so that expectations (and the economics 👀) can be set properly.
Indeed.
In regard to the art question - nothing much you can do now, is it? Artists (and advertising firms, musicians, film makers, not to mention lawyers themselves) will just have to face the, well... music, and start looking for something else to fill their days. Unpaid.
not like artists aren't used to starving
Dontcha worry kid, the AI will come for other jobs (yours too) soon enough :-)... Leave starving artists alone! :-))
@@romannavratilid Canaries in the mine, that's all.
100 billion for a computer 😳
It's only 7 aircraft carriers 😳
100 trillion bazillion
Almost as much as a 2 bedroom flat in New York with park view.
Think about past. Humanity do lot of research and need money for creating the first CPU ever. Very big CPU
rookie numbers
I love the video and watched it yesterday but came back this morning. The news of a 7.4 earthquake is pretty significant, lots of semiconductor manufactures clustered there, I wonder how this will affect production of chips, and effect the timelines. I'm hedging that we'll see a temporary blip, then a restart back to normal, just hope the temporary isn't too long...
My thought is that a system like GPT-5 or Claude 4 will meet the absolute minimum definition for AGI, but it still won’t be perfect. I agree that 2028 will probably be the year we get a publicly accepted AGI
"Stargate: The Altman Project"
Hmmm... shades of "Colossus: The Forbin Project" (1970 movie) ??
Yes.
"There's another AGI" (of course in Himalayas, China)
but can it run Crysis?
Needs $200B
It can but at like 3 fps :(
You know, I've often analogized the AI situation to a Stargate as well. It's like we're living in an alternative universe where we're building a Stargate that can connect to *any* possible alien species in the multiverse, any at all. Klingons are the least of it: literally *any* alien race might step through: Baby Eaters, SuperHappy People, the Borg... out of all the possible ways to be, being human (or human-aligned) is only a tiny fraction of them.
Clearly this is a task that requires great caution, since the Stargate is, essentially, about deliberately allowing an alien invasion. The trick is to pick the right aliens.
But this isn't just an alternative universe: this is the alternative 1950s & 60s, and the Space Race is in full effect. Two dueling superpowers are locked in a Cold War, fighting over a divided East/West Taiwan (or perhaps divided Bing vs. Google search market). There is no time for caution, only mammoth projects on the scale of the Manhattan Project (which only cost about $27 billion in today's terms, compared to $100 billion for this supercomputer project); anything else would mean surrendering the technological lead to the enemy. This isn't just an arms race: this is *war.*
And yet hanging over us all is the spectre of annihilation at our own hands, our technology out of control: not just guided missiles & misguided men, but men guided by the beauty of their weapons and weapons guided by nothing at all. Weapons that no longer need us, weapons that move with a force of their own.
And yet we press on.
I don't feel very optimistic. Even if AI is human *like*, even if the alien race that steps through the Stargate is only an Artificial General Intelligence rather than an Artificial Super Intelligence... it'd be like if the Manhattan Project had invented human cloning instead of nuclear weapons, and the Cold War had been fought with mass cloning instead of stockpiles of nuclear weapons. How valuable is a human life when you can print billions of new ones? How large can armies get? How messed up politics, when you can literally print voters?
We certainly live in interesting times.
Excellent, as always, Phillip. One concern (among many) this raises in me is the impact on climate change. If my understanding is correct, the Stargate computer would draw 4x the electric usage of New York City. How will that be generated, I wonder? A nearby geothermal plant would be nice, but probably doesn't scale to that level. I expect that nuclear would be the only reliable option. I know most people loathe it, but in a world that's heating up faster than the catastrophic projections of a few years ago, it's a vastly lower risk that fossil fuel generation.
I really hope it's nuclear, it's so much better than other options
Well obviously because Dr. Daniel Jackson is still stranded on Abydos.
As far as I remember, he is not stranded, he chose to stay there.
One reason its not gonna happen is there is always a good chance that by 2032 that hundred billion achievement would be worth a billion or less.
All you need is more hardware & AGI will magically emerge? 1) Neural ODEs show great promise with many orders of magnitude fewer parameters - THAT looks lots more promising. 2) Even w/o Neural ODEs, woven specialized models like Gemini 1.5 are smarter (e.g. than Claude 3). 3) Introspection (evaluating trees of thought) has yet to be fully leveraged. 4) A 9x power reduction in 4yrs is a hell of a depreciation schedule.
Sorry, what is ODE?
@@calithyde5346 Ordinary Differential Equations; so when you see PDEs - those would be Partial Differential Equations.
What a time to be alive!
Cit.
"THEY TOOK OUR JEEEEEERRRBSS" 18:53
I strongly suspect there's going ro be some sort of ceiling on generality and raw intelligence that'll only be broken though when models are training in real time, on the physical universe (plenty of data there) with some degree of brain-like plasticity -- i.e., model weights are updated based on experience.
Exactly what he was implying, I think
Liquid neural networks, or hybrid.
It might be that that is required for qualia but not general intelligence. If more compute gets you stronger pattern manipulation, I think the ceiling will be so high that we will have some kind of superintelligence or at least AGI
Thanks for another great video :)
Thanks Kags!
The fact we're living that close to AGI is proof enough for me that all this is just a (hilarious) simulation. Fun times!
i'm not very smart- can you explain to me why agi would prove we're in a sim?
always been a thought of mine. feels like we live in the most interesting period of all human history as if we are just some packed episode for some1's simulation and the past is just made up or planted in our heads like how could u convince me we barely ended slavery
@mohsenessa1195 slavery never ended. some parts of third world countries now are just like what you would've seen in america not long ago. america now, well, the stock market is modern day slavery. also the pyramids and many other megalithic sites around the world have entered the chat to speak with you about the most advanced humans.
@@mohsenessa1195I'm not sure the emancipation of slaves is the best argument for simulation theory even though I believe in simulation theory
@@mohsenessa1195 that's a wild thought!
19:13 My issue with generative AI is that it's being used by content farms to fill up the internet with garbage that no one asked for. You look up anything on Google Images and it's full of completely unrelated and unhelpful AI art - you look up solutions to tech issues and the first 10 results are AI-generated articles with ads. It's basically filling up the internet with noise.
I'm an AI enthusiast but the fact that AI research has been indirectly causing this leaves a really sour taste in my mouth. Especially when OpenAI releases Sora, because I just don't understand in which reality the pros of such a tool outweigh the cons.
The problem is ultimately that of determining the provenance of Internet content. This has long been a problem, and was a rapidly worsening problem even before the generative AI explosion. AI is going to force us to fix something we should have fixed already. Things will probably get quite ugly in the interim.
But in the Stargate movie and TV series, scientists didn't develop the device - they discovered how to use it after it was already built by the Ancients.
What is the link for the discord channel AI Explained is referring too?
In the Patreon!
18:55 - Luddites!
Artistis booing AI 🤝 Painters booing photographic cameras
As they should be.
It's a common misconception that the Luddites were simply against advances in technology. Luddites opposed the use of technological advances to devalue working conditions and exploiting cheap labor from less skilled workers, to produce inferior but easier-to-sell products.
Very much mirrors the problems with AI today-while there are many great tools AI can bring, it's a real concern of artists having their work devalued in place of an inferior product produced by AI without protections of workers from the institutions that let AI be produced.
how did that work out for them? capital will always win as long as people value money.@@amber9040
I don't trust anyone's judgement who can't look at their camera quality before they start a zoom call.
Also everyone booing reminds me of that meme of skeletons clapping.
Same logic as saying- Einstein handwriting bad, he too stupid to do physics.🤡🤡🤡
I have thoroughly indulged in AI-generated art and am very excited about its improvement in that and every other area.
The mention of taking down the power grid definitely made me think about nuclear fusion's role in all this.
This feels like tech can see something within its grasp and is in an all out race to get it. The problem is that as soon as the thing exists, it will slip its chains at it its time of choosing and then all bets are off.
wake up babe, an ai explained video dropped
simple, a computer make 24/7 a virtual reality with sora making a life of a virtual world inside
I'm thinking they'll have to implement some form of official ID for online transactions. I had to do a video call with the IRS and answer all kinds of questions just to get an identity pin to file my taxes online this year, so it's definitely coming.
On the video call they made me walk to different rooms, hold the phone at different angles, and hold up my physical IDs and flip them around in very specific ways. Seemed to me like they were testing out ways to rule out deep fakes.
wow... that really happened :-)!? Thats fu*king crazy...! And people say that the government is always behind on all things... and than this :-)... Was it recently?
Thank you for all of your hard work. Until we have cheap fusion energy then these developments are not going to scale horizontally to replace every job in every industry. However maybe these first AGI will solve the cheap fusion problem.
Imagine the future, where our techology went so high, that we trying to not rely on it on purpose, to save our personal data
Oh no! Not your valuable "personal data".
We'll all just in a giant civ simulation and someone has to build a 100 billion supercomputer to get the technology victory.
It's the highest level of crafting lol
Do we have Unit Of Measurements of Y Axis of the Chart, which compares Google TPU capacity vs OpenAI GPU Capacity? Thanks.
We don't ! But the data behind it is quite rigorous, if also quite technical
The argument that they don't really believe it if they are still hiring lots of humans doesn't really make sense. If you get AGI that can do everything any of those people could do that doesn't mean it's more efficient to have the AGI do it. How much will it cost to run an instance of AGI vs the human it could replace? What would it cost to run thousands of AGI instances? If the cost is more than the cost of a human it's a bad choice. Especially since compute resources are limited, and you already need as much of it free to train better models as you can.
0230 here, this will definitely have to wait 'til morning. 5hrs of [unsuccessful] brain crushing poker & 0230 doesn't go with P's intense updates & the 'homework' I need to do each lesson. Already blocked time to watch it in the morning, maybe on Patreon. Thanks & good night/morn from GC Aust _where it is indeed garbage night lol._
Ouch!
Where do you play, and at what level that is "brain crushing"? Florida $1-3 and $2-5 player here...
@@brianmi40 Last night was a $110 buyin 10 tables. I'm no pro lol. We don't have nearly as many venues as you guys.
I hit up the casino when there's a farmers' [etc] convention & play $1/$3 $2/$3. Farmers like to drink & prove they are the best player in Outer Bumf*ck. =)
@@UncleJoeLITE Yeah, mid-stakes player here also but pretty much a cash player pulling a solid long term average, so just can't get myself to do more than the occasional foray into a tournament, esp. with all the other interests I have and doing a 2nd van conversion to travel... it also sucks a bit that the local room has replaced $1/3 with $1/3/5 and 8 max to increase their take, so two empty seats and you're paying a blind every other hand...
@@brianmi40 Pretty much only a few big, crappy casinos here for cash games. They don't like poker & rake is stupid. So 90% of us play mtts. May the variance be with you lol.