Chapters (Powered by ChapterMe) - 0:00 Coming Up 0:54 What models get unlocked with the biggest venture round ever? 5:35 Some discoveries take a long time to actually be felt by regular people 9:53 Distillation may be how most of us benefit 14:26 o1 making previously impossible things possible 21:17 The new Googles 23:47 o1 makes the GPU needs even bigger 25:44 Voice apps are fast growing 27:05 Incumbents aren’t taking these innovations seriously 31:52 Ten trillion parameters 33:15 Outro
One order of magnitude higher is referring to compute, not parameter size. The current frontier model (4o/pro/sonnet) is all much smaller than their previous sota (4/ultra/opus). GPT4-1.8T MoE might continue to be the largest model people train for a while.
Winning AI company capturing all the value is speculative, therefore an answer can also be speculative that the more any AI becomes powerful, the more complex society becomes, hence the value of being human to cope with new contexts to feed the "hungry AI" proportionately increases and hopefully it's fun.
Even if AI doesn't improve from here companies like Microsoft, palantir etc. are building out the AI eco systems that you will start seeing big companies using for all their backend systems. The average Joe will see the changes pretty fast I would think.
Fact Check: Terence Tao's parents are first-generation immigrants from Hong Kong to Australia. Which makes him Australian-American after he naturalised for duo citizenship.
Feel like 01 would make more sense for Agents vs chat Interfaces. GPT-4 is good enough for chat-interfaces. The next step is Agents, 01 with chain of thought is perfect for building agents that can control operating systems to achieve outcomes.
How do you solve quantum coherence multibody physics problems. I think they said a practically useful quantum computer has 200,000 to 1,000,000 qubits. They will probably need AI's help to solve such hard problems.
Yes, but how you interface with that ai will be important. So its important to build a platform or network or processing company now. People will need platforms & how they interface will be different in the future. Device startups realize this, as Steve Jobs said, the best UI is no UI .
@ikleveland Temporarily. Soon the AI will build the required interface on the fly, and will manage all networking tasks. The era of software is coming to an end.
I love the runtime paradigm of o1 but I sure do struggle to find a use case where its better than a solid prompt template and 4o, quite the opposite infact o1 gives me worse results most of the time
Yeah this whole thing was odd. We didn’t have o1 then maybe, but we do have it now and it’s not even as good as claude. Certainly not the leap they seem to claim it is.
Compute is indeed the king. Obviously NVIDIA had a head start but I think there's so much room for decentralized compute that's not being used. AI's future will depend on decentralization imho
this is unfortunately nearly impossible. weight, activation, and gradient exchanges are prohibitively expensive over ethernet (both bandwidth and latency). despite looking compute intense, language models have a even larger bandwidth than compute problem
I think you would have to build some pretty novel tech where you're really trying to "chunk" the stuff out so each decentralized comp would be handling almost like a "vertical slice" of the model and just sending the results back at the end, but I would suspect that may not be possible as it likely needs data from the other nodes to properly train each stage. It's probably possible to some degree, but I would suspect it's not advantageous enough to compete with the current solutions. The only way to find out is to try it I guess. Maybe there's something there. 🤷♂️
Btw, a little thing. I think you guys may (or not) want to hide your feet. They sometimes dangling take a little of the attention of the content... And thanks for the content.
Open AI 4.0+ cannot work effectively at the same time successfully in multiple windows. It cannot retain a topic if you come back later, it's like talking to a new person. I'm ADHD this feels like my failure when any request cannot be met. It frequently cuts me off even though I pay for plus to come back in two hours that's some bs. I'm creating gold, shot down by you used too much come back in two hours 🤣lame
I really get bugged by the name OpenAI now when we know its so closed that they dont even disclose any model related information in their technical reports.
I feel bad for you. It seems like such a pointless thing to obsess over. It takes money to do this stuff, and unless you are ready to fund their requirements, you should probably just get over it and focus on something that actually matters. One of their goals was to provide AI to everyone, and considering anyone with an Internet connection and smartphone or computer can use it for free, they are following through. But they said!!! Blah blah blah.... Move on and be thankful they put this technology out into the world so that it's not being hoarded by the elite.
problem is that while compute poring over data can progressively build more and better the 'routine' apps and systems we know and use today you will be stuck with the 60pt system with no inhouse ability to reach 61points and you have to wait for AI to be able to generate code that can get to 61 and nobody of even average talent would care to, understand or work on that pile of code that was conveniently generated in minutes, even bother to push it to 61 - and that is a latent threat to the system development ecosystem
Can you have a ten trillion parameter model? Can you extract that many parameters from the dataset/language? I honestly don't know, but at some point you end up with a larger portion of parameters that have very few (if any?) instances, no? (ie most columns are all NA).
You’re massively underestimating the problems with AI’s inaccuracy. You act like scaling will fix all problems when it makes hallucinations worse and you’re just pouring money into more transformers instead of fixing the problems
Hallucinations are and will continue to be a problem, but if models continue to get smarter and cheaper, you can use a series of models to fact check the output. Just like how when I finish a paper, I have several people proof-read it. Probably closer to peer review because you can have models check the logic, check the facts and sources, etc. Hallucinations do not seem like an intractable problem - just a hard one.
I'm convinced at this point that these comments come from people that don't know how to use AI. They ask murky, unclear questions without realizing it.
They require the ability to hallucinate just as the human brain needs the ability to form hypotheses by free association. So most likely the result will be multiple models for each step in the chain - free association > hypothesis formation -> testing by falsification.
@ says random bot on the internet. Do your research, everything points to scaling as a logarithmic curve, particularly for CNN and Large language model networks.
what's the use of your AI, when you accept Arab funding, add policies that hide the any islamic things from GPT, AI. Literally, GPT dont talk directly but give excuses like Mullah of ISLAM to legtitimise the atrocities by founder
Chapters (Powered by ChapterMe) -
0:00 Coming Up
0:54 What models get unlocked with the biggest venture round ever?
5:35 Some discoveries take a long time to actually be felt by regular people
9:53 Distillation may be how most of us benefit
14:26 o1 making previously impossible things possible
21:17 The new Googles
23:47 o1 makes the GPU needs even bigger
25:44 Voice apps are fast growing
27:05 Incumbents aren’t taking these innovations seriously
31:52 Ten trillion parameters
33:15 Outro
Lol did sam altman tell yall to avoid talking about claude and comouter use update
are you drunk? they said Claude has the fastest adoption in YC from 5 to 25% in few month
One order of magnitude higher is referring to compute, not parameter size. The current frontier model (4o/pro/sonnet) is all much smaller than their previous sota (4/ultra/opus).
GPT4-1.8T MoE might continue to be the largest model people train for a while.
Winning AI company capturing all the value is speculative, therefore an answer can also be speculative that the more any AI becomes powerful, the more complex society becomes, hence the value of being human to cope with new contexts to feed the "hungry AI" proportionately increases and hopefully it's fun.
Even if AI doesn't improve from here companies like Microsoft, palantir etc. are building out the AI eco systems that you will start seeing big companies using for all their backend systems. The average Joe will see the changes pretty fast I would think.
Fact Check: Terence Tao's parents are first-generation immigrants from Hong Kong to Australia. Which makes him Australian-American after he naturalised for duo citizenship.
Thanks for the inspiring sharing! Let's leverage the power of AI to improve human lives and building a more sustainable future!
but can it code crysis?
The girl misrepresented Fourier
Feel like 01 would make more sense for Agents vs chat Interfaces. GPT-4 is good enough for chat-interfaces. The next step is Agents, 01 with chain of thought is perfect for building agents that can control operating systems to achieve outcomes.
What do you think about the speed? Would that be the next barrier to overcome?
@@Gpt6master the next Google, will be a company that scans all the new agents and combines the best of the best.
This has me thinking when we’ll be able to run these large models on a usb drive size computer ….
Good thinking but not likely until “AI” helps human invent the sub atomic circuits to replace today’s semiconductor technology.
A great Notification
How do you solve quantum coherence multibody physics problems. I think they said a practically useful quantum computer has 200,000 to 1,000,000 qubits. They will probably need AI's help to solve such hard problems.
It's quite obvious that in a very few years there will be no software industry. Your AI will be your only application.
The only people who say this build neither software nor AI.
Yes, but how you interface with that ai will be important. So its important to build a platform or network or processing company now. People will need platforms & how they interface will be different in the future. Device startups realize this, as Steve Jobs said, the best UI is no UI .
@ikleveland Temporarily. Soon the AI will build the required interface on the fly, and will manage all networking tasks. The era of software is coming to an end.
@@kangaroomax8198 but I do, my friend. That's why I KNOW.
It's quite obvious that you are not in the software industry.
I love the runtime paradigm of o1 but I sure do struggle to find a use case where its better than a solid prompt template and 4o, quite the opposite infact o1 gives me worse results most of the time
I do like it better for coding tougher problems and debugging problems.
It can't do 90% of knowledge work today. It cant even do basic second year accounting journals right.
Skill issue
It can, you just don’t know how
Yeah this whole thing was odd. We didn’t have o1 then maybe, but we do have it now and it’s not even as good as claude. Certainly not the leap they seem to claim it is.
@@Brain4Brain What do you mean?
@@AaronBlox-h2t RAG
What does it mean for our healthcare system? Perhaps an AI doctor assistant to every human one?
0:35 I heard that as "then Windows will just be whoever builds the best...".
_Well..._
Compute is indeed the king. Obviously NVIDIA had a head start but I think there's so much room for decentralized compute that's not being used. AI's future will depend on decentralization imho
this is unfortunately nearly impossible. weight, activation, and gradient exchanges are prohibitively expensive over ethernet (both bandwidth and latency). despite looking compute intense, language models have a even larger bandwidth than compute problem
@ that’s interesting. Are there any papers on this problem? Or any companies that are trying to solve it?
I think you would have to build some pretty novel tech where you're really trying to "chunk" the stuff out so each decentralized comp would be handling almost like a "vertical slice" of the model and just sending the results back at the end, but I would suspect that may not be possible as it likely needs data from the other nodes to properly train each stage. It's probably possible to some degree, but I would suspect it's not advantageous enough to compete with the current solutions. The only way to find out is to try it I guess. Maybe there's something there. 🤷♂️
Ant Financial's fundraise was larger?
Btw, a little thing. I think you guys may (or not) want to hide your feet. They sometimes dangling take a little of the attention of the content... And thanks for the content.
Take your eyes off them feet and focus on the content. I don’t even know they were there. I would have never seen them, focus!!!
Hahahahahaha!!!
@@Enedee007 I love to know the brand of their shoes, don't you?
Sure, I do! 😝
@@Enedee007no I completely agree with him, now he mentions it, the feet are incredibly distracting.
Open AI 4.0+ cannot work effectively at the same time successfully in multiple windows. It cannot retain a topic if you come back later, it's like talking to a new person. I'm ADHD this feels like my failure when any request cannot be met. It frequently cuts me off even though I pay for plus to come back in two hours that's some bs. I'm creating gold, shot down by you used too much come back in two hours 🤣lame
I really get bugged by the name OpenAI now when we know its so closed that they dont even disclose any model related information in their technical reports.
I feel bad for you. It seems like such a pointless thing to obsess over.
It takes money to do this stuff, and unless you are ready to fund their requirements, you should probably just get over it and focus on something that actually matters.
One of their goals was to provide AI to everyone, and considering anyone with an Internet connection and smartphone or computer can use it for free, they are following through.
But they said!!! Blah blah blah....
Move on and be thankful they put this technology out into the world so that it's not being hoarded by the elite.
problem is that
while compute poring over data can progressively build more and better the 'routine' apps and systems we know and use today
you will be stuck with the 60pt system
with no inhouse ability to reach 61points and
you have to wait for AI to be able to generate code that can get to 61
and nobody of even average talent would care to, understand or work on that pile of code that was conveniently generated in minutes, even bother
to push it to 61 - and that is a latent threat to the system development ecosystem
Can you have a ten trillion parameter model? Can you extract that many parameters from the dataset/language? I honestly don't know, but at some point you end up with a larger portion of parameters that have very few (if any?) instances, no? (ie most columns are all NA).
You can, but there’s no evidence it will enable abilities at a relative scale and the hallucinations would be wild
@@WearyTimeTraveler That is total bullshit.
What are we expecting to gain from more parameters? I’m not feeling a lack for parameters.
Scale. The bigger the model, the smarter and the more well it generalize
Excellent comparison with Fourier transform!
LOVE ✊🏿
Is this mono audio 😭
You’re massively underestimating the problems with AI’s inaccuracy. You act like scaling will fix all problems when it makes hallucinations worse and you’re just pouring money into more transformers instead of fixing the problems
Read the scaling laws paper
hallucinations are barely an issue
Hallucinations are and will continue to be a problem, but if models continue to get smarter and cheaper, you can use a series of models to fact check the output.
Just like how when I finish a paper, I have several people proof-read it. Probably closer to peer review because you can have models check the logic, check the facts and sources, etc.
Hallucinations do not seem like an intractable problem - just a hard one.
I'm convinced at this point that these comments come from people that don't know how to use AI. They ask murky, unclear questions without realizing it.
They require the ability to hallucinate just as the human brain needs the ability to form hypotheses by free association. So most likely the result will be multiple models for each step in the chain - free association > hypothesis formation -> testing by falsification.
Incredible video and discussion. High quality debates. Congrats for the work and study
❤❤❤❤❤
❤️☺️🍓
Scaling like this doesn’t work.
Says random guy on internet…
@ says random bot on the internet.
Do your research, everything points to scaling as a logarithmic curve, particularly for CNN and Large language model networks.
what's the use of your AI, when you accept Arab funding, add policies that hide the any islamic things from GPT, AI. Literally, GPT dont talk directly but give excuses like Mullah of ISLAM to legtitimise the atrocities by founder
What you are on