Alpha Everywhere: AlphaGeometry, AlphaCodium and the Future of LLMs
Вставка
- Опубліковано 17 січ 2024
- Is AlphaGeometry a key step toward AGI? Even Deepmind's leaders can't seem to make their minds up. In this video, I'll give you the rundown of what AlphaGeometry is, what it means and what it doesn't meann. Plus I'll cover AlphaCodium, dropped open-source tonight seemingly out of nowhere, and causing a big stir for what it might mean for coders the world over. And I'll touch on what I foresee is the future of large languages models and their alliance with search.
AI Insiders: / aiexplained
AlphaGeometry: www.nature.com/articles/s4158...
Deepmind Blog Post: deepmind.google/discover/blog...
Shane Legg Tweet: / 1747670093348176140
AlphaCodium Paper: arxiv.org/pdf/2401.08500.pdf
Alpha Codium Blog: www.codium.ai/blog/alphacodiu...
Alpha Codium … Code: github.com/Codium-ai/AlphaCodium
AlphaCodium Tweets: / 1748043513156272416
/ 1747971746047627682
Twitter Math Kardashian: / 1746168116546093291
Hassabis New Tweet: demishassabis/sta...
AlphaGeometry NYT: www.nytimes.com/2024/01/17/sc...
AIMO Prize: aimoprize.com/
Metaculus IMO: www.metaculus.com/questions/6...
Paul Christiano Lesswrong: www.lesswrong.com/posts/sWLLd...
Professor Rao, On the Planning Abilities of LLMS: arxiv.org/pdf/2305.15771.pdf
Eureka: HUMAN-LEVEL REWARD DESIGN VIA CODING LARGE LANGUAGE MODELS arxiv.org/pdf/2310.12931.pdf
Google Offers Salary: www.theinformation.com/articl...
Mathematics Will Fall First: / 1731096582932578653
Samsung Galaxy S24 Gemini Ultra and Nano: blog.google/products/android/...
Lead Author Trieu Video: • AlphaGeometry
V100 to X100: pbs.twimg.com/media/FbkEJX1WY...
substackcdn.com/image/fetch/w...
AI Explained Eureka: • State of AI 2023: High...
Check out the amazing Donato Capitella: / @donatocapitella Non-Hype, Free Newsletter: signaltonoise.beehiiv.com/ - Наука та технологія
Okay I really want Robin Williams to teach me math.
Update: but also getting AI to master math is absolutely critically accelerating us to AGI.
@@person737 that is a comically misguided statement by him.
Who invited this guy?😁
Inventors rarely understand the full impact of their intentions. Especially in the long run.
david shapiro is the most correct predictions, i still believe biological immortality by 2030
@person737 From everything I have seen Sama tends to misdirect. He's like a magician and magicians assistant all rolled into one. Wasn't it at Dev Days He said the stuff they were working on would make everything so far seem quaint. I think he is trying hard to keep a lid on things and preserve the moat. Well imho 🤷♂️
As different things are bolted together it reminds me of various regions of the brain that are specialized and work together. Thanks again for a great video.
I always thought that'd be how we achieve AGI.
love the way you put it and I have thought the same.
This is what I think about when people say AI can’t do X. The emergent properties that can (and have) come from these parts combining and working together is nuts.
This channel is like setting up a lawn chair on a highway, sitting comfortably and watching a massive truck coming closer and closer 🍿
Nailed it.
It's interesting to think that if an asteroid was gonna hit earth people would probably just vibe.
It's always a good day when you see that black text on a white background
Great video!
Aw thanks py
Best AI channel on UA-cam, hands down. Been following you since the start!
Oh wow thanks cow!
it's because you stick to the research instead of hype-y things and don't dumb things down! @@aiexplained-official
yeah for people who actually want to dig into the details at a decently approachable level 100%
10:06 it’s becoming more like a conversation between LLMs and their environments.
Something I have recently started doing about a month ago, is taking a narrative from my life or someone else’s when I am using ChatGPT and placing ChatGPT inside of that narrative for the totality of that conversation. I’m not talking about telling it to act as a copywriter or systems engineer, but actually giving it a full story with emotional context of the human characters it is going to be helping.
I haven’t tested this in a scientific way, but it seems that the more narrative I provide the more productive the conversation is.
I’m not sure if there is even any connection but what you said made me think about how I’ve been using it and how happy I’ve been with the results personally
I envision systems with a bunch of specialist-ish modules, bolted together the right way, with some sort of integrational module routing inputs to the appropriate domain specialist modules, with an attentional/executive component steering the whole thing.
And eventually it would be nice to approximate the plasticity of brains with ongoing updating of the nets.
That's as good a definition of consciousness as any other accepted ones that I've heard.
I think specialization modules would provide a sort of statistical bias for the models that allow them to learn faster or more accurately than any single module trained to perform a task.
That's more or less exactly what MoE (Mixture of Experts) is, which is how Mixtral 8x7B works, and how many suspect OpenAI works.
I think he means a more complicated and higher level structure that has some self-similarity to MoE I guess@@MrHuman002
I don’t think that will be the path. LLMs are already, everything experts.
Just the right prompt and you can access an expert.
I really appreciate how you always leave citations in the description. This is easily one of the most transparent and well researched channels I watch. Really excited to see how the rest of 2024 plays out with AI developments.
Thanks Quick
Speaking as an IMO Silver medallist, I salute my new robot overlord. But seriously, I've long wanted humans to be able to convert their body of math knowledge into computer-verifiable symbolic form, so at the very least I'm excited for the prospect of LLMs being able to do that for us.
Also, the real advantage of solving math problems symbolically is that there's no hallucination issue - the system knows whether it's gotten things right or wrong. As the DeepMind folks pointed out, the system might not come up with an _elegant_ proof, but it will always be a _correct_ proof. That's awesome!
Great to have you here Snap
And I’m sure eventually they can be trained to prefer elegance. Same way image generation is fine-tuned for human aesthetics.
AlphaGeometry and Eureka demonstrate that LLMs are already conducting their own form of semi-research. Many people don't recognize this, but LLMs are essentially AGI, only requiring enhanced reasoning abilities to solidify this, akin to a child maturing into an adult. For those who perceive LLMs as merely advanced word-predicting semantic algorithms, I suggest trying to learn a new language and constructing novel sentences. You'll realize the effort involved in predicting the next word in order to form meaningful sentences. This challenge is often overlooked because, as children learning a language, we struggled to form coherent sentences. We trained repeatedly until we could recall the patterns in language that made sense. Similarly, LLMs can reason; they just need to deepen this ability. Once that occurs, LLMs will likely solve mathematical problems on the first attempt and be capable of generalizing mathematics. At the end of the day, math is a language for understanding natural order.
Thank you Phillip, it seems like you're prompting of the LLM'S last spring in the way you did was on target not just with this news, but some of the things covered in the QStar video, have a great night, and thank you for sharing your time and work, peace
AI Explained and Dave
Shapiro are by far the best channels dealing with AI right now
Yeah but Shapiro is not a close second.
Kim telling taylor that the "dx" was way too thick is some AI comedy gold! Nah Kim its not thick enough and we all know you could make it thicker.
Indeed, an AI that can work as a kind of motherboard to manage and interface between several specialized AIs would be interesting to see how it would perform.
Just started reading about this today, very excited to hear your thoughts
Don't really know what to think of all the "AGI is near", "well, not THAT near", or "AI will take all human jobs", "I said that? I just meant _transform_ jobs" statements these days - and these conflicting comments are from the SAME people... Backtracking is the way of 2024 apparently. 🤷
Even if it were here today, which depending on your perspective some might say is the case, it takes time for the world to change. It isn't like a light switch and everything changes. We can probably look back at the first few Industrial Revolutions for guidance. There is always a buffer period. Society is going to need to adapt, and a lot of infrastructure still needs putting in place. That means digging up roads and building data centers, education. All manner of things that take time. It is going to look quite soft for a while yet.
it's hard to predict what new tech will come out tomorrow. agi means above human intelligence, and we can't define what human-level intelligence is still. what i think is clear for now is that if you work with software (programmer, developer, even banking and accounting) you better start learning how to use ai as a force multiplier to not get laid off
Thank you for providing up to date, quick and concise videos.
Thanks alpha, appropriate name!
Was waiting for this vid and your insights did not disapoint...love you bro...thanks for keeping us informed
I really feel like sometimes optimists do not appreciate problems surrounding search and storage.
Examples like this are a great way to approach the type of opptimism expressed by the likes of Wolfram for example.
Thanks meme, as always
Great video as always. Keep going dude!
Thanks Noah!
Yay, a new AI explained video!
Thanks as always! Brilliant content. 🙏🏼
The second part is MASSIVE. It shows how much there is still possibilities withing GPT-4, just by proper prompt engineering.... Fantastic!
Regarding detailed prompting, I’m starting to create a computer game completely assisted by LLM (gpt4, Claude, bard, dalle). And the first thing I found out was that not skipping steps gets me much far. I.e. if I go from “idea to code” I fail completely. Instead, I go from “brainstorm” to “concrete idea” to “roadmap” to “vertical slice tickets in first version” to “architecture discussion and definition” to “ actionable tickets” to “help me implement these tickets”. Always giving relevant context I got from the previous steps to achieve results on the next step. This is done manually for now, but I guess if I find this working I’ll wrap it up into a more formal product.
I know this is harder than it sounds but I really believe what will make things happen is - Giving the model continuous updatIng to the world model while updating a "belief" system with RL.
Another brilliant share! 🏆
Thank you so much! I wish I could explain how happy I get when I see you releasing a new video, you rule since day 1!❤
edit: I've decided, I NEED to get access to your patreon no matter what!
Thanks so much my man. Feel free to dip in and out!
Alpha geometry has part of the code public in a repo! That's very interesting coming from Google 😮
AI is not going anywhere, things are heating up, crazy how much cash, energy and passion is now behind developing and sharing this, future gonna be absolutly mind-blowing in all the best ways.
So much value packed in one video. Thanks
Thanks so much OB
When you says give the model space to explore (e.g. through Reflexion) its important to remember that this is only successful with the super powerful models such as GPT-4 or GPT-4 Turbo. With GPT 3.5 Turbo Reflexion and chain of thought is super inconsistent and at times arbitrary. This is why I like the Test Time approach - which has been proven to upscale the performance of smaller models. Also, a verifier model ranking according to scalar estimates such as ‚estimate the revenue this business plan will make in €‘ is a super straight forward scoring system which makes it super easy to filter out ideal solutions among thousands of suggestions, i.e. it can be scaled easier and implement in a straight forward way.
I can also imagine that the most powerful AIs will integrate these engines as modules (like AlphaGeometry) they can call upon when needed. So you will have some particularly powerful GPT organising and understanding and pulling in the resources as needed.
Backtracking: What if we can train an LLM with, "This is simple sentence that illustrates jahskd correcting previous tokens." i.e The training data is augmented by just inserting a noisy token followed by a special backspace token. At inference, the special token removes the noise and backspace from the context window on just that itteration.
Extend the idea, and you now have an LLM that can re-think things starting from an arbitrary point in the past?
Brilliant content, you should have a podcast 😊
On my AI Insiders I do!
Thanks for the hard work!
The pieces are getting in place. Who knows how long we have before the puzzle is finally complete?
AGI hype
I actually saw this guy on IG several times, and although I would say that explaining math using celebrities COULD be beneficial, I think this guy is just good at explaining math
What’s his ig handle? Thx
@@ChristopherLegend @onlocklearning
🎯 Key Takeaways for quick navigation:
00:00 🧠 Alpha Geometry Achievements and Implications
- Alpha Geometry achieves high scores in the International Math Olympiad (IMO) for geometry problems.
- The system combines neural networks with symbolic pre-programmed systems, showcasing the alliance between language models and traditional problem-solving methods.
- Despite not achieving AGI, advancements in neuro-symbolic systems like Alpha Geometry suggest progress towards more sophisticated AI models.
03:12 🔍 Mechanism of Alpha Geometry
- Alpha Geometry's approach involves a neural language model proposing solutions combined with a symbolic engine for mechanical deduction.
- The system utilizes synthetic data for training and can generate novel mathematical constructs to solve problems.
- It iteratively refines solutions through a loop of proposing constructs, solving, and adjusting based on feedback.
05:20 🔄 Neuro-Symbolic Approach and Generalization
- The neuro-symbolic approach of Alpha Geometry isn't entirely novel but represents a significant advancement in solving mathematical problems.
- Generalizing such systems across mathematical fields is a current focus, with potential applications beyond mathematics.
- While achieving impressive results, these systems still lack the symmetrical elegance of human-discovered theorems but focus more on functionality and effectiveness.
07:21 💻 Hardware and Future Prospects
- Alpha Geometry's performance could further improve with advancements in hardware, notably with GPUs like Nvidia's V100.
- The evolving hardware landscape suggests continued progress in AI capabilities, potentially leading to solving more complex problems in the future.
- Discussions about the implications of AI advancements, including reasoning capabilities and the potential for AGI, remain ongoing.
09:38 🌟 Alpha Codium and Iterative Problem-Solving
- Alpha Codium, an open-source project, demonstrates a similar theme of iterative problem-solving through LLMs proposing solutions and adapting based on feedback.
- The process involves reasoning about problems, generating potential solutions, and refining them through testing and iteration.
- This approach contrasts with the traditional prompt-to-answer method, showing a shift towards more interactive and conversational interactions with LLMs.
Made with HARPA AI
There are many algebraic problems that can be transformed into geometric problems. I predict there will be future developments that integrate all of math.
Honey wake up, new AI Explained video just dropped
i cant help but notice i did sort of call (in a comment on this very channel) that using auto deduction/formal inference to generate synthetic data would help. not necessarily hard to call, but still very nice and quite surprising as well to see it happen so quickly! on the other hand, we saw gpt-4 solve imo problems before, so it wasn't entirely new. i thought the approach they used could probably be refined (a lot). for instance, the synthetic data generation step ought to be tried to be filtered for "better" proofs (shorter, more generalization, more symmetry, etc.). the idea that this would represent some "aesthetic" bias is absurd. those kinds of "biases" help human math people because, as it turns out, on any number of fundamental levels, nature looks symmetric. also, the more mystical and magical ai looks, the less progress we will make in thinking of ways to improve it (not that we are in any danger of that!!). thanks for the video👍
Excellent video
btw, if you draw the given data correctly on geogebra, it automatically shows the result.. because geometry problems (intependent from its difficulty) are made with at least enough or more information on angles or lenghts etc (the data), in order to secure that there is only one solution to that particular problem.. so sometimes it can be hard to solve for humans to see.. but it cannot be the case for machine. it outta be precise.. it easily measure the distance or angeles etc and show us the result.. but solving with rules, theorems, lemmas, axioms, trigonometric tabels etc. i mean solving without measuring is another thing.. in that case, the AI comes in.. but all my tryouts via gemini, chatgpt, copilot.. i didnt get right answers on math or geo problems.. i hope it will change in the near future... hopes up on alphageometry
The biggest accelerator is probably gonna be continuous learning... Because assume you use such a structure, as seen in these examples, but the LLM that does the work is improving as well with each step. The more it fails the better it gets, the more it solves the even better it gets. Until the limit for its size are reached (and now assume you could introduce new neurons after that point to top it off)
I wonder when the first real world system uses this structure, for example a cloud lab for chemistry
That's what I'm waiting for, and I agree wholeheartedly.
I'd imagine we will get something like that which has a sort of accelerated growth until it hits a sort of compute/resource ceiling from allocating new neurons/connections or whatever. It's not hard to see after that a quick adjustment that allows it to cleverly forget things that aren't important, which is exactly the sort of breakthrough that brought us the attention mechanism, just focusing on the opposite thing. Who knows what happens then
Good video mate 🎉
Thank you anderson!
did you read my comment about the "haitch"? :D
i was wondering if this was a similar thing as "RTX 3080 Tee Eye" vs "Tie" where nvidia itself is inconsistent
Wow just that short clip finally made me understand integrals 😂😂😂
No one ever explained It to me like that 😂
Santiago's steps are also eerily similar to standard TDD, Test Driven Development.
Good video as always
Thanks X!
Awesome time to be subscribed to AI insiders! You absolutely carpet bombed us with content today.
Amen! 14 hr day for sure, based on weeks of previous research.
4:58 so the ai has an ooda loop now. Sweet so it’s better than before. Plus if we speed up this looping process and add more then we basically are backwards engineering human through a different medium kinda like how a copy is a portion but then put the portions together and you have the real thing just separate from the current one.
Told ya we need Symbolic systems still
Winning a math Olympiad is very significant.
When you can not just replace a bad worker, but the best worker, uuuhhh weee.
Mathematics isn't just about proving things. It's about making abstractions and recognizing patterns and associations. In order for mathematics to "fall" an AI would have to at least develop conjectures and prove them (it should probably also be able to develop a construction that could be usable to study a problem of interest). Making sensible conjectures may demand AGI. And the search space for developing new constructions, whether to make associations between different areas of knowledge or just to prove a conjecture, is likely way too large without judgment. But a proof bot could be a tool that a mathematician uses to explore mathematics.
It should also be noted that it sounds like this bot was trained on proofs in a well-developed area of mathematics. If I understood, the training data was other proofs in that area. Using well-developed methods in new situations that closely follows the original use is certainly very useful in mathematics, but even were the bot to be used only to find proofs as directed by a human, and nothing more, if the bot were to lose effectiveness in a novel area of math where new axioms and definitions were introduced it would greatly reduces its usefulness, especially if it's even unable to abstract known methods from other areas of math and apply them to the novel area.
Most people just don't understand what mathematics entails. The fact that math can be parsed easily and uses a very precise language is great for an AI. But mathematics requires a tremendous amount of creativity, ability to abstract, and ability to reason. Symbolic reasoning is not enough, there needs to be deeper reasoning. You can get a whole lot done with effective symbol-pushing. You can get a whole lot done with intelligently searching through a database of proof ideas. You can get even more done when you combine the two together. But you can't crack mathematics with it. You're not even close.
Well articulated, thank you matt
All of science can be crudely summarized by the guess and check iteration method, so why not AI?
It's interesting, LLMs and ML in general are created by essentially bruteforcing weights & testing until it gives you the answer you want, what alphageometry, codium & eureka do use LLMs to get a good guess as to where to start instead of from scratch. Its like one off last mile training
Thank you for referring to Twitter as Twitter.
Interesting paper, but if they said to not overhype, there is a reason. The euclidean geometry considered in this paper is a so-called *decidable* theory, which means that there is actually an algorithm that mechanically solve *all* problems in a finite amount of time: it is the algebra-based one. They just didn't let the algebraic algorithm run for more than 48h 😁 So this kind of problems allow brute force, because no intelligence is actually required to find a solution. It is a step to AGI as Alpha-go is: none. This is an example of narrow AI that can solve problems in domains that just require "smart brute force". Alas, true mathematics is a different beast. It is undecidable: there cannot be any algorithm that solve all problems. So the methods in this paper won't really scale, not at human-level. They could provide just useful proof-assistants. My prediction of no AGI in foreseeable future holds true once again.
very interesting!!
Thanks Kevin!
Great video! Can you please add a link to the Kardashian teacher video?
In the description!
Why is the estimate for gold in the IMO 2027? Is there a reason why this exact approach wouldn't work for the other questions?
I think this "intuition-er" + "reasoner" back and forth will work for the other areas of the test and there's about an 80% chance gold in the IMO will fall in 6 months time.
The future of education is mind boggling! The engineers of tomorrow are going to have fun jobs.
Our children are going to build things we lack the imagination to even conceive 🤯🧠
These large language models is actually reflective of progress and development happens in the real world. First principal allows us to not pollute the solution space and these AI models seem to be very good at suggesting novel solutions.
another 5 years of AI significance worth jampacked in one AI explained video
This looks very promising!
One approach might be applied to not just geometry algorithms but other standard standard algorithms as well such as Dijkstra's algorithm (such as used for route planning) and Auction algorithm (the resource assignment problem) as well with dynamic program algorithms and game theory.
Also, the AI might help with the costing of these.
AI seems to serve much as human intuition were more standard algorithms are more logic based where the two can work together.
Every single one of your videos makes me increasingly excited for the future
Thanks!
Thanks Christoph, kind of you!
Baby wake up, ai explained just dropped a video
will we get life extension explained?, in a older video you said that liked the accompany the news
Maybe sometime
I'm curious would using more principled reasoning mechanisms from RL make this even better?
Tech ceos are now always talking about agi in X, they are probably easing the public for its arrival. Looks like it really is coming very very soon.
Crazy times we live in.
Babe wake up new AI Explained video dropped
She's already watched and subbed
will we ever get those videos on the main channel ? like maybe later? :( i cant afford it otherwise i would have 100% been joined insiders :(
I understand ! One option is to sub after a whilr and then binge everything! Then unsub after. Many people can write it off as training expense too, if you are employed.
Meta just announced they are aiming to have the equivilent compute of 600k H100s by the end of the year. That is around 2.4 zettaflops AI compute power (FP8) or 1.2 zettaflops with FP16. With 3x gain from good data quality, Active learning provides another ~3x gain and 3x gain and architecture improvements (RoPE, SwiGLU ...), and using FP8 with 50% utilisation we could see a model trained with 11,718x effective compute over GPT-4 in 90 days at the start of 2025😂💀 (assuming 2.15x10^25 FLOPs for GPT-4s training). Also i could see upwards of 10x gain from data quality which would give 23,437x effective compute over GPT-4 assuming about 50% utilisation. If that isn't an exponential cureve i don't know what is lol.
I mean, they just have a neural network whose purpose is to hallucinate some steps that are commonly used... What makes it a viable strategy is just the mechanical solver. But it makes for such a great idea to implement in a chess engine. A neural network that first hallucinates a sensible move to make the alpha-beta pruning faster.
At 8:44 you say that Google plans to add AlphaGeometry to Gemini, but you show the post that seems to talk about adding the visual input to AlphaGeometry using Gemini. Aren't those different?
As in it might be a yet improved version that gets in
@@aiexplained-official ah, ok, thanks for the clarification. Could you please share the source that promises to add AlphaGeometry to Gemini?
I'm interested because I'm an IMO participant myself, and my math teacher is worried that AlphaGeometry could be really detrimental to Math education. Because if this tool becomes easily available, this will make students less motivated (because it's so easy to cheat and ask the model to solve the problem instead of solving it yourself)
I guess the AI has already kind of ruined the home assignments like essays.
Do you have an opinion on whether AI will have a net positive or negative impact on education?
I could see arguments for both. I'd guess the education system will definitely have to adapt.
Hey reason, all sources in the description, think it was the NYT one. And well done on getting to IMO, further than I ever did. As for education, honestly I feel, by the 2030s, it will be much more like edutainment for most people, even schools. IMO participants might even be like chess superstars, where everyone can watch along while an AI explains what they are attempting.
Phillip, I think the thing I want to know is whether or not you think the development of AlphaGeometry is proof of the validity of what that one Chinese guy said in the interview in AI Insiders a couple weeks ago where he said something akin to "Functions (or tokens, I forget which) are all you need", since it seems to me, anyway, that LLM -> Solver loop demonstrates that his intuition (or, as he said, YOUR intuition) within that statement really is beginning to open up a whole new class of L(X)M innovations that are so much more sophisticated in their end use cases
It's definitely linked and could be expanded with time, for sure
Would you consider some discount on the insiders for people living in developing economies?
I wish I coupd but Patreon doesn't allow this, unless you can find a way that I can't. Don't even allow promo codes.
@@aiexplained-official thanks for the reply and for the amazing content, cheers!
okay but how do you do all these videos and prepwork, it is genuinely easy to be in disbelief since you do all this alone.
Yeah so far everything, the channel, Insiders, all alone
nice i needed something to watch
Let me know what you think!
@@aiexplained-official I am quite stun by the ability of deep mind to produce new expert AI on various theoretical field. It also make me wonder why gemini feels a bit of compare to the deepmind standard. It felt like it was push out with the deepmind brand a bit early in the process. But with what we learn from the secret GPT4 sauce and Mixtral. I expect gemini to really shine once all of this army of expert will be gathered under one maestro to work in concert. Wait and see.
8:30 The ability to reason is not a magical concept -- it is grounded in the relationships between symbolic representations of concepts. A task perfectly suited to LLMs. I don't really get why there's still so much 'debate' about this. The only reason why public-facing LLMs appear to have trouble reasoning is because they aren't given space to reason before providing an output due to compute costs.
It seems Hallucination can be used as a benefit for finding novel solutions that can be tested and thrown away if not applicable just like a human would have epiphanies and try and test said epiphany?
Thanks Dude. 2024 is going to be even more exciting than 2023 it seems.
This is an incredible project but, to me, it could be more remarkable.
Imagine the human brain, which is somewhat already adept in Mathematics, being able to practice 100 million geometry problems and being able to retain each method and solution - we would perhaps see the same level of “scaling”, particularly because human cognition abilities, in terms of contextual and intuitive reasoning, is still somewhat superior to AI and also explainable.
What would make this remarkable to me is if we had a way to test the sample complexity, in terms of:
i) Finding the bare minimum amount of training examples needed to reach SOTA level.
ii) The exact examples in which the AI “abstracts” its answers from when approaching a new question i.e. can the AI solve a difficult geometry problem from a collection of unrelated geometry problems or is it genuine few-shot learning?
There are many more things that can be researched, especially the explainability part, but nevertheless this is a fantastic step towards enhanced reasoning and problem solving capabilities.
DeepMind all the way! 💯
"self play" on pure mathematics and code, which has clear objective functions. i can't wait to see more of this.
I find the exponential curve quite worrisome, in 2 years we have cracked geometry, and this too with models which are reportedly, by sam altman of all people, supposed to be "very bad" at reasoning. They didn't even use the H100s for this, not only this but meta training an open source GPT-4 competitor just doesn't sit right with me. I am all for opensourcing but we have only had GPT-4 for about a year now and we are still only just figuring out its abilities to crack geometry. What if in 2 months a prompt engineering technique comes along that makes GPT-4 really good at nuclear science? We don't know what the ceiling of this model is and until we can find that, open-sourcing such powerful models is risky
0:46 I would have thrown a laptop at a wall if I had to learn from that video.
All of math should just be explained by 3Blue1Brown AI clones.
I don't think a lot of the machine optimized geometries/solutions etc. look like trash, but they do look alien for sure, something no human would or could design. They often also seem/are more "natural/biological" because apparently evolution having optimized structures for billions of years has gotten quite a bit of it, if not right, close enough to us to want to mimic for next step up in technology.
It's insane that Ilya already proposed this by 2020, i can't believe they downgraded his authority given that he's THE MAN in Open IA...
Nice
Bro. Can you start a Twitch Channel and stream reading papers and giving your assessments?
Where do I find that math stuff? I have kids that I need to have that kind of help with.
11:39 yeah but you aren’t forcing it to reason logically, you’re making it gather more data. When you get something wrong, that’s data to be used for tuning the answer. If i think a ball should spin when i drop it and then it has no spin that’s something against my prediction so instead of just trying again, look at the details and then see the difference and maybe the wind was countering or maybe i never twisted my hand to make some spin or something but that is what it’s doing, gathering more data to tune the answer. Just like negative prompting or positive and negative reactions. It’s all the same but a little different and that’s the reason for expansion. There are more things that are not than there are. Imagine how a tree has the pieces to the chair but the chair would never be if it wasn’t grown through the details. Life is detail and we absorb and change because of it in all directions, like time. Time is a measurement of difference so the current is just like the waters current in a river flowing but less like a river because a river flows in one direction because of itself. So yeah maybe it is a river and we can move the river like the fish but we need to absorb differences to gain more differences to leave the water and go to the next level like layers of environments and we travel through them and gaining and learning as well as shedding and changing. If the fish were smarter like us then they could manipulate the river kinda like when we manipulate a tree or metal and make things. The river doesn’t only flow one direction but all directions and that’s why differences are everywhere. Expansion is time but we have limited time so we see one direction. Every potential is a direction the pieces can flow in this river. It’s like when we imagine our future and then fill in the details to the current to reach there.
I love the joke that alphacodium's name is. There is a text editor by Microsoft called vscode that, while technically open source, is distributed with blobs. There is a project called vscodium that distributes the open source part of the project without the ms telemetry
The better general purpose ai we get is going to be much more useful than these super overfit boutique solutions to specific problems
3 in one day!
The problem is that LLM as a role for drawing auxiliary lines is too inefficient. It is hard to imagine people deploying a large number of machines to solve a simple IMO problem. This field must be in the early stage of development, and much work remains unfinished
TIL I pronounce my Hs in a Cockney way. Must call the Trouble and Strife on the old Dog and Bone to let her know.
Haha
Drinking game: Drink whenever he says LLM. (I died)
Oops!
Is there a link for the AI generated calculus teachers?
In description
@@aiexplained-official ty. I missed the ai Kardashian link at first. I am skeptical of this. It feels too polished as though there was lots of post production editing to make it look like a natural interaction. This is probably not too far away though. As a math teacher, the casual conversation design intrigues me- this is how I engage students. These AI Kardashians remind me of the Nancy Pi videos on UA-cam. I would love to see an AI teacher that uses her Lightboard style of math explanation.
If it's going to solve all geometry problems, my question becomes "aren't all problems geometry problems?"
If you think hard enough