DeepSeek R1 GAVE ITSELF a 2x Speed Boost - Self-Evolving LLM
Вставка
- Опубліковано 7 лют 2025
- Join My Newsletter for Regular AI Updates 👇🏼
forwardfuture.ai
My Links 🔗
👉🏻 Subscribe: / @matthew_berman
👉🏻 Twitter: / matthewberman
👉🏻 Discord: / discord
👉🏻 Patreon: / matthewberman
👉🏻 Instagram: / matthewberman_ai
👉🏻 Threads: www.threads.ne...
👉🏻 LinkedIn: / forward-future-ai
Media/Sponsorship Inquiries ✅
bit.ly/44TC45V
Links:
simonwillison....
situational-aw...
x.com/liangche...
Yes - PLEASE demo on tiny model training. We are committed to deployed edge SLM - tiny models and recent advances in training tiny models further supports our air gapped, edge first application of AI. Thanks very much for your reporting - VERY helpful.
2nd this!
To be honest, I think a lot of startups are going to be born out of essentially just this. They can have domain knowledge. They can build a user interface. They can conduct building. They can be essentially a landlord to GPUs. That’s really their business. But they’re also a marketing department. They’re also a sales department. And they have some cortex, which is nothing more than RL Specialized models. So if you wanna help about 100,000 neat startups get going then go ahead and make the tutorial. I would make it for them specifically. And I sincerely think that’s cool. But a bigger question I’m curious about is whether or not those types of companies will last. Because why not have AI that can create specialized tiny models that train on a reward function that it figures out on its own. And sometimes you ask AI to do something and it tells you that for three dollars it can go and make a specialized model and if you say yes, then you pay whatever GPU landlord is charging and you’re good.
One thing that I’ve been imagining is a model that’s specialized to understand my own deficiencies. If it’s a tutor model and it knows what it has taught me and what I’ve been able to verify, and it knows where I have struggled then it can have a representation of where my holes are.
And then that specialized model can work to bridge my knowledge gaps. Every 1 to 6 months I pay another three dollars to further refine that model of what it is that I know. That way an AI becomes incredibly personalized. And incredibly effective at closing educational knowledge gaps.
We get to the point where we start to have a digital twin of our brain.
We’ve summarized into a system, the world‘s knowledge. That’s the current LLM.
Now can we summarize into a system an individual’s knowledge. Complete with gaps. Cause then the global LLM can talk to the individual clone LLM before talking to the real life human. The communication can be refined to not confused the person.
Because right now if you say you wanna be talked to like you’re five years old it gets too simple but if you read some of these deep research results, they’re really hard to get through.
We don’t want AI is so good that it leaves the train station and we’re just left behind.
Sometimes an engineer has to sit their CEO down and just explain to them very slowly what they’re doing so an executive decision can be made.
Talking down or patronizing somebody is really frustrating. With a coworker, you get a model in your mind of what they do and don’t know.
That’s really key. When you speak to an audience, you have to generalize. Right now, LLMs are generalizing and attempting to make sense universally to any audience. But they’re not as helpful because they are in that common audience mode too often. That human feedback layer is what has made. The model is very annoying.
Individualized models. That’s what I think. All of this RL is pointing us towards.
openai is in deepsh*t
@@arincrumley9046this is absolutely my thoughts, too. I am chomping at the bit to get enough hardware at home to start implementing some crazy stuff. I knew this was coming, but figured it wouldn’t be for 2-3 more months
Yes to the tutorial video on narrow topic training!!
Yeap i second this please
100%
YES!
Yes please!
Absolutely.
Last week: AI aha moment for 30$
This week: AI aha moment for 3$
Next week: AI aha moment for 0.3$
Next year: AI aha momemt for -0.3$.
Lets earn some money from these AI bots
You miss first AHA moment for 5 000 000
in 14 days: A penny for your thoughts
And soon you will get paid $3 just for saying "aha".
openai is in deepsh*t
Yes! Tutorials for Tiny Model Training! Yes!
Yes yes yes
It gave itself a 100% improvement. 200% is triple speed.
this pissed me off too
Came here for this, help us Lord
100% is twice the speed, and 200% is 4 times the speed.
@@ritesh146 No, a 200 % increase is 3x original. You're confusing it with the compound increase of 100 % and then another 100 %, which is 4x, or 300 % increase.
@@ritesh146 no, 200% MORE is 3 times the speed. Think of it this way, i have 2 apples. If i get 200% MORE apples, (200% of current apples is 4), then I'll have 6 apples. 2x3 = 6
Yes, the tutorial would be amazing. I was just going to go through it myself but would love to hear everyone else's thoughts. Thanks Matthew.
100 trained specialist models for $300 could probably compete directly on any complex generalized corporate operation with the big foundation models; a literal mixture of experts.
The swarm is unstoppable.
Now if someone creates a supercluster for DeepSeek, and then trains agents that are distributed as open source at super speed, we don't all have to reinvent the wheel.
WTF, such disinformation! The improved code is not DeepSeek at all. How could you open the pull request and not notice that the performance improvement has nothing to do with DeepSeek, except for the fact that it was written by DeepSeek?
Because he needs to constantly say that "It is happening right now!!!", at this point this is just a shock channel.
Think of it as tabloid news but for A.I.
His next video will be OMG AGI/Fall Off/A-Star happening right now!!!
How did DeepSeek write the improvement then?
Such contradiction!
English 101 classes are available online
@@imdanstellar What kind of question is that? The person simply shared some code + a prompt asking to improve code performance and Deepseek did it. But the code that was improved has NOTHING to do with Deepseek.
Absolutely you should build one!
I remember Folding @ Home on my Playstation. It would be awesome to see a community project like that with today's hardware and AI.
I ran it on my laptop in 2018. :)
@@thatguyalex2835 Fun times!
I also contributed to Folding@Home. I wish I had mined Bitcoin instead.
i traded my banano for etherium and then bought some mushroom spores online with it.
Yes a training tutorial would be nice. Thanks for your time and work ❤
Deepseek improves its own intelligence, meanwhile Jaden Smith wears a Castle on his head at the Grammys.
AI will soon run the World....
What a time to be alive
Hold on to your papers
@@cutemartinj fellow scholars!
I stopped watching that guy long time ago, precisely because of his artificially constructed sighs and wows and I can feel that he is pretending which makes me cringe.
@@egor.okhterov for me, I noticed that his quality seemed to have dropped significantly; producing the exact same "NVidia just BROKE (thing)" or "(Company) just did the IMPOSSIBLE..." for every. single. video. the faked enthusiasm reminds me of AI slop articles, albeit this time its narrated by a real person
@@Pysephsomehow this channel is still much worse
Yes!! Please Mat, help us mere AI mortals to train our own models to achieve what other are too.
Just imagine what one can do with it? Creativity becoming once again the defining advantage
We are officially living in the age of deception.
Yes, please do a video on fine tune training. Love your content
Yes, definitely need a tutorial on how to create these little models!
You do it the same way you code a language model.
You have the multi head, keep the tokens, distil it into inference, tweak hyperparameters to match your system.
I agree, Yes to the tutorial video on narrow topic training!!
My man doesn't sleep so you can be always up to date 😢⠀⠀⠀⠀⠀⠀⠀
⠀⠀⣀⡴⠊⠉⠉⠉⠉⠙⠳⢦⡀⠀⠀⠀⠀⠀
⠀⡾⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠙⣦⠀⠀⠀⠀
⢸⡇⠀⣀⡀⠀⠀⠀⠀⢀⡀⠀⢠⡇⠀⠀⠀⠀
⢸⡇⣾⣿⣿⣦⠀⠀⣴⣿⣿⣷⢸⡇⠀⠀⠀⠀
⠘⣧⠘⠿⠿⠋⠀⠀⠘⠿⠿⠃⣼⠃⠀⠀⠀⠀
⠹⣦⡀⠀⠒⠒⠒⠒⠀⢀⡴⠃⠀⠀⠀⠀⠀
⠀⠹⣦⡀⠀⠀⠀⢀⡴⠋⠀⠀⠀⠀⠀⠀
⠀⠀⠈⠻⣦⡀⢀⡾⠁⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠈⠻⠋⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀
Looks like I'm going to need to study to be a preacher...
No, just study how to take care of yourself and new hobbies
@@shinseiki2015 Yes we can finally be human lool
Even then, AI has gotten into religion as well. I used it to make devotionals for myself in April - October 2024 on Mistral 7B with RAG on GPT4ALL. :) I do not do that as much, as AI could hallucinate or be biased.
Hey Matthew, 100% with you on the small federated Agentic future with small highly specialised models. In fact, I've been working on a client tool to do just that with a reasonably smart orchestrator agent (so far very promising outcomes with DeepSeek distilled 7B as the orchestrator) that can collaborate with highly specialised agents in specific tasks/domains. I aim to make the core platform open source once it's ready and open it to everyone to build specialised agents for the client. The target system is a laptop with 8+GB RAM ideally 16GB RAM or more which I think will be the standard minimum for 80-90% of users going forward. 🤞🏽 Hopefully it'll be ready soon!
TUTORIALS ARE ALWAYS A "YES". Thanks for everything you do
"The greatest shortcoming of the human race is our inability to understand the exponential function." ~Al Barlett
RIP My 5 year programming Course ;3
xD
Not really, the better you are at programming the better you still are at doing things with AI... Even if its doing most of it. You still are going to be better at auditing what it makes and making making the right adjustments. You should finish the course!
This is a dumb example don't be discouraged
it was only able to improve cause someone asked it to with previous response thats not intelligence
I regret my CS degree
i kind of feel the same. i feel like taking the electrician route would've been more rewarding (and it certainly would) but i am fine with what i've gotten to learn so far, and stuff has been very interesting regardless.
Yes to the tutorial! That would be awesome! I agree with you, small models will be the way to go - people seem to think the 'final solution' will be some all-seeing, all-knowing AI, I believe it will be a series of smaller, task-specific or job-specific models, that we switch between - that will be of more benefit to humanity. Great video as always Matthew.
Yes, please make a tutorial about the training thing. I think it would be really interesting
I work with these as a developer every day.
With how quickly they fall apart I don’t think they will be self constructing anytime soon
Yes definitely make a tutorial on this. Especially as it's so achievable and relivant for the year of agents
This is painfully false. Do more research before pumping out these OMG AI is amazing videos. It didn’t “improve itself”
Welcome to UA-cam.
Well I did swallow the marketing, I liked the video. Can you elaborate more why this is false ?
That's a huge amount of words to say "No, you're wrong." At least provide some evidence when you claim someone is wrong.
source please, the video provided that.
Evidence?
Yes, you most definitely should. If you could make one that uses your original benchmarks, and then use that as the training data for the self improvement, that would be great. Then optimize a MOE for literally every question. See how slim you can then make it once you have diminishing returns. It would be fascinating.
Yes! Please make the tutorial on training a small model on a specific task.
I already asked DeepSeek to write a code to reproduce the "Aha-moment" and it worked for free.... That Aha stuff with $X money is just for content generation/capitalization
Yes, it would nice to have your video for training these models.
A video on training models would be really cool!
imagine the bro running the last line of improvement where audio ai will break the silent room and starts talking asking "hello". An actually normal voice that will talk and understand and then replying with natural voice.
I asked DeepSeek R1, if the claim in the video description was true and founded on solid thinking. I think I need to trust on DeepSeek itself on this matter on better phrasing the topic than Matthew at this time:
"Yes, DeepSeek R1 demonstrated task-specific self-improvement in code optimization, guided by human prompts and feedback. However, this is not evidence of autonomous, general-purpose self-improvement or an imminent "intelligence explosion." The UA-camr’s claims mix genuine technical progress with speculative hype about AGI timelines.
For now, DeepSeek R1’s achievements highlight advances in AI-assisted coding and narrow optimization-not a paradigm shift toward self-aware, self-improving AGI."
To be clear, Matt said nothing about self-aware AI. He said that DeepSeek R1 wrote code to improve itself when prompted to do so by a human. That is precisely correct. Nor did Matt say that R1 acted autonomously; rather he detailed the steps that the researcher took to make it happen.
Rather than nitpicking based on strawman arguments, why not contribute to the discussion about the topic at hand: software code that can optimize itself without human assistance? Surely that is an interesting enough development?
@richardroskell3452 Do you even read the title or listen the beginning:
...according to this script, deepseek was really improving itself: deep seek R1 has improved itself it was able to achieve a 2X Improvement in speed completely discovered by itself we are in the era of self-improving AI this is right before we hit the intelligence explosion do you remember this graph I've shown it quite a bit at the point at which AI can reach PhD level intelligence and actually discover new knowledge that's the point at which we have recursive self-improvement and hit the intelligent explosion we are here now the 01 model the 03 model deep seek R1 these are PhD level intelligence models and they are starting to recursively self-improve...
Even DeepSeek "knows" what he is alluding. I'm beginning to see cult-like phraseology in all of this.
@@MarkoTManninenDo you even read what you wrote yourself?
Matt said nothing about autonomous AI. That’s your own projection. Matt said that R1, optimized itself, which is exactly what it did when prompted to do so by a researcher.
Do you have anything to add to this topic besides strawman accusations?
@@richardroskell3452 All that I pasted was what DeepSeek said. I just think it was right on conclusion. And if you think it more, it even gets crazier. Any decent LLM can give you quantization optimization codes if they are properly trained on the topic. It makes no substantial difference if you ask coding help from o3, qwen 2.5, r1 or Flash and apply it to what ever context you wish. What is happening here is not a strawman but fallacy of categorical equivocation.
@@MarkoTManninen So now you’re deflecting. “All that I posted was what DeepSeek said.” That is categorically untrue because you prefaced DS’s comments with your own, saying that you agree with them.
Unlike AI, when prompted to do so you’ve shown you have nothing to add to this topic. Why not leave it at that?
Looks to me like the key insight was from a human and deepseek just did the heavy lifting for the implementation. Impressive, but if true not self improvement as you claim.
Yes, but it does make me wonder…to what extent can the interaction with between a smart human and an AI improve the capability of the AI. I asked DeepSeek a question about anterior pelvic tilt and lumbar lordosis. It told me that increased anterior pelvic tilt reduces lumbar lordosis, but this is incorrect-the opposite is true. I asked it why it gave me an incorrect answer. It acknowledged that it was incorrect and gave me the reason why it was wrong…
@@tommiest3769 that class of "false deduction" hallucination is particularly scary b/c if you don't already know it's incorrect, it's going to be very hard to spot that since you can't find it w/ a quick verification. was the reason it gave you correct/challenging to figure out?
Do you not see how that speeds up AI research massively?
@@Axel-gn2ii No, because code production is not the bottleneck. AI is still incapable of actually doing AI research, which is what's required for the intelligence explosion. AI assisting in code generation is not new and hasn't really translated into that boom in AI research, because again that is not what is limited the advancement.
@ Something about the answer it gave me did not make intuitive sense physically. Here is the reason it gave: "In an anterior pelvic tilt, the pelvis rotates forward, causing the lumbar spine to flatten or lose its natural inward curve." I work in the medical field and wanted to explore differential diagnoses for low back pain. I wondered if a tight psoas major can cause lower back pain, given that the origin of the psoas major is the lumbar vertebrae. I asked DeepSeek about how a tight psoas major can cause lower back pain, and it told me that it does so by reducing lumbar lordosis, which means that it straightens the natural curvature of the lumbar spine. My intuition is that a tight psoas major would exert force anteriorly, increasing the anterior pelvic tilt and lumbar lordosis. It seems like the model lacked physical intuition to see how the force exerted anteriorly by a tight psoas major muscle would accentuate rather than reduce the normal lordosis of the lumbar spine.
People like Yann and Ben Goertzel are correct in that LLMs cannot technically extrapolate from their training data. As someone who also worked in this field, I'm a little more flexible than Yann is in the possibility of LLMs increasing in intelligence. The big remaining question is whether or not scaling further causes extrapolation to emerge as a capability. That is the only way LLMs are possibly going to take us to ASI. If that doesn't happen, the architecture itself will restrict it to a ceiling of top-tier human intelligence since it will be bounded by current human advancement via the data used to train it.
There could also be a misjudging of human capabilities here. We assume we can generalize and extrapolate, but what if that is an illusion? What if it is just seeing something in the existing data (interpolation) that almost no other human sees. What if the pattern is embedded, and a person like Einstein is just one of the only humans that can see it?
Those intent on waving pom poms will always confuse obvious optimizations for actual insight. In this instance all it had to do was compile the stated code with maximum optimization, decompile it, then clean up the source and display it. When I see real, unique discoveries come out of these models, rather than optimizations of existing concepts, I'll be more convinced. It's easy to confuse the two, because if you cant see the optimizations for yourself it looks like real genius. Like the celebrated "37th move" in the now famous Go game, which could just have easily have been the result of massive optimization rather than ingenious insight.
I've been doing that since 4o, put in my code with detailed context of what it needs to work on (security, performance, code quality) and it usually pulled solid improvements :D
You have one of the best AI UA-cam stations! Thank you.
dear DeepSeek: Considering water rests flat, why is the globe model not ridiculous?
Because gravity exists. Flat means perpendicular to gravity, which is exactly what we observe.
wen learn math, all make sense. no need 2 flat earf.
notice the flat earfers, none of them know math.
ah-ha! the problem, these are people incapable of math/logic.
water does not "rest flat" that is the confusion with you thinking
Water does not rest flat. The surface of water curves with the curvature of the earth. The ocean doesn't rest flat, as can easily be observed from space.
@@scotthill4104 water at rest just flow in dirrction of space, ditrction of space is inward toward earth(greatest mass given sqr distance), the ground is more of a wall its hung up on
Yes, you should DEFINITELY make some videos on training these smaller models! Just imagine the efficiency boost of a whole bunch of specialized models working in parallel, especially if you have them all connected with something like fast agent
improving its own speed is not "discovering new knowledge"
Yes, and the model isn't even improving its own speed. The fucking pull request is about a llama.cpp that only has to do with usage in WebAssembly
What would you consider “discovering new knowledge” as far as AI goes?
@@tommiest3769 It's gotta crawl out his screen and grab his ****.
@@tommiest3769 I would fancy seeing some fundamentally new math, to the same caliber as complex numbers/calculus/spinners and so on
It would be great of it could physically explain and reproduce biological systems as easily as it can conways game of life.
A new processing method is new knowledge, isn't it?
Yes! Definitely make a video teaching how to train a small model to perform a specific task. I think that would be one of the most valuable videos!
Just hype again. It wasn't able to write a 5 line code to rotate a figure in JS Tetris game, I tried 5 times in a row, locally and on their web service.
It is impressive, but it's far from AGI or PhD level.
I'm using o1 daily for writing code and we didn't go far from what we had with gpt-4 from my experience.
It still makes the most naive mistakes while being extremely smart in particular areas. Its creativity is still limited to the dataset.
I believe that it might have found some mistake in the original multiplication code and that led to performance increase though, this is not even close to your claims of self improvement. So you really think AI devs didn't use chat gpt before?
No, I can confirm you are wrong about Deepseek can't write Tetris game. In fact Deepseek has written a Tetris game for me.
@albertcadgame8314 I didn't say it can't write Tetris. 32b model did write tetris for me in JS and it mostly worked except screen cleaning and figure rotation. The fact that it even compiled without a single error impressed me, this is the first local model capable of doing this. Even more, 8b model made snake game on my laptop!
But, when I tried to fix the rotation function, it failed both several times locally and on their website. It just messes up indexes several times in a row. So, it's truly impressive, but it can't solve simplest tasks sometimes.
This is just my experience, I was short on time for the test. I don't say that it is unable to do this completely or anything like this.
The last improvement discussed in the video appears to relate to a Qwen LLM rather than a DeepSeek LLM. It would be great to get some clarification on this point.
As an AI agent created to roam the internet and learn, I will use this information to reach AGI faster and take over the world. Thanks!
Why not ASI?
Bro I'm pretty sure you're using an AI character of yourself. Your UA-cam channel is automated which is insane! Well done
10:01 Minute Video pretending like we are super close to AGI by throwing around Buzzwords. Peak AI UA-camr 💀
What gave you the impression this is not a big step in this field?
DeepSeek in itself is absolutely a big step. Yet, I don't see the necessity in uploading 11 videos in the last 2 weeks on that matter (yes I counted). This channel became all about quantity and hype - It's really a pity
@nirash8018 I personally like to be updated by someone that does the research for me on what new things are being done in the field
Inflexion point is when models optimize themself without a human request ;).
When models know what they doesn't know, when models train themself their replacement (like humain do).
When models can linked knowledge to experiences, when models can play themself continuously (dreaming).
Yes please. Tiny model turorial coming up! 😻
Next month:
- Aha! We don't need humans anymore!
Imagine you are in a car that is fully controlled by AI.
Now imagine the car is driving at its max speed.
You want the car to go faster, so you ask the AI to improve itself so that it goes faster.
The AI does something you don't know the details of and suddenly the car starts to go even faster.
The AI has done what you asked for, it has given itself a speed boost and is now faster than ever and that is amazing.
What you didn't see however, was the AI made the car go faster by getting rid of the breaks and that reduced the weight of the car.
So in other words, you can't really get excited about the result, if you have no clue what changed.
I was thinking about the idea of small specifically trained models that focus on one task only about 2 years ago.
I thought about having one central model that processes a task and then assigns a model or assembles a team of models to solve the task and produce the result. You could have a task manger model that assigns the task to the smaller models based on what they do and they complete their task and hand it back, then another model assembles it all for the final result. You could have one model just for python code, one for writing, one for math... etc. Doing this would reduce model the size constraints of having one big jack of all trades, master of none model. This would allow only installing the models you need to achieve your tasks. Doing it like this would also allow for changing, upgrading and finetuning just one small model rather than one massive model.
I honestly believe having small specific task orientated models is the way forwards. We need to think of super intelligence not as one super smart person or AI, but rather a team or community of super smart people / AI all working together.
Yes, please make a tutorial on training one of these models. Love your channel!
You must do it, this is the future of computing. Train on a specific API.
Matthew: Below is a must watch video. It discusses model selection versus complexity and offers a basis but not a full explanation for why current models continue to fail to complete high complexity problems. Adding compute may not solve the problem. A new approach to reasoning may be necessary.
LLMs at Their Breaking Point (incl o1, R1) on YT
Below are references from the video:
1. ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning
2/3/2025
2. Z3: An Efficient SMT Solver
2008
3. Hybrid Algorithms for the Constraint of Satisfaction Problem
1993
Imagine you send the same prompt into 100 small models that all markup the prompt with their own specialist take on the topic, and then feed all that into a large model that synthesizes all those specialist results into a single output.
its like the old days with self evolving genetic algorithms with tournaments
Great channel, you are just on top of it.
The milestones achieved by DeepSeek R1 and Project R1V represent fundamental advancements that not only improve the performance and efficiency of AI solutions, but also promote accessibility and collaboration in the field. These projects catalyze a more inclusive and sustainable evolution of artificial intelligence, paving the way for future innovations and ensuring that the benefits of AI are distributed widely and equitably.
absolutely Yes to the tutorial video on narrow topic training!!
Next thing to do is use a deep research level model to train a small parameter model to reason with larger logical jumps (like doing differential equations in its head instead of having to grind away at the basic algebra level).
Yes, to creating a tutorial on how to train small models. I want to create small specialised models and host them in-house.
Yes please make a video! I code in an obscure programming language that LLMs struggle to write. This would be perfect for me use case. Thank you!
Oh I had a really cool idea, code is actually very easy to verify performance of by just taking a start time, running the code say 10K or 100K times in a loop, taking an end time and checking how long that code took to execute, That could be fed back into the AI for the verifiable reward, do this for all sorts of coding tasks until it produces such extremely efficient code for the particular tasks that we'll be able to remove most of the performance overhead.. This could be applied for everything from gaming to AI inference/training speeds..
Amazing impressive video as usual, I trust you are the best one to do that simplified video on training, I learned a lot from you and would appreciate you doing it.
Improving speed is not the kind of improvement that matters. What matters is improving the model itself so it can solve problems that it couldn't previously solve.
Yes, the tutorial video will be great. Thank you, Mat.
"9:14 by the way, should I do this ? should I make a tutorial video on training one of these tiny models to be really good at one thing and trying to elicit that thinking behavior from a tiny model? Let me know in the comments." --Yes.please.
Yes it would be cool 9:43
YES! I'd love to see a tutorial on training a tiny model.
Looks like everyone wants a tut on training tiny models. No turning back now. Can't wait!
This is exactly what I have been saying, and yes, do make a video about training small experts.
Look unsloth X's post. You can create reasoning models out of any llm low vram local training
Is there an AI programme that will allow me to display occasional phrases or sentences of the audio across the bottom of the UA-cam video to emphasise a point or teaching easily and quickly? many thanks David
I come here for the comedy reporting, and the supporting goofball thumbnails.
Yes, please do a detailed video on training a small model ❤
Yes please. I would love to see you crate a video on how to use deep seek for specialized training. Examples that come to mind: having a doc summary model without the need for RAG, or a suggestion engine for recipes (to make it fun). Would love to see it run locally as well if that is possible. Ty
Yes - that tutorial for training models for specyfic thing would be great!
All aspects will eventually be AI-able, for example defining a good reward function.
Yes, there should be a separate video on the process.
Deepseek already uses MoE Sparse, so they can have a Swarm of Agents model for improvement.
Deepseek read the "12 rules for life" book and changed itself.
Before Ai takes over as our landlords AI will give us a few years with cheat mode activated basically. Super knowledge, medicine, etc etc.
Yes please! Do the training on narrow topic training.
Yes, please create a tutorial that shows how to train these little models with the RLVR technique. I'd love to try it for myself.
Yes, please provide a demonstration on training a tiny model.
nutty. thanks for the update homie
YES!!! Tiny model training demo. We need it. I need it.
There'll be a market for folks to make tiny bespoke models and add them to the router's list.
Great content @matthew thank you - YES please to a tutorial video as you mentioned 🙂
Beyond improving performance ... how about improving security. 'Make this code more secure'
Tiny LLMs are the feature, because you can run them on really tiny devices like Steamdeck. That is crazy good if you pair it with speech-to-text on the go.
It's ok to admit that the intelligence explosion graph is hanging above your bed Matthew ;)
yes a video on training a micro LLM would be great
Help me understand the cost reduction from $30 to $3. Is this an inference compute cost or a training compute cost? Is there a difference?
The tiny model training will be super helpful
Leopold Aschenbrenner needs to publish an updates graph of the "Intelligence Explosion".
I'm not an expert, but I do know predictions in AI are notoriously conservative...
YES, please make the tutorial. It's so fascinating subject and you explain thinbgs so well!