@@Sofiazinha-j2w it's not just about their talking speed, it's more about the depth of their knowledge and how well they articulate it. (*Never had the need to slow it down for Marc)
IMO... Listening to Andrej Karpathy talk about innovation, work on deep learning research (e.g. assembling LayerNorms, Skip connections, and other parts to create a trainable "tissue" called transformers) and current/future problems is a breath of fresh air for young researchers like myself.
🎯 Key points for quick navigation: 00:02:00 *🚗 Andrej Karpathy discusses his work in self-driving technology and its parallels with AGI, emphasizing the impressive strides made over the past decade.* 00:03:21 *🤖 Karpathy believes Tesla is ahead of Waymo in self-driving capabilities, attributing Tesla's strength to its scalable deployment of vehicles.* 00:04:03 *🌐 Tesla's approach to sensors entails extensive use of expensive equipment during training rather than in operational deployment, making it a cost-effective strategy.* 00:06:23 *🏭 Karpathy argues Tesla should be viewed as a robotics company rather than just a car manufacturer due to its innovative approach in robotics at scale.* 00:08:45 *🔄 He suggests that the best early applications for humanoid robots will likely emerge in businesses, focusing on tasks like material handling to minimize legal liabilities.* 00:10:37 *🌱 The promise of humanoid robots includes niche applications like leaf blowing, showcasing the practicality of their development.* 00:11:36 *🧠 Karpathy emphasizes the importance of transfer learning between tasks, advocating for a single humanoid platform to optimize efficiency.* 00:15:22 *✨ He highlights the transformative impact of Transformers in neural network architecture, marking a significant leap in research capabilities and scalability.* 00:16:33 *🔍 Karpathy concludes that the bottlenecks in AI research have shifted from architecture to loss functions and datasets.* 17:15 *📊 Data innovation focus is shifting towards synthetic data and enhanced data collection methods due to limitations in available internet data.* 17:57 *🧠 Understanding the brain's reasoning processes could advance AI development, as current models help refine future designs.* 19:10 *⚠️ Ensuring diversity and richness in synthetic data is crucial to prevent model collapse and maintain data quality.* 21:16 *🧬 Transformers may eventually outperform human cognitive abilities in specific tasks but are currently constrained by data limitations.* 22:40 *🤖 Human-AI augmentation is likely to evolve, blurring the lines between technology and human cognition.* 25:30 *🔄 The ecosystem of large AI labs may create a reliance on closed platforms, raising concerns about ownership and control of cognitive enhancements.* 27:09 *📏 Smaller, efficient AI models could be viable, focusing on cognitive core functionalities rather than unnecessary data retention.* 29:55 *🌍 AI education tools could democratize learning, empowering individuals with personalized and scalable access to knowledge.* 32:06 *🎓 AI has the potential to revolutionize education by serving as a personalized interface that adapts to students' needs, enhancing their learning experience.* 34:26 *🏆 Better educational tools and curricula could significantly enhance individual performance, similar to advancements in sports training methods.* 34:54 *🌍 Language adaptability in AI translations is effective, serving as low-hanging fruit for improving education.* 35:50 *🔍 Personal learning context varies; AI should adapt to individual backgrounds and learning rates for effectiveness.* 36:44 *🏛️ Lineage in the AI research community influences knowledge propagation, but it's crucial to minimize its gatekeeping effect.* 37:12 *🤝 Community clustering in AI facilitates knowledge sharing and education, shaping personal motivations and cultural values.* 39:06 *🎓 Education should become more accessible, helping people learn for practical reasons while also shifting towards personal enjoyment.* 40:04 *🧠 Learning should be viewed as mental exercise; the challenge of education can parallel the effort needed in physical fitness.* 41:29 *👩🎓 The upcoming course targets undergraduates in technical fields, emphasizing continuous learning as technology evolves.* 42:26 *📚 Young learners should focus on math, physics, and computer science as foundational skills for effective problem solving and adaptability in the future.*
Joy! By accident, I stumbled onto this podcast and it's brilliant! Andre Karpathy is so likeable, one wants to concentrate to catch the meaning as he speeds through his ideas. He impresses me as a good person and good for the world. Thank you for this!!! I'm a follower, not with understanding of physics but in getting a peek at the edges.
Andrej is amazing. On the topic of education, he is proposing that current technology would allow for AI to be the front-end to the "perfect" course, where a human designs the course, and uses AI to scale themselves. One critical piece, especially for younger learners, is the human connection. Some kids (like the young Andrej's of this world) can be entirely self-driven, but many need to be inspired and guided by another person. I wonder how AI can be used to "super-charge" a teacher, rather than replacing the human at the front.
About this. I saw a startup out of Stanford that leverages gen AI to create course content. The teacher designs the course, topics, etc, & the machine fills in the material and tests. Allows to update courses quickly and potentially dynamically tailor the material to the student’s needs.
One way would be for the AI to take on different characters and pretend to be SpongeBob or whatever teaching long division. Imagine you could pick anyone in the world to be your teacher
3 місяці тому+1
Well given the modern state of the humans doing it and their obvious failures from even older methods.. nah let ai work it.
Andrej is truly a visionary:) I hope with AI technology, we can eliminate mental health disorders and eliminate neurological disorders in humans. Consciousness I believe is in the brain and fixing certain things in the brain like mental health disorders and neurlogical disorders would enhance the mind and enhance intelligence and enhance one’s quality of life. I’d love to have a mind like a Sadhguru, and a mind like a spiritual guru badly in this life:)!
Andrej is a legend! He answers normal questions by saying “this will take you down to a local minimum”. This should now be an expression in the English language. Great interview, thanks for putting this together and letting Andrej share more gems with the rest of us.
They are probably trying to game the algorithm. It's about 1.25x speed so I assume they are trying to make it seem like you watched the video longer than it's actual time. I shoot and edit for a living....no one talks that fast.
@@OceanLlamaMediaActually I had a colleague who was at Stanford as well. His talking is like 2x… so such a speed does exist. Won’t believe it either until I saw one example in real life
I think Andrej will be written into history books for years to come, he’s taught so many and brought open source to the forefront of technological innovation
In every conversation with Andrej Karpathy, I learn a lot of things. In 27:10 I think it is very interesting hypothesis and I believe it would be interesting to test it. Thanks for sharing this.
Yes we need technical people, we also still need storytellers, educators, performers and philosophers to give meaning to what we can create with technology.
42:33 I think Karpathy’s points about what learners should focus on are excellent. I especially appreciate his openness to 20% non-“symbol manipulation” subjects. The only thing I would add is that subjects like writing and history can also strengthen your reasoning abilities. I agree that memorizing facts is not a good use of a learner’s time, but reasoning about cause and effect is. Such reasoning can be practiced outside of “STEM”. I think a lot of folks with STEM background (including myself) have a slight bias against non-STEM subjects, so I feel it’s important to add that perspective. Finally, there are some important “human skills” such as empathy that can arise from subjects like literature, so I appreciate Karpathy’s openness to being well rounded. Thanks for the solid interview!
The woman agreeing to pay to "rent her brain" is the actually frightening part about this "progress". Reminds me a very much of soldiers willing to "shut their conscience" in favor of following someone else's orders (and getting their pay). To make such statements without stopping to imagine the consequences you have to be either dumb or evil, or maybe both.
That comment belongs to the No Priors podcast channel. You meant to say “Thanks for highlighting and discussing the interview.” Credit where credit is due!
Finally… someone giving Andre the opportunity to reinforce the idea that TESLA is leading in this space with the right approach. Funny how neither one of these interviewers want to go there… silence… the bias is so thick and filled with disgust. This discredits their own knowledge because they clearly can’t get out of their own way. They don’t want Tesla to win… so dumb.
much insight into areas you normally don't hear much about especially autonomous driving and robotics (although they are closely related). There are many sources for AI but most of them of low quality, so it's very important to focus on excellent sources to get deeper insights, Andrej is one of them.
Excellent interview as always with Andrej. Just hearing him describe things is inspiring and makes a lot of the optimistic viewpoints on the future come into sharper focus
Thank you for the great talk. Thanks to Andrej’s lecture, I learned so much about how tokenizer works and built gpt2 from scratch. Regarding the distillation, I think it kinda resembles how proverbs works when transferring the knowledge from experiences in a short compressed manner. I do hope the cognitive part will reach really small parameters and embedded into our handy devices, and democratize AI for all.
Andrej is one of my favourite AI legends. His love for AI and teaching is what makes sets him apart. The way he spews out Python code is like Python flows in his blood.
I love the teaching models idea that caters to the specific person, their age, demographic, skills, languages, handicaps, and what not. I would love to be taught the things that are important to me by a perfect teacher.
When everyone sends their kids to school in self driving cars America will realise the mistake it made in being so car dependent. Congestion will be insane. The new equilibrium of inconvenience with people having breakfast or working/sleeping on the commute, will mean that the wealthiest might prefer to live in a walkable city centre free of cars.
12:03 An arm on a platform, on wheels, sounds like a great way to minimize cost. It would still work great for many simple and repetitive tasks, that might not warrant the expense of a full bipedal bot. Make the base heavy and allow it raise and lower the arm. The sophisticated hands that Optimus brings aren’t always necessary either. For example, at a BBQ restaurant, towards the last hours of a cook it is better to add smaller quantities of wood repeatedly, so as to target higher temp and airflow. Airflow helps evaporation, which helps the exterior parts from getting overdone. Very fussy work that could be automated.
From a consumer perspective, I’d be more comfortable with arms on wheels in my home rather than a humanoid. I know it’s irrational, but I have this fear of waking up in the middle of the night and my humanoid robot standing in my doorway looking at me. Just creeps me out lol.
Insights By "YouSum Live" 00:00:05 AI's impact on education and self-driving technology 00:01:04 Self-driving cars have reached significant capabilities 00:02:30 Tesla's software problem is easier than Waymo's 00:04:15 Tesla's approach uses cameras for cost efficiency 00:07:12 Humanoid robots share technology with self-driving cars 00:07:55 Robotics at scale is Tesla's true identity 00:09:01 Humanoid robots will start in B2B applications 00:31:04 Education should empower people, not replace them 00:35:00 AI can provide personalized learning experiences 00:35:50 Future education will require adaptability and accessibility 00:43:09 Math and physics are essential for critical thinking Insights By "YouSum Live"
Sarah, you just casually drop that you have a Unitree G1 robot and leave it there??? How is it programmable? You've got to do an unboxing video or at least explain what it's all about!
34:32 - This might be a little recency bias. The guardian did a nice article on 100 yard dash for the last 100 years and surprisingly the numbers were very similar. This analogy may be off between physical limits and the brain's limits however.
Great discussion. I was surprised at his comment at the end. If education is for you rather than for money, I’m not sure why you would choose math and CS first. I would think that curricula that catalyze understanding of yourself and the world would be far more important.
Andrej, if you're reading this: You mentioned using the older hardware heavy lidar approach to use as training data for the pixels. Why not just train on video from 99th percentile human drivers?
That is Tesla’s strategy. As videos were collected, human raters categorized driving on a 5-point scale. Training occurs on the “5” videos. Separate training occurs on crash videos - system practices crash avoidance…
Regarding the cognition question - 20:55 - Can recommend: "Maturana - From being to doing - The Origins of the Biology of Cognition". Memory, memory, memory :-)
Transfer learning on simulated data does really great on mathematically based models, then the entropy or somewhat clean real world data whether it is sensor data or encoded data works. I’ve trained models 100% translated data that transferred to real world data with over 90% accuracy and balanced error. It was really impressive.
hehe excellent interview and I really like Andrej .. this being said, regarding Canada, I live there and I guess, it's true, most people don't care about startups, me included. I mean, I had one for some time but I did not really care, I've enough money already and I prefer to keep my energy and time for my kids and hobbies. I sold my shares and while my partner might become richer than me at some point, he works like crazy and does not look very happy. Also I could have moved to the Bay Area like some of my colleagues. But I look at their linkedIn and it seems pretty boring so aside from the weather, I'm not sure I would enjoy the SF startup mentality, I could not care less about status, coworkers egos and shareholders profits for example but it does not stop me from investing by buying stocks.
Will smaller models have disadvantages over larger models? I have seen gpt4o struggling on problems which turbo models are more accurate on. Does moving from larger models to smaller models have tradeoff with accuracies
Is the transformer the correct final storage shape of the knowledge entropy? Its too expensive. Retraining LLMs is a waste of power and compute resources. We need to use compute to categorize, classify, gather, abstract, delta compress, and many other steps to create dense storage, just like huffman did 40 years ago. 27:40
education should not be about knowledge but about methods, processes, inspiration, exercising intelligence instead of raw memory. Definitely add multiple languages and music to curriculum, as these change the shape of your mind and thinking skills.
Key prediction: in a post AGI society education will be entertainment. I concur. We already see people like me educating themselves for fun, especially autodidacts! Anyone who understands AI realizes the possibility of a Utopia, if governments and AI companies align to solve ethical and cybersecurity problems.
10:05 - I thought the same thing the moment I saw that "Leaf Blower Challenge". Just tell a humanoid robot to do it - done. And then something else occurred to me: why would you need a dishwasher or a washing machine if you have a humanoid robot? It won't substitute a fridge, but some automation we take for granted wouldn't be necessary when you have a universal robot. A lawnmower? Just get some grass shears :D
Quietly remove leaves? Use a "rake" if they are on grass/sand/soil, or a "broom", if they are on other types of ground. Picking up leaves individually would be cost (time and energy/electricity spent on the task) prohibitive.
now is the time to do a national or world wide questionnaire on problem solving to get to agi. at 62 im facinated by all this machine learning, llms. can i learn enough about this to create my own useful model.
I like this idea of silent collapse - I'm reminded of WestWorld and the shutdown the characters would go into (and of their personality training by means of inner dialog).
The humanoid form is not optimal for picking up the leaves from the street @Andrej. This problem is much better suited for a Unitree quadruped robot. You just stick a robotic arm on the top and ta-da. It will be much quieter and easier because it has fewer degrees of freedom and the humanoid would have to constantly bend down. Also cheaper and faster lol. The quadruped form is actually the way to go here! You can put it on wheels too so it's more stable.
28:18 the guy is asking the billion dollar question, and it should be very embarrassing to Karpathy (and all ML researchers) that they are utterly unable to provide any reasonable answer. Or more likely, they dare not look into it and speak about it because it could lead to an emperor's new clothes kind of situation.
![[1hr Talk] Intro to Large Language Models](/img/n.gif)
I generally listen to podcasts at 1.5 speed. Andrej is one of the few people whom I have to play at normal speed to keep up with.
That Adderall is kicking good! 😂
I was thinking about slowing it down
Good luck then trying to listn do Marc Andreessen
@@Sofiazinha-j2w it's not just about their talking speed, it's more about the depth of their knowledge and how well they articulate it. (*Never had the need to slow it down for Marc)
They already speed up the playback before uploading the video. So ur good
IMO... Listening to Andrej Karpathy talk about innovation, work on deep learning research (e.g. assembling LayerNorms, Skip connections, and other parts to create a trainable "tissue" called transformers) and current/future problems is a breath of fresh air for young researchers like myself.
I want more talk about stuff not related to first principles, get this karpathy guy out of here!
@@thelazyanalyst3723he isn’t a researcher anymore just to be clear bud
Karpathy is a truly great human being.
I concur.
do YOU guys know HIM
❤
he's a very good engineer
how does that qualify him to be a "great human being" wtf
is he known for charity or something
genius first, then altruist, benevolent, (a lot) empathetic, etc.
🎯 Key points for quick navigation:
00:02:00 *🚗 Andrej Karpathy discusses his work in self-driving technology and its parallels with AGI, emphasizing the impressive strides made over the past decade.*
00:03:21 *🤖 Karpathy believes Tesla is ahead of Waymo in self-driving capabilities, attributing Tesla's strength to its scalable deployment of vehicles.*
00:04:03 *🌐 Tesla's approach to sensors entails extensive use of expensive equipment during training rather than in operational deployment, making it a cost-effective strategy.*
00:06:23 *🏭 Karpathy argues Tesla should be viewed as a robotics company rather than just a car manufacturer due to its innovative approach in robotics at scale.*
00:08:45 *🔄 He suggests that the best early applications for humanoid robots will likely emerge in businesses, focusing on tasks like material handling to minimize legal liabilities.*
00:10:37 *🌱 The promise of humanoid robots includes niche applications like leaf blowing, showcasing the practicality of their development.*
00:11:36 *🧠 Karpathy emphasizes the importance of transfer learning between tasks, advocating for a single humanoid platform to optimize efficiency.*
00:15:22 *✨ He highlights the transformative impact of Transformers in neural network architecture, marking a significant leap in research capabilities and scalability.*
00:16:33 *🔍 Karpathy concludes that the bottlenecks in AI research have shifted from architecture to loss functions and datasets.*
17:15 *📊 Data innovation focus is shifting towards synthetic data and enhanced data collection methods due to limitations in available internet data.*
17:57 *🧠 Understanding the brain's reasoning processes could advance AI development, as current models help refine future designs.*
19:10 *⚠️ Ensuring diversity and richness in synthetic data is crucial to prevent model collapse and maintain data quality.*
21:16 *🧬 Transformers may eventually outperform human cognitive abilities in specific tasks but are currently constrained by data limitations.*
22:40 *🤖 Human-AI augmentation is likely to evolve, blurring the lines between technology and human cognition.*
25:30 *🔄 The ecosystem of large AI labs may create a reliance on closed platforms, raising concerns about ownership and control of cognitive enhancements.*
27:09 *📏 Smaller, efficient AI models could be viable, focusing on cognitive core functionalities rather than unnecessary data retention.*
29:55 *🌍 AI education tools could democratize learning, empowering individuals with personalized and scalable access to knowledge.*
32:06 *🎓 AI has the potential to revolutionize education by serving as a personalized interface that adapts to students' needs, enhancing their learning experience.*
34:26 *🏆 Better educational tools and curricula could significantly enhance individual performance, similar to advancements in sports training methods.*
34:54 *🌍 Language adaptability in AI translations is effective, serving as low-hanging fruit for improving education.*
35:50 *🔍 Personal learning context varies; AI should adapt to individual backgrounds and learning rates for effectiveness.*
36:44 *🏛️ Lineage in the AI research community influences knowledge propagation, but it's crucial to minimize its gatekeeping effect.*
37:12 *🤝 Community clustering in AI facilitates knowledge sharing and education, shaping personal motivations and cultural values.*
39:06 *🎓 Education should become more accessible, helping people learn for practical reasons while also shifting towards personal enjoyment.*
40:04 *🧠 Learning should be viewed as mental exercise; the challenge of education can parallel the effort needed in physical fitness.*
41:29 *👩🎓 The upcoming course targets undergraduates in technical fields, emphasizing continuous learning as technology evolves.*
42:26 *📚 Young learners should focus on math, physics, and computer science as foundational skills for effective problem solving and adaptability in the future.*
Nice map of the interview, thanks! Would you mind sharing the process you followed to prepare it?
Joy! By accident, I stumbled onto this podcast and it's brilliant! Andre Karpathy is so likeable, one wants to concentrate to catch the meaning as he speeds through his ideas. He impresses me as a good person and good for the world. Thank you for this!!! I'm a follower, not with understanding of physics but in getting a peek at the edges.
Just discovered this podcast. What a gem . Now I need to catch up with 80+ episodes 😆
thank you Andrej for explaining technical things that a layperson such as myself can understand, or at least get a sense of😃
Andrej is amazing. On the topic of education, he is proposing that current technology would allow for AI to be the front-end to the "perfect" course, where a human designs the course, and uses AI to scale themselves. One critical piece, especially for younger learners, is the human connection. Some kids (like the young Andrej's of this world) can be entirely self-driven, but many need to be inspired and guided by another person. I wonder how AI can be used to "super-charge" a teacher, rather than replacing the human at the front.
About this. I saw a startup out of Stanford that leverages gen AI to create course content. The teacher designs the course, topics, etc, & the machine fills in the material and tests. Allows to update courses quickly and potentially dynamically tailor the material to the student’s needs.
One way would be for the AI to take on different characters and pretend to be SpongeBob or whatever teaching long division. Imagine you could pick anyone in the world to be your teacher
Well given the modern state of the humans doing it and their obvious failures from even older methods.. nah let ai work it.
"but many need to be inspired and guided by another person."
This, as with literally everything, will change.
Andrej is truly a visionary:) I hope with AI technology, we can eliminate mental health disorders and eliminate neurological disorders in humans. Consciousness I believe is in the brain and fixing certain things in the brain like mental health disorders and neurlogical disorders would enhance the mind and enhance intelligence and enhance one’s quality of life. I’d love to have a mind like a Sadhguru, and a mind like a spiritual guru badly in this life:)!
Andrej is a legend! He answers normal questions by saying “this will take you down to a local minimum”. This should now be an expression in the English language.
Great interview, thanks for putting this together and letting Andrej share more gems with the rest of us.
I had to check my playback speed settings, because I thought I’m on x1.5
Glad we didn’t have to follow at the speed Dr. Karpathy’s brain races!❤
They are probably trying to game the algorithm. It's about 1.25x speed so I assume they are trying to make it seem like you watched the video longer than it's actual time.
I shoot and edit for a living....no one talks that fast.
Same! It’s annoying. They’ve totally sped this up.
@@OceanLlamaMediaActually I had a colleague who was at Stanford as well. His talking is like 2x… so such a speed does exist. Won’t believe it either until I saw one example in real life
@@FrunkensteinVonZipperneck
God is so unfair...some people are so smart and some are so dumb. and kaparthy is also tall and gl
great stuff! Glad to see Andrej very happy doing his own things rather than being limited at OpenAI
As always with Andrej, awesome insights
This guy is an actual genius. You can tell just by the expresses his ideas and is so forward thinking
❤
Best interview ever relating to AI, FSD and Optimus.
That was such an amazing episode.
I think Andrej will be written into history books for years to come, he’s taught so many and brought open source to the forefront of technological innovation
legendary guest
most underrated new pod in the world imo.
get ilya, elon, geohot, friend founder, sama next 💙
In every conversation with Andrej Karpathy, I learn a lot of things. In 27:10 I think it is very interesting hypothesis and I believe it would be interesting to test it. Thanks for sharing this.
Yes we need technical people, we also still need storytellers, educators, performers and philosophers to give meaning to what we can create with technology.
I want andrej’s courses, but I want him on podcasts everyday 😂🖤
42:33 I think Karpathy’s points about what learners should focus on are excellent. I especially appreciate his openness to 20% non-“symbol manipulation” subjects.
The only thing I would add is that subjects like writing and history can also strengthen your reasoning abilities. I agree that memorizing facts is not a good use of a learner’s time, but reasoning about cause and effect is. Such reasoning can be practiced outside of “STEM”.
I think a lot of folks with STEM background (including myself) have a slight bias against non-STEM subjects, so I feel it’s important to add that perspective.
Finally, there are some important “human skills” such as empathy that can arise from subjects like literature, so I appreciate Karpathy’s openness to being well rounded.
Thanks for the solid interview!
The woman agreeing to pay to "rent her brain" is the actually frightening part about this "progress". Reminds me a very much of soldiers willing to "shut their conscience" in favor of following someone else's orders (and getting their pay). To make such statements without stopping to imagine the consequences you have to be either dumb or evil, or maybe both.
This man is one of the most important technologists of our time. Thanks for the interview!
That comment belongs to the No Priors podcast channel. You meant to say “Thanks for highlighting and discussing the interview.” Credit where credit is due!
Finally Andrej on Tesla . Been waiting for this a LONG time 😊🙏
Finally… someone giving Andre the opportunity to reinforce the idea that TESLA is leading in this space with the right approach.
Funny how neither one of these interviewers want to go there… silence… the bias is so thick and filled with disgust. This discredits their own knowledge because they clearly can’t get out of their own way. They don’t want Tesla to win… so dumb.
He joined long ago
@@developit1152he joined what?
@@michaelholmes8848 Tesla
what?
Well done guys! You are the coolest!
Loved that. Andrej is such a gentleman with sublime knowledge.
Thank you for doing this pod!
much insight into areas you normally don't hear much about especially autonomous driving and robotics (although they are closely related). There are many sources for AI but most of them of low quality, so it's very important to focus on excellent sources to get deeper insights, Andrej is one of them.
Excellent interview as always with Andrej. Just hearing him describe things is inspiring and makes a lot of the optimistic viewpoints on the future come into sharper focus
So good, always fun to listen to Andrej
AI being the Front End for a backend heavily technical course. That´s it. I am home now. Thanks GOAT.
Thank you for the great talk. Thanks to Andrej’s lecture, I learned so much about how tokenizer works and built gpt2 from scratch. Regarding the distillation, I think it kinda resembles how proverbs works when transferring the knowledge from experiences in a short compressed manner. I do hope the cognitive part will reach really small parameters and embedded into our handy devices, and democratize AI for all.
Accelerando! Love that book! So great! OMG I love this channel you people are so freaking cool!
shes a great interviewer and Andrej is all around a great speaker and has great ideas
what an inspiring episode
The ABSOLUTE last person I expected to find on an AI technical interview video. Love your Craig Ferguson compilation work
Andrej Karpathy is a wonderful human being!
Andrej is one of my favourite AI legends. His love for AI and teaching is what makes sets him apart. The way he spews out Python code is like Python flows in his blood.
very insightful and the hosts are great.
I love the teaching models idea that caters to the specific person, their age, demographic, skills, languages, handicaps, and what not. I would love to be taught the things that are important to me by a perfect teacher.
When folks at Tesla are sending their children to school in self driving cars that will be a sign of confidence.
Agreed. That authentic assessment will define the critical inflection point.
Confidence from the regulators
When everyone sends their kids to school in self driving cars America will realise the mistake it made in being so car dependent. Congestion will be insane. The new equilibrium of inconvenience with people having breakfast or working/sleeping on the commute, will mean that the wealthiest might prefer to live in a walkable city centre free of cars.
This video is so good. I want everything karpathy says to turn true.
Andrej is a rockstar. Love everything he says and does
Great talk with Andrej!!
Such intelligent questions asked by the interviewers. So much better than the usual surface level stuff you see.
Andrej seems to speak in 1.75X
I usually whatch videos on 2X or 1.75X but this one I had to watch at 1X
I had to check my playback speed to be sure it was in normal 😂. I am sure he’s mouth was trying to catchup with his brain
@@babatundeadewole2488 exactly. I did check it twice
They are speeding the video up. He doesn't actually talk this fast in his own videos.
ua-cam.com/video/l8pRSuU81PU/v-deo.html
Thank you, great show!
Awesome interview and nice to see knowledgeable hosts for once
u didnt see the Ilya episode?
this Karpathy guy is a king. like if there ever was or ever will be a king. it's that guy.
Great interview. Subscribed!
Amazing interview. Thank you!
Great show
12:03 An arm on a platform, on wheels, sounds like a great way to minimize cost. It would still work great for many simple and repetitive tasks, that might not warrant the expense of a full bipedal bot. Make the base heavy and allow it raise and lower the arm. The sophisticated hands that Optimus brings aren’t always necessary either.
For example, at a BBQ restaurant, towards the last hours of a cook it is better to add smaller quantities of wood repeatedly, so as to target higher temp and airflow. Airflow helps evaporation, which helps the exterior parts from getting overdone. Very fussy work that could be automated.
From a consumer perspective, I’d be more comfortable with arms on wheels in my home rather than a humanoid. I know it’s irrational, but I have this fear of waking up in the middle of the night and my humanoid robot standing in my doorway looking at me. Just creeps me out lol.
Great episode!
Andrej is our generation's Richard Feynman.
Nice!!! Waiting for his course!!!
Dr. Karpathy developed Stanford’s first AI curriculum❤
I like Andrej Karpathy❤
Great show as always!
Wow Eureka is a fascinating company
awesome interview!
Insights By "YouSum Live"
00:00:05 AI's impact on education and self-driving technology
00:01:04 Self-driving cars have reached significant capabilities
00:02:30 Tesla's software problem is easier than Waymo's
00:04:15 Tesla's approach uses cameras for cost efficiency
00:07:12 Humanoid robots share technology with self-driving cars
00:07:55 Robotics at scale is Tesla's true identity
00:09:01 Humanoid robots will start in B2B applications
00:31:04 Education should empower people, not replace them
00:35:00 AI can provide personalized learning experiences
00:35:50 Future education will require adaptability and accessibility
00:43:09 Math and physics are essential for critical thinking
Insights By "YouSum Live"
Love how he just tells the optimal training recipes under the analogy of human learning and education
Sarah, you just casually drop that you have a Unitree G1 robot and leave it there??? How is it programmable? You've got to do an unboxing video or at least explain what it's all about!
34:32 - This might be a little recency bias. The guardian did a nice article on 100 yard dash for the last 100 years and surprisingly the numbers were very similar. This analogy may be off between physical limits and the brain's limits however.
Great discussion. I was surprised at his comment at the end. If education is for you rather than for money, I’m not sure why you would choose math and CS first. I would think that curricula that catalyze understanding of yourself and the world would be far more important.
AI is a subset of psychology - the science of knowing oneself.
Love his take on Toronto too lol
Pretty good content👍 thank for sharing!
Ohh man, Is the videos is on 2x?
Andrey is shockingly wise. This field has a lot bright minds, but this guy is just watching otgers from above.
Andrej, if you're reading this:
You mentioned using the older hardware heavy lidar approach to use as training data for the pixels. Why not just train on video from 99th percentile human drivers?
That is Tesla’s strategy. As videos were collected, human raters categorized driving on a 5-point scale. Training occurs on the “5” videos. Separate training occurs on crash videos - system practices crash avoidance…
Very insightful 🎉
Great channel! New subscriber.
Me too
Regarding the cognition question - 20:55 - Can recommend: "Maturana - From being to doing - The Origins of the Biology of Cognition". Memory, memory, memory :-)
Great guy to listen to, articulates complex subject well... At 0.75 speed for me.
Transfer learning on simulated data does really great on mathematically based models, then the entropy or somewhat clean real world data whether it is sensor data or encoded data works. I’ve trained models 100% translated data that transferred to real world data with over 90% accuracy and balanced error. It was really impressive.
Sensei Karpathy🙇🏻♂
The only man on earth I have to slow down to 0.75x just to keep up.
hehe excellent interview and I really like Andrej .. this being said, regarding Canada, I live there and I guess, it's true, most people don't care about startups, me included. I mean, I had one for some time but I did not really care, I've enough money already and I prefer to keep my energy and time for my kids and hobbies. I sold my shares and while my partner might become richer than me at some point, he works like crazy and does not look very happy. Also I could have moved to the Bay Area like some of my colleagues. But I look at their linkedIn and it seems pretty boring so aside from the weather, I'm not sure I would enjoy the SF startup mentality, I could not care less about status, coworkers egos and shareholders profits for example but it does not stop me from investing by buying stocks.
Will smaller models have disadvantages over larger models? I have seen gpt4o struggling on problems which turbo models are more accurate on.
Does moving from larger models to smaller models have tradeoff with accuracies
Cool interview.
University of Toronto is a very under rated school. Their grads are mostly humble low key but highly skilled.
🇧🇷🇧🇷🇧🇷🇧🇷👏🏻, Great interview guys, thanks for sharing!
Is the transformer the correct final storage shape of the knowledge entropy? Its too expensive. Retraining LLMs is a waste of power and compute resources. We need to use compute to categorize, classify, gather, abstract, delta compress, and many other steps to create dense storage, just like huffman did 40 years ago. 27:40
education should not be about knowledge but about methods, processes, inspiration, exercising intelligence instead of raw memory.
Definitely add multiple languages and music to curriculum, as these change the shape of your mind and thinking skills.
Go Team (Augmented) Human! 😁
Key prediction: in a post AGI society education will be entertainment. I concur. We already see people like me educating themselves for fun, especially autodidacts! Anyone who understands AI realizes the possibility of a Utopia, if governments and AI companies align to solve ethical and cybersecurity problems.
10:05 - I thought the same thing the moment I saw that "Leaf Blower Challenge". Just tell a humanoid robot to do it - done. And then something else occurred to me: why would you need a dishwasher or a washing machine if you have a humanoid robot? It won't substitute a fridge, but some automation we take for granted wouldn't be necessary when you have a universal robot. A lawnmower? Just get some grass shears :D
Quietly remove leaves? Use a "rake" if they are on grass/sand/soil, or a "broom", if they are on other types of ground. Picking up leaves individually would be cost (time and energy/electricity spent on the task) prohibitive.
my brain hurts, its too fast, dense, always on point, no BS
U don't have to understand everything
Just skim through it
Watch it on slow playback?
❤
Inspiring.
thanks for mentioning The Diamond Age!
This guy is very smart, he will go places
go places?
Already has! And he's helping everyone go further with his education efforts!!
Great episode!
1. Who is Nate Friedman (sp?) and what is the "Leaf Blower Challenge"?
2. What IS the real price of a G1? Website says $16k.
Am I high or is the playback speed hiked up on this 😂
A.K. thinks so fast, his lips always race!
He talks so fast😂
His name is Car Pathy and he designs AI so the car goes on the right path, I can't make this up people!
It's a fun talk from Andrej, but sounds very speculative.
now is the time to do a national or world wide questionnaire on problem solving to get to agi. at 62 im facinated by all this machine learning, llms. can i learn enough about this to create my own useful model.
I like this idea of silent collapse - I'm reminded of WestWorld and the shutdown the characters would go into (and of their personality training by means of inner dialog).
"The demo is near but the product is far." Ha ha! Shots fired! Take cover Sama!
The best speaking speed I like.
The humanoid form is not optimal for picking up the leaves from the street @Andrej. This problem is much better suited for a Unitree quadruped robot. You just stick a robotic arm on the top and ta-da. It will be much quieter and easier because it has fewer degrees of freedom and the humanoid would have to constantly bend down. Also cheaper and faster lol. The quadruped form is actually the way to go here! You can put it on wheels too so it's more stable.
28:18 the guy is asking the billion dollar question, and it should be very embarrassing to Karpathy (and all ML researchers) that they are utterly unable to provide any reasonable answer. Or more likely, they dare not look into it and speak about it because it could lead to an emperor's new clothes kind of situation.
Amen, brother!