David Silver: AlphaGo, AlphaZero, and Deep Reinforcement Learning | Lex Fridman Podcast #86
Вставка
- Опубліковано 4 тра 2024
- David Silver leads the reinforcement learning research group at DeepMind and was lead researcher on AlphaGo, AlphaZero and co-lead on AlphaStar, and MuZero and lot of important work in reinforcement learning.
Support this podcast by signing up with these sponsors:
- MasterClass: masterclass.com/lex
- Cash App - use code "LexPodcast" and download:
- Cash App (App Store): apple.co/2sPrUHe
- Cash App (Google Play): bit.ly/2MlvP5w
EPISODE LINKS:
Reinforcement learning (book): amzn.to/2Jwp5zG
PODCAST INFO:
Podcast website:
lexfridman.com/podcast
Apple Podcasts:
apple.co/2lwqZIr
Spotify:
spoti.fi/2nEwCF8
RSS:
lexfridman.com/feed/podcast/
Full episodes playlist:
• Lex Fridman Podcast
Clips playlist:
• Lex Fridman Podcast Clips
OUTLINE:
0:00 - Introduction
4:09 - First program
11:11 - AlphaGo
21:42 - Rule of the game of Go
25:37 - Reinforcement learning: personal journey
30:15 - What is reinforcement learning?
43:51 - AlphaGo (continued)
53:40 - Supervised learning and self play in AlphaGo
1:06:12 - Lee Sedol retirement from Go play
1:08:57 - Garry Kasparov
1:14:10 - Alpha Zero and self play
1:31:29 - Creativity in AlphaZero
1:35:21 - AlphaZero applications
1:37:59 - Reward functions
1:40:51 - Meaning of life
CONNECT:
- Subscribe to this UA-cam channel
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridmanpage
- Instagram: / lexfridman
- Medium: / lexfridman
- Support on Patreon: / lexfridman - Наука та технологія
I really enjoyed this conversation with David. Here's the outline:
0:00 - Introduction
4:09 - First program
11:11 - AlphaGo
21:42 - Rule of the game of Go
25:37 - Reinforcement learning: personal journey
30:15 - What is reinforcement learning?
43:51 - AlphaGo (continued)
53:40 - Supervised learning and self play in AlphaGo
1:06:12 - Lee Sedol retirement from Go play
1:08:57 - Garry Kasparov
1:14:10 - Alpha Zero and self play
1:31:29 - Creativity in AlphaZero
1:35:21 - AlphaZero applications
1:37:59 - Reward functions
1:40:51 - Meaning of life
OMG THANK YOU
Thank you very much Lex 🙏
Thanks
Please invite Humberto Maturana: He had develop theories on human intelligence, consciousness and understanding. He is in his 90s, we could lose his takes on artificial intelligence
Bring David Deutsch please! :)
"He'll be remembered as the last person to beat AlphaGo"
man!!
,,, kudos n respect on that comment! ... greetINX from s.lem jr ... .. . ...............
Seeing this after the AlphaGo doc!
Watching the documentary before watching this interview definitely adds value. ua-cam.com/video/WXuK6gekU1Y/v-deo.html
As have I! I was searching of an Alpha Zero doc. This is where I got so far. Not disappointed at all!
Yes came here directly after the Doc as well. Had never heard of GO! prior to 3hrs a go. Indelibly registered and imprinted now :D
Same
maap no need to capitalize and exclaim, any more than you’d write CHESS!
His answers are so articulate!
THIS IS THE ONE I'VE BEEN WAITING FOR!
@@mikhailfranco dude, thanks 🙌
Again, Mr. Fridman, THANK YOU for keeping this going, especially now. When I need to get my mind off the current world situation I come here. Your talks always take me to a better place. Thank you. Be safe. Stay healthy.
I can't describe or express how valuable this interview is for understanding what's going to happen in the future
Amazing, this conversations are so meaningful to the future of humanity that they should be broadcasted on national television. That way children would more easily find meaningful role models and access to the type of insightful ideas that give birth to passions and eventually discoveries.
I am very happy to see that 3.22M people are watching this channel.
This is a banger of an interview. AlphaZero is a harbinger of the future
Thanks for making this podcast. David Silver chooses his words very well, his stories are very clear and inspiring! I could have listened much longer ;-)
Discovery is a joy. Discovering the existence of David Silver and his amazing way of thinking is pure gold. Thank you Lex.
3 years later I am here... Latest AI developments makes me ask for a second round with David Silver. Thanks for sharing 👍🏼
Awesome conversation, David is incredibly interesting and humble also amazing questions from Lex. Thanks to both of you for making it.
you just gotta love David Silver and his ideas, thoughts and accent
Incredible podcast, probably my favourite! It would be incredible to have a second part!
Wow! This was an incredibly insightful and inspiring conversation. Thank you Lex, David, and your teams for this.
Outlining the episode is the MOST awesome and thoughtful thing foru2have done...
This interview is LEGENDARY!... watching it for the second time. Definitely in the top 3 on youtube!
I watched Alpha Go vs. Lee sedol tournament documentary Deepmind recently uploaded, and I cried. It was so inspiring, touching and beautiful. Thanks very much Lex for this podcast.
Thank you for Lex and David! Very interesting and inspiring conversation about first principles of Artificial Intelligence.
Thank you both!
It was, again, an awesome conversation.
Lex,
It is very clear that you love what you do. It totally shows.
You are always super prepared and well engaged with your guests.
Yours has become my absolutely favorite podcast. Listening to a 2 hr podcast of yours is as intellectually fulfilling as reading a 400 page incredible book.
This is a really great interview and very enlightening. Thanks for all of your hard work bringing this stuff to us. Keep up the good work.
Mind teased, tantalized, and finally thrown into a tizzy. Love every one of your interviews Lex. All I want to do is watch them to get inspired to think in new ways. THANKS MAN!
I love the content you put out man! It's always interesting, always paradigm challenging, calm, informed, you! Thanks!
1:40:51 : One of the best answers for the purpose and meaning of life I have heard so far. Incredible!
Thank you for another enlightening, exploratory, and meaningful conversation that pushes us towards self-questioning and, one hopes, self-understanding.
I love your guests and the way you carry the conversation brother! Great job, love your channel.
Man, David Silver is so incredibly humble...
Awesome interview. I start jumping around with excitement. Get so eager to learn more!
Thank you so much LF! Great job.
I can ignore everyone else but David Silver talking about AI. His lectures and courses taught me RL.
Many thanks for sharing this amazing interview!
Trying to reproduce the MCTS results on some other tasks. After several weeks of struggling, I learned that David Silver is really great in a sense that he foresee the future of deep learning research -- computational power really matters.
its beautiful to see a man that lives his passion. a man that is what he is creating.
Thanks for Boss content empowering people, many young people enjoying this content and in my opinion, such a treasure it is, the exponential tune to your tone.
Very proud of my old university - University of Alberta. Dr. Silver got his PhD there under Richard Sutton. Great interview. Was looking forward to this one.
Many academics are terrible at explaining their domain of expertise. David is a quality academic and has remained grounded enough to explain himself to normal folk like me. Well done.
Thank you!! Been looking forward to this.
Crazy Lex.. I just went down the alpha learning machine rabbit hole this week. I watched the documentary on alphago, which was fascinating. I also watched the matches between the pro starcraft players and alphastar, which was even more fascinating (partially because I'm familiar with the game). I wonder in this sphere, how far a deep learning machine like this can go. This podcast was the icing on the cake at the bottom of the rabbithole, thanks brother!
This interview was so good it brought a tear to my eye!
My Saturday blockbuster, thanks Lex. David is a cool dude, have to get Demis in now :)
I love how the wall and window are decorated to resemble a go board
Love David Silver's lectures on RL
Brilliant interview. Articulate and like yourself, I believe AlphaGo was a tipping point for the progress of humanity.
Fantastic one!! So many cool ideas in there!! Thanks Lex 🤘🏽
Thank you for this amazing discussion!
thank you again lex, another phenomenal interview, i cannot get enough of this wonderful channel!
The great conversation! Now I finally understand how alphaGo and alpha Zero were created.
David is an amazing being.
Thank you, one of the most interesting talks in a long time!
Thank you Lex, Great convo.
Excellent podcast, thank you
Wow, very insightful, nice to get our minds off of the pandemic and look to a bright future. Incredible potential behind DRL!
Thanks Lex! Even bigger greatness is coming your way!! Cheers! Stay safe!
Oh man! That meaning of life interpretation! I think I'm gonna click this 1:41:20 every night before sleep from now on.
Thank you Lex for making this possible! ❤️
I initially cringed a little when Lex decided to "go there" with the meaning of life question but pshew! Silver gave a great answer.
sabelch yes that answer was very impressive and I think demonstrated his capacity of deep thinking
I was laughing to myself and thinking: "All he needs to do now is ask him the meaning of life question". And then he did!
Indeed, probably David's comment regarding the meaning of life was by far the most philosophically meaningful I have ever come across.
there's a book called 'the fabrics of reality'
Good to hear the logic based programming language PROLOG mentioned.
Hey lex, really interesting episode. A guest I think you should have on your podcast is Leo Gura. His work is more particularly focused on the nature of consciousness and he is for me one of the most insightful people I have ever listened to.
Thanks for putting the ads in the beginning !! It's way better than getting your concentration broke mid interview
David is adorable, I have watched his RL Course 3-4o times. Brilliant guy and funny too
I am struck by how small the audience is for this astonishing talk. It is so important that it should number in the millions, even billions.
David Silver is a real legend
This is an instant like from me :)! Many thanks Lex!
his course on youtube is amazing
This is the best of all episodes and I know I am biased. Thanks Lex.
Well done. Its great how you went into the deep background at the end there/
This was the AI interview I've been waiting for - it did deliver. It could have been a bit longer and included the protein folding work, though. Perhaps that's ongoing and still a competitive area. There is a certain clarity of articulation from the guests I enjoy most - reminds me of Jeff Hawkins. Also a sense of practical application.
Pala
They figured it out
@@Jacob-sb3su they?
This interview is eye opening👍👍
Thank you lex David you seem like a real gamer very competitive. Great podcast
Great stuff, guys! Keep up the hustle
What a fantastic conversation!!!
Mate thank you for your videos. your channel is great.
Get Demis on here please!
Amen
Yes!
Yes please Lex Demi’s would be awesome 😎
You, Sir, are a gentleman and a scholar.
I learnt about New dimension of thinking and understanding things.
I've been taking his rl lectures currently.Thanks
Absolutely amazing.
6 months ago I didn’t even know who Lex was, now I can’t get enough of his podcasts. The powers of the internet. I hope he does become a billionaire.
Hey man, awesome interviews! You seems to be a really good person. Thank you for what you are doing.
It's funny I got chance to watch it today again. Now this interview.
Such an inspiring conversation, as a phd candidate who works on deep RL, I am quite motivated to try even harder! Thanks for your efforts Lex!
such an annoying comment, as someone who hates humble bragger, I am quite motivated to downvote your comment! Thanks mr poo on road!
@@smegmaprince314??? He just said he's inspired because he's working toward entering the same field as the podcast guest. Don't be dumb and weird.
Really enjoyed this one
This talk is so inspiring.
YES DEEPMIND!!! (I had decided to write in all caps when I saw the thumbnail)
Man, David Silver is such a genius! I've enjoyed the interview so much.
I wouldn't say Lex interview policy can be considerd as optimal yet, but the story you create through your questions, the way you try to go to the essence when you close your eyes and just the way you are make it be really close. If you read this, thank you
What a cool view of the meaning of life, it was enlightning!
Very enlightening thanks.
Man i can’t thank you enough ❤️
Those who don’t have sophisticated backgrounds in Programming can really appreciate the way you relate what the computers are doing and capable of doing to the romantic human narratives
amazing episode
changing the world, by bringing us the people who are changing them :) Thanks Lex! you rule :)
I must say, one of the best podcasts. Thanks, Lex and David
Haven't watched yet, just settling in for it but I really wanted to say something. Yay!
Greatly enjoyed it, and I have a feeling there are more interviews with Deepmind team and I am sooooo stoked. Be safe & have fun.
Anyone else get excited by Deepmind's latest "muzero" algorithm that David discussed, starting from about 1:28:00 into the video? Supposedly a new algorithm that is able to figure-out the rules and constraints of the environment by itself. I'd love to hear more in depth discussions about Muzero's capabilities in future talks with Deepmind's finest 😎!
Good interview this one.
1:06:48 That part implies that Lee Se-dol retired because of AlphaGo, while in reality he retired because of his dissatisfaction with the Korea Baduk Association, from which he quit in 2016. He mentioned AlphaGo but it is not the reason he quit.
Lex I really admire your interviewing style, you made Silver really light up a number of times. Your voice is like liquid morphine, and you could see how easy it was for your subject to just 'let go', which is great. It's clear though that AlphaGo was his baby .. AlphaStar not so much. I was really hoping to have the same blow by blow for all of the Starcraft games.
Great interview.
We are in this together we will win!