David was the right dude to do this video, his commentary is always so sharp and it translates really well to interview questions. I’m grabbing more popcorn and binging this straight through🍿
People who are as knowledgeable as this guy about a particular field and are able to breakdown complex concepts from their fields into these palatable bits of information for us normies really fascinate me. Great interview, definitely one of my favs!
Danu is a superb guest. He's a gift for explaining complex ideas beautifully. Not surprised he's in a senior position at Google in such a pathbreaking area of computer research. Thanks for a well-organized podcast, David, and for asking Danu these productive questions.
I usually watch the podcast in pieces and finish it right before the next one arrives....but this time I actually finished it on Sunday and had to wait 5 days and I'm so happy right now that they released the bonus episode right when I needed it the most Haha...thankyou guys 😂
Ok, this was excellent. Thank you David for organizing, and Danu for obliging. Loved it, would love to see more stuff like this alongside my fav part of friday - getting home to a new episode of the very enjoyable waveform podcast!
Very cool episode! Congrats to mkbhd for giving space for everyone else in the team to shine. Great work David and overall congratulations to the team for putting great content!
This episode was worth leaving my first ever comment on UA-cam. I love this type of topic and how in this video, the very complex and technical parts are explained so well. Makes me feel both smart for understanding and like an idiot compared to Danu at the same time.
That was super interesting! I love what he said at the end about how one of the practical benefits of AI is that it lowers the barrier of entry for people to be able to implement their ideas. I think this is a much better way to frame how we will use AI, rather than people who might say it promotes laziness or not having to learn things for yourself. I think you still need to have your own knowledge and skillset to generate an idea, but there can be so many other things involved in bringing that idea into reality, and we can't possibly become experts in everything in our lifetime. So that's where you can use AI to fill in certain gaps in your knowledge to some extent to allow you to implement your idea. Could be something like someone who is an awesome baker, but they aren't too business savvy. AI doesn't replace that person being good at baking, but it could help to ease the complications around running a bakery.
David, that was awesome. I have to say I was confused why you joined the team based on your complete differences However, I think you're probably the best edition to MKBHD just for that alone. I think that your team keeping egos in check could become an actual network of force in our industry well in your industry, but definitely for everyone involved. Awesome job that was awesome
I love the way Ellis ask questions. He doesnt just have question, you get an insight into what is going into that question, getting understand a little bit mroe about him as a human.
Interesting that a person from Google, talking about Google was sitting in front of a Mac book. However, putting my kindergartenesk sense of the whimsy aside. This was a brilliant interview. And one I am going to have to re-watch a few times in order to brain soak all the information delivered. Many thanks David.
This is the best explanation for AI I listened to. Like the way you ask the question and how he answer it. Everything is so beautifully weave together. Looking forward for more content like this. I almost missed this because of the title. Many videos with this title is just a clickbait. But you guys really delivers it 🫰🏻
Incredible interview, Danu Mbanga is so intelligent, and knows how to explain things in simple ways… we need more people like him talking about deep tech topics
Wow! That was quite something! Danu was a great, and fascinating guest, and David was excellent as host, with some very intelligent questions... The deep thought processes behind AI and Deep Learning is incredible! Brilliant bonus episode guys; thanks very much. 😃 👍
Great episode. Really valuable information here. David is great at hosting this. Went straight ahead to check Tom Scott and his typing right after watching this :P
Wow, absolutely brilliant discussion!! Love these more technical deep dives. Danu is incredible and David, fantastic interview. Thank you for bringing this to us!
I get this is mostly a sales pitch for Google AI, but there's some major issues today. The first big issue is interpretability of the model. Today, we can't interpret the model weights and most of the research shows that we don't have a formal model of how to interpret weights. The second is we know a fraction of the weights contribute to most of the "important stuff" in neural network models. It's common known as "over parameterized" models. We can see this comparing smaller LLama2 models to GPT4. Llama2 get similar performance to GPT4, but with an order of magnitude fewer parameters. Then there's recent papers that suggests larger models are harder to align, which means scaling transformers up 10 or 100x could be impossible to align. As Jensen Huang has said about data centers, all data centers are power limited. Meaning, data centers don't have enough power to handle 2x the TPU or GPU. This is why microsoft and google are building new data centers. It's also why Tesla is building their own data center ie DOJO. Scaling these models to handle 100 million concurrent users isn't trivial. OpenAI claims to handle 10 million users today and it's constantly having performance issues. Some people have tried to estimate the cost to handle a single chatGPT request. Depending on how to calculate it, it's any where from 2-4 cents per request. In contrast, classic google search is less than 0.1 cents/request. If you look at the latency of chatGPT vs classic google search, it's seconds versus 0.01 milliseconds. That's 1000x slower to run a chatGPT request.
David Awesome Podcast with so much great information and has to be one of my favorite episodes with the person on this. I hope we get to see much more like this.Very interesting and much enjoyed.✌️💯
dang this was unexpected! Loved the podcast david and what great timing :) it was a surprise. Gonna get some coke and popcorn to watch this episode now!!
It was interesting to hear the perspective of someone who fully works and believes in AI. There's a lot that I don't agree with and the absence of awareness on how generative AI is currently hurting traditional media creators (artists especially) feels concerning. Thanks for the interview, Danu and David.
At around 20mins Danu mentions that the fact that the transformer could perform a derivation / write code / etc were it having emergent properties because the transformer wasn't trained to perform those specific tasks. But in the context of a transformer being trained to chain together the most likely strings of tokens & producing a string of tokens to do with each of those topics, you would expect that the strings should be accurate based on training data. So it would seem to not necessarily be an emergent property in the same capacity as something that was trained to get colour that could also provide shape. But then I guess you could argue that it's emergent in the context of not expecting it to be able to perform those activities purely based on the likelihood ratios of strings of token? 🤔
so cool you were really going into depth of what's AI on a platform that can reach the mainstream. hope that more people will understand AI better and not just see as this random magic box - and I loved Danu's positive outlook on how AI will transform the economy. I only missed the discussion on what are the social implications those models can have - eg. how implicit bias is trained in the models and how this problem is addressed by developers. But I enjoyed listening to this episode a lot, thanks for this extra bit!
I think you can mainly say that a model has emergent capabilities, because it has not been trained to have those capabilities. Many text models are trained to predict a token (~= word) in a sentence given the tokens that that appear before that one. Despite that simple instruction they learn to reason.
I think it's more helpful to define Machine Learning not just as the "math behind AI" but in terms of the high-level problems it is invoked to solve: classification, clustering, regression. This makes the distinction between AI, ML, and DL even more clear, at least for me 😅
I wish I was smarter to really appreciate this conversation. This sounds like a really fantastic conversation, hopefully I will learn more about AI and specific terms/ideas, and then I can come back to this and enjoy it. Thanks to Danu for doing this, he is incredibly well-spoken and it's a real delight just hearing him explain anything haha, amazing podcast ep!
This is my second "deep dive" into the "Ai,ML,deeplearning" stuff... I've learnt a bit more about it here((i did not know that the scalability of machinelearning was an unexpected phenomenon))...
The sacred timeline is branching
This is what He Who Remains told us about
Fully 😂👏🏽
if you look closely at him, he looks more like the Watcher
Fr, freaked me out the instant I seen the thumbnail 💀
Internally “is it already Friday?!”
Lol
Love the Loki Reference! 😂
This has thrown off my weekly podcast schedule
i started panicking thinking that the week is already over and I haven't gotten any work done
NO WONDER I WAS SO CONFUSED.
No way we got a bonus episode, it's good day😭
NOO WAY! LETS GO!
🎉🎉🎉
22 minutes in, and I feel I should be getting college credit for this course. Love this one.
Professor David Imel 🤓
David was the right dude to do this video, his commentary is always so sharp and it translates really well to interview questions. I’m grabbing more popcorn and binging this straight through🍿
More than that, he breaks down concepts so easily. I couldn't understand Danu, but David simplified it so anyone can understand it.
Wait, it's working. I time travelled. It's Friday 🎉
I'm glad I got recommended with this gem. Actual good podcast with people discussing things that really matter these days..
People who are as knowledgeable as this guy about a particular field and are able to breakdown complex concepts from their fields into these palatable bits of information for us normies really fascinate me. Great interview, definitely one of my favs!
Danu is a superb guest. He's a gift for explaining complex ideas beautifully. Not surprised he's in a senior position at Google in such a pathbreaking area of computer research. Thanks for a well-organized podcast, David, and for asking Danu these productive questions.
I usually watch the podcast in pieces and finish it right before the next one arrives....but this time I actually finished it on Sunday and had to wait 5 days and I'm so happy right now that they released the bonus episode right when I needed it the most
Haha...thankyou guys 😂
For a moment, I was confused if today maybe is friday
As a pattern recognising algorithm I must conclude that it is probably Friday today
Also, I love that individual members are allowed to host! Such a cool culture in the office.
David is a real one
He really is.
Frfr
The breaking down of the complexities of AI to be more simple and digestible bits was just amazing. Cheers to you guys. A great bonus episode
Love a good bonus episode. And I love learning with David. Maybe he needs your own podcast on the MKBHD network. "David Wants to Know"
Or save it for his own 😉
Such a good episode. I really appreciate the attempts to make the complexities of AI much more simple and digestible. Cheers to the both of them!
amazing interviewer and interviewee. Such a complex and dense topic but you guys made it enjoyable and comprehensible
Ok, this was excellent. Thank you David for organizing, and Danu for obliging. Loved it, would love to see more stuff like this alongside my fav part of friday - getting home to a new episode of the very enjoyable waveform podcast!
Danu was awesome. Very knowledgeable in his field. I enjoyed his thought process and his understanding of issues and events. Thumbs up Dan
Very cool episode! Congrats to mkbhd for giving space for everyone else in the team to shine. Great work David and overall congratulations to the team for putting great content!
If I want to know something as deep as possible, I would trust David to explain it to me. Good choice!
This episode was worth leaving my first ever comment on UA-cam. I love this type of topic and how in this video, the very complex and technical parts are explained so well. Makes me feel both smart for understanding and like an idiot compared to Danu at the same time.
David is my favorite member of the podcast.
This was an episode made only out of passion and curiosity wasn't it?
WHAT!!!! this is too good to be true! Thanks for the extra upload, guys! Still waiting on that old UA-cam features trivia 😂
Danu is well spoken and covered so much! Thanks for a great episode.
Love these episodes where David gets to do a deep dive and cover a topic like this! Good stuff to all involved on this one :)
Bonus episodes led by Studio team members are definitely worthwhile! I will be rewatching this one next week in half term!
i genuinely enjoy listening to this podcast, appreciate everyone working on the Waveform team.
This was the first interview episode of the podcast that I’ve watched. Love David and Danu here
Omg my brain is just blown. I have to take a break and come back. How amazing! Thank you David. This is great
OMG bonus episode!!! You guys just made my day. I literally made a confused face when I saw the notification 😅
This so probably one of the coolest episodes to date! Also I could listen to him talk for days
OMG this like a dream bonus episode, I love you David ❤
Can't say I understood most of this but I did watch it all the way thru giving the episode the respect it deserved.
This is a Masterclass, Danu is a master.
That was super interesting! I love what he said at the end about how one of the practical benefits of AI is that it lowers the barrier of entry for people to be able to implement their ideas. I think this is a much better way to frame how we will use AI, rather than people who might say it promotes laziness or not having to learn things for yourself. I think you still need to have your own knowledge and skillset to generate an idea, but there can be so many other things involved in bringing that idea into reality, and we can't possibly become experts in everything in our lifetime. So that's where you can use AI to fill in certain gaps in your knowledge to some extent to allow you to implement your idea. Could be something like someone who is an awesome baker, but they aren't too business savvy. AI doesn't replace that person being good at baking, but it could help to ease the complications around running a bakery.
David, that was awesome. I have to say I was confused why you joined the team based on your complete differences However, I think you're probably the best edition to MKBHD just for that alone. I think that your team keeping egos in check could become an actual network of force in our industry well in your industry, but definitely for everyone involved. Awesome job that was awesome
This is a fantastic overview; thanks David and Danu!
I love the way Ellis ask questions. He doesnt just have question, you get an insight into what is going into that question, getting understand a little bit mroe about him as a human.
Interesting that a person from Google, talking about Google was sitting in front of a Mac book. However, putting my kindergartenesk sense of the whimsy aside. This was a brilliant interview. And one I am going to have to re-watch a few times in order to brain soak all the information delivered. Many thanks David.
This is just an amazing piece of content which I would pay for.
This is the best explanation for AI I listened to.
Like the way you ask the question and how he answer it. Everything is so beautifully weave together.
Looking forward for more content like this. I almost missed this because of the title. Many videos with this title is just a clickbait. But you guys really delivers it 🫰🏻
Incredible interview, Danu Mbanga is so intelligent, and knows how to explain things in simple ways… we need more people like him talking about deep tech topics
Wow! That was quite something! Danu was a great, and fascinating guest, and David was excellent as host, with some very intelligent questions... The deep thought processes behind AI and Deep Learning is incredible!
Brilliant bonus episode guys; thanks very much. 😃 👍
A Tuesday bonus episode was just what I needed today
This was an awesome watch.
Learning thing i had no idea i needed.
P.S This dude sounds too inteligent.
Up for David leading the podcast anyday.
Great episode. Really valuable information here. David is great at hosting this. Went straight ahead to check Tom Scott and his typing right after watching this :P
Midweek Waveform Episode !
It's a blessed week ! 💯
The David Deep Dive (DDD) is always great
we need more in depths like that!!!
Really hope this 2 episodes per week becomes a norm!!
Love me some bonus David. Well done, loved this episode.
Great episode, we need a part 2 with this guy !
Wow, absolutely brilliant discussion!! Love these more technical deep dives. Danu is incredible and David, fantastic interview. Thank you for bringing this to us!
I get this is mostly a sales pitch for Google AI, but there's some major issues today.
The first big issue is interpretability of the model. Today, we can't interpret the model weights and most of the research shows that we don't have a formal model of how to interpret weights. The second is we know a fraction of the weights contribute to most of the "important stuff" in neural network models. It's common known as "over parameterized" models. We can see this comparing smaller LLama2 models to GPT4. Llama2 get similar performance to GPT4, but with an order of magnitude fewer parameters.
Then there's recent papers that suggests larger models are harder to align, which means scaling transformers up 10 or 100x could be impossible to align. As Jensen Huang has said about data centers, all data centers are power limited. Meaning, data centers don't have enough power to handle 2x the TPU or GPU. This is why microsoft and google are building new data centers. It's also why Tesla is building their own data center ie DOJO.
Scaling these models to handle 100 million concurrent users isn't trivial. OpenAI claims to handle 10 million users today and it's constantly having performance issues. Some people have tried to estimate the cost to handle a single chatGPT request. Depending on how to calculate it, it's any where from 2-4 cents per request. In contrast, classic google search is less than 0.1 cents/request. If you look at the latency of chatGPT vs classic google search, it's seconds versus 0.01 milliseconds. That's 1000x slower to run a chatGPT request.
thank you for having these kind of special episodes esp about AI, aside from the weekly show of course. keep it up and looking forward for mmore
David Awesome Podcast with so much great information and has to be one of my favorite episodes with the person on this. I hope we get to see much more like this.Very interesting and much enjoyed.✌️💯
Please keep this guy on regular rotation
such an interesting discussion, David u did a great job with the questions and Danu is such an expert!!
This was one of my favorite bonus episodes.❤ Great video. 👌
This was just awesome. Sometimes life is uninspiring and then you come across pple like this and think ok maybe humans are something special. Awesome.
Very good episode full of information
I even watched it twice
Thanks David
So cool that you guys actually did this interview
Was not expecting this! It’s a good day today😃
One of the best podcasts to date
Danu does an amzing job explaining this stuff. Bravo
the break down of AI>Machine Learning> Deep Learning was fantastic
Can't believe we have two Fridays this week
Refreshing my feed and seeing this just as I got my food delivered is a sign that good things still happen.
dang this was unexpected! Loved the podcast david and what great timing :) it was a surprise. Gonna get some coke and popcorn to watch this episode now!!
Ayyy extra episode 🔥
Such an interesting video!! awesome to see david doing this episode. You asked just the right questions!
Yes! Thank you Ellis, that was my question as well. So glad you asked it for me
David always tackles the real questions and im here for it!
It was interesting to hear the perspective of someone who fully works and believes in AI. There's a lot that I don't agree with and the absence of awareness on how generative AI is currently hurting traditional media creators (artists especially) feels concerning. Thanks for the interview, Danu and David.
At around 20mins Danu mentions that the fact that the transformer could perform a derivation / write code / etc were it having emergent properties because the transformer wasn't trained to perform those specific tasks.
But in the context of a transformer being trained to chain together the most likely strings of tokens & producing a string of tokens to do with each of those topics, you would expect that the strings should be accurate based on training data.
So it would seem to not necessarily be an emergent property in the same capacity as something that was trained to get colour that could also provide shape. But then I guess you could argue that it's emergent in the context of not expecting it to be able to perform those activities purely based on the likelihood ratios of strings of token? 🤔
Yeayyyyy more content! David you’ve made my day
This is so good on so many levels. Great job!
Love this. Wanted to learn about generative AI.
He's Cameroonian and that's the coolest hting about this whole video
Yes we need a part 2 David
so cool you were really going into depth of what's AI on a platform that can reach the mainstream. hope that more people will understand AI better and not just see as this random magic box - and I loved Danu's positive outlook on how AI will transform the economy. I only missed the discussion on what are the social implications those models can have - eg. how implicit bias is trained in the models and how this problem is addressed by developers. But I enjoyed listening to this episode a lot, thanks for this extra bit!
This is a interesting episode,so to say.
MY WEEK HAS BEEN MADE! 😭
Will I refresh my feed every Tuesday seeing if we get a bonus episode?
Yes? Maybe?
No, definitely.
Not every Tuesday, but we do have a couple extra episodes planned before end of year
I think you can mainly say that a model has emergent capabilities, because it has not been trained to have those capabilities. Many text models are trained to predict a token (~= word) in a sentence given the tokens that that appear before that one. Despite that simple instruction they learn to reason.
I think it's more helpful to define Machine Learning not just as the "math behind AI" but in terms of the high-level problems it is invoked to solve: classification, clustering, regression. This makes the distinction between AI, ML, and DL even more clear, at least for me 😅
I wish I was smarter to really appreciate this conversation. This sounds like a really fantastic conversation, hopefully I will learn more about AI and specific terms/ideas, and then I can come back to this and enjoy it. Thanks to Danu for doing this, he is incredibly well-spoken and it's a real delight just hearing him explain anything haha, amazing podcast ep!
Nice! The slides on the info being talked about is great. Like a learning podcast class lol
This is my second "deep dive" into the "Ai,ML,deeplearning" stuff... I've learnt a bit more about it here((i did not know that the scalability of machinelearning was an unexpected phenomenon))...
David's noises of agreement and understanding seem to be the only thing I can grasp out of this. JK, it was very informative, keep it up.
this was a very illuminating discussion 🔥🔥🔥
Heck yeah! Bonus episode!!
Let's gooo!
I'd love extra episodes on Tuesdays!
Such a good insight, thank you Danu, David, and the team :)
Wooow! One of my faves so far
Much needed video! Thank you muches!