You might think that your videos are fodder for university students boning up on a subject, or mathematicians/engineers in the early stages of their careers - basically that you cater to a younger audience. I'll have to prove you wrong. I'm in my early 60's and have been involved with information technology in some form or fashion my entire career. I enjoy learning; always have. I've viewed many of your videos only because they interest me and have you subscribed on my UA-cam account so as to get notifications of updates. I find the topics about which you speak fascinating and am a bit jealous of those university grads today who now have access to this material at their fingertips. I wish with all my heart that I was able access these videos back when I was in university. It would have made life SOOO much easier for me back then. Your pedagogic skills are astounding, demonstrated by your ability to communicate difficult subjects precisely, concisely and simply. The animation format is integral with the presentation, adding to the delivery of the material. I salute you!! Please keep these videos coming.
That is very inspiring to hear you say I am a student rn, not in university though, but I love learning and hope to continue to foster a love of learning.
And great thing about democratizing this learning is that kids in Africa or Europe or Asia have access to same content to what kids in America have. This makes me wonder if learning becomes easier or easily accessible, what the kids of future will be learning?
I can't claim to have understood everything from the first watch-through of this series, and I will watch these videos again with pen and paper in hand, but even this first viewing has made neural networks go from pure witchcraft and wizardry to something that actually makes sense in my head. I can't possibly thank you enough for posting these videos.
"...has made neural networks go from pure witchcraft and wizardry to something that actually makes sense in my head" I died coz I've never related to something so much xD D:
@@GabrielCarvv If this video did not explain this to you then nothing will I'm afraid... Don't get me wrong - it's not about you - its a tough subject but this is by far the most approachable presentation I have seen out there. I can't even imagine how one would present it to the fellow human being in a more approachable way... ;-) PS. Have you watched the previous 3 vids from his series?
This series is totally brilliant. I am 73 years old and used to teach mathematics. I am still learning stuff and with the help of sites like yours it makes it so much easier. Have you thought of doing any videos on the really complex subject of real analysis/. Keep up the good work. Kevin Connolly
hello, I think you are 76 years old now, if you want, you can become a Muslim and enter paradise forever in the hereafter. "But those who believe and work righteousness,- no burden do We place on any soul, but that which it can bear,- they will be Companions of the Garden, therein to dwell (for ever)". (Al-A'raf [7] :42) ( i am not a bot)
These visualizations are spot-on. Only a few people in the entire world need to make a great explanations backed by powerful visualizations about a topic - the rest of the world just needs to discover these. So much time wasted by learners trying to locate easily-digestible information, among all the inferior presentation methods out there. Glad to have found one of the best for this topic.
I must say it. I am 100% seriously learnt English just to be able watch your videos. This is the content that will help everyone wandering grow stronger in their favourite subjects. I thank you for your work from the bottom of my heart ❤️❤️
I find it astonishing how well you convey some of the intricacies here, way better than most of ML practitioners who teach the public, whether as youtubers, online instructors or public speakers who end up on video on the internet. I very much resonate with the way you frame things, the metaphors you choose, your visualizations and of course your evident love for understanding and sharing thereof. Your work is a great gift to all of us - students, engineers, researchers, philosophers, random viewers from all walks of life. Thank you.
Great explanation, your team is awesome. "A drunk man stumbling aimlessly downhill,but taking quick steps" is the best analogy ever for Stochastic gradient descent. :-)
"A drunken sailor walking a plank" is often used to describe fixation of an allele by genetic drift in a small population. So next time you've had one too many, explain that you're feeling a little too stochastic to drive.
Note to myself: Aditya, if you're having trouble understanding, read this. Scroll to 06:17. Listen to what he's saying, "In a sense, the neurons that are firing while seeing a 2, get more strongly linked to those firing when thinking about a two." Now, pause the video and listen. All this is is just a fancy way of saying: when we show our model a picture of a handwritten 2 and we tell it, "hey listen up this thing is a 2" (like in our training set we have labels for our input pictures) and then we find out the activation units that have a say in influencing the hypothesis value of the 2 label, ie, the activation units which can heavily increase or decrease the value of the output unit for the label 2, we tell those activation units "hey guys when you see something that resembles this thing, fire up the 2 label, ie increase the hypothesis value of the label for 2. Basically, we assign those activation units greater weights (or parameters) which influence the hypothesis value of the 2 label more (so that they can actually give us the right answer and say "oh look this is probably a two". I hope this helped and didn't complicate it further. If you don't get it, go over the video a few more times and review Andrew Ng's notes on this in the ML course on the Backpropagation lecture in week 5. Cheers, bro. Love you.
Thanks dude, it really helped. I am trying to follow Ng but after a point it sounds me like an Indian language. Sometimes it might be a bit difficult to visualize it in mind, and Ng is really a bit 'bad choice' about that part. I am not saying that he is a bad teacher but sometimes I feel like I am surfing through the universe...
@@hzmuhabbet I understand. Ng gets caught up in notation quite a lot so it's difficult to follow his intuition. I actually just noted this down for myself for future reference but I'm glad that you and other people are finding it helpful in understanding! I wish you well on your ML journey!
I finished the Andrew NG course except for the programming assignment part. I felt his explanation wasn't quite reaching my head and now watching this series has blown me away at how easy it's to understand things when they are visualized. Thanks for sharing your note snippet, pretty cool!
First time I caught myself having a moment of awe while watching educational content. The production value is incredibly high... the way the connections twinkle and move to represent adjusting the weights, the small animations, the descent into various shapes, how the little arrows indicating the desired change move and change size. Beautifully put together! Thanks a lot!
I am a neuroscientist, and all your comparisons of actual neural networks to artificial neural networks seem pretty spot on to me. This is a great video series.
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
I've watched these videos 3 times and everytime I watch them, I feel a bit smarter. It starts with understanding little to progressively understanding more and more and finally seeing the big picture. Can I say I've understood everything? No. I am getting there and I'll be coming back for these again. For anyone who's feeling discouraged, I can assure you that you'll get there. Thank you so much for creating quality content that brings the driest, most theoretical concepts to life! You're a hero.
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
Thank you for saying this, I was pretty confused by some things in this series and I'm glad to hear that it gets better the more times you watch it. Learning some fundamental stuff like linear algebra and calculus would probably help as well, I really should get onto that
@@vgdevi5167 Hey, this might be a little late but there are some links to learning about Neural networks/Backpropagation in the description if you haven't looked at them already.
No form of words can express enough what magic you're creating! I don't know how much you actually think you impact us.. but let me tell you Grant, your effect on my life is immeasurable! And the fact that you learnt it the hard way, and made it so simple for us, so that we don't have to go through the same, makes me respect you even more and more every single day.. Thank you so much.. 3B1B is undoubtedly the best channel on UA-cam..
All of the videos shown on this channel have been written so well. Even I can understand, mathematics was a complete mystery for me in school. I have a sixth grade education and I feel really smart after watching one of these programs. Thanks so much. I'm really grateful that you have taken the time to educate the people that have a hard time understanding but still want to learn. What you're doing is just as important as any other volunteer or charity work. I'm excited I found this .
Ngl man one of my Goals, once I get a good enough job. Will be to be able to live contribute to your patreon. Your videos have been so entertaining, inspiring, and helpful that I hope the move that I give allows you to continue with this channel.
I love how you enunciate the words as if you are truly interested and into the topic. The art direction and the animations are also on point. Kudos to the animator!
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
I've been reading the book for quite some time but your explanations using animations have pushed my understanding to new levels. So many thanks to you. PS: Keep the background music. Holds our concentration for a long time.
Thank you for what you are doing. I have just started learning ML and I find your video really helpful. Thank you again from bottom of my heart for everything you are doing. I just wanted to let you know that you are really making changes and inspiring youngsters like me.
I am a “seasoned” systems engineer, inventor, and IT professional. IOW, I’m old (40+), lol. Like @Robert MacKinnon has said, you may think that your wonderful videos are only for those new in the field or younger folks in general. This couldn’t be further from the true. Your ability to explain things with clear diction, amazing graphics, and compelling story telling is phenomenal. They help me to not only refresh to get spun up quickly, but also serve as a point of reference for me to direct those newer in my teams; as your ability to run through this at just the right level and pacing far exceeds mine. I would go so far as to suggest that you freelance as a guest speaker or instructor for hire for larger firms having folks wanting to get spun up quickly. Apologies if this is old news and you’re already pursuing this. I simply feel compelled to tell you as we often don’t understand how awesome we are and need to be reminded by others. Consider this your reminder ;) Thanks again for your wonderful videos where you ask for nothing. I would pay for a Patron page if you get one to be able to ask for more content or explanation on topics such as the vanishing gradient problem prone to recurrent neural networks and how LSTM addresses it. I’ve been looking for a good one that does this graphically with intuitions. I haven’t found a good one yet, but I did come here first to check ;) All the best, CCM.
I'm here doing a Masters in computer science and having seen these things in my classes, this video gave me full idea of just how beautiful this whole development is. Someone was really smart
All I can say is thank you! Every 3b1b video that I have watched is simply incredible. The chronology of ideas introduced throughout a video makes complex topics easy to follow, and the animations are BEAUTIFUL. I have never been able to visualize math so well before. You are an amazing teacher and have contributed invaluable knowledge to society! I'm still in shock that educational resources this profound are available for free. I truly appreciate the work you are doing!
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
So far, I think this is the best intuitive intro to backprop I've seen. This channel means a lot to me. Because of its teaching style, I managed to get back to learning and grok maths while I was studying machine learning last year, 13 years since doing my undergrad in CS. It's almost the end of 2017 and I still keep hearing from the people I talked to that they fear/hate maths because they think it's a tough subject to tame. May be this example tell us why our education system (internationally) is still broken? As an aside, David Perkins, in his book "Making Learning Whole" also touch about this widespread diseases of the educational system, namely "elementitis". I think we can do better. Grant is doing great work to lower the barrier and making math more accessible to everyone. This is not an easy feat. I think we need some sort of concerted effort for encouraging more people to teaching maths or any subject through intuition. Visualization is one way to improve the teaching methodology. We can also distill intuition from stories, feelings, situations, etc. More examples, see: distill.pub/ colah.github.io/
I am just in awe with the presentation of the concepts. The graphics, the use of animations, ... It is all very helpful in making us visualize such complex concepts. THANK YOU!!!
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
It is stunning how much better and clearer you can explain all this than my professor at university, and you are even a lot faster! Thank you so much for producing all those brilliant videos. Some of some help me understand what my professors are trying to tell me, and some of them are just fascinating. Keep on going!
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
I'm graduated in Statistics an Data Science by my local university. Study the subject for almost a decade. For years I heard about backpropagation. No one knew how to explain it (or worse, knew how it worked). It always was "a magic algorithm that adjusts weights". This video did in 13 minutes what many professors during years couldn't do to me: explain in a simple way the logic behind it. It's just perfect: well done, free an with a relevant sponsor in the end. Best video I ever watched. Congratulations!
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
I love the statement "the most confusing part is the notation". This is so true 😂 It would be really, really nice if you did a last video on how to program this network in a comprehensive way 😄 IMO the inventors of DNN chose all those activation functions and matrice multiplications really carefully to make computation feasible. But all this beautiful simplicity is gone once these unnecessarily complicated math formulas come into place. Honestly, you did an amazing job with your videos on DNNs and backpropagation. The first time I seemed to understand it.
Im taking a Neural networks course at my university right now and I'm finally starting to understand this stuff after watching this video series! Thank you so much!
This is a lifesaver for completing week 5 of Andrew Ng's Machine Learning course!! Been stuck at backpropagation for hours and finally found some clarity in this video. I love your channel! ❤️
I have just found your videos. I have enjoyed watching science educators on youtube for many years, but these videos are the best examples of complex ideas being explained clearly whilst still being entertaining. I can't believe I had never heard of this channel before.
This should be art. I don't normally comment on videos but I feel I must express my gratitude and appreciation for your amazing explanations and animations! Visualisation is so important and helpful and you have nailed it beautifully!
All of the other commenters beat me to it, but I can't express enough how insanely high the quality of these videos is, and how much they help teach people new things. Thank you so much for making all of them and sharing some of your vast amount of knowledge!
I had attended 3Blue1Brown sessions on Vector and Calculus earlier before recently attending Machine Learning and Neural Networks. It is just amazing to see a mix of great content, great description along with excellent graphic visuals. It just makes it so comprehensible that way. I want to convey my thanks to the whole team and wish them continued success.
Sorry, just noticed a mistake and wanted to point it out. What you are calling Stochastic Gradient Decent is actually called Mini Batch Gradient Decent. SGD is actually when you take a gradient and update the parameters at every single observation, while MBGD is where we split the data into subsets and update the parameters based on each subset (what you were referring to in the video). Loves the video though, a lot was clarified for me. Keep up the good work :)
wow, I am surprise anyone knows this sophisticated concepts to explain them clearly, and yet you also accompany them by amazing visuals which I believe not easier to do. Thank you!
"This is where we were in the 80s/90s" - I did my degree in the 90s, so this is the level I learned to. The more up to date stuff would be interesting.
Normally, I never comment on youtube videos, but I'm making an exception for this one. Even though you might not see this, I want to thank you for your contribution to everyone's education. And for FREE access to it too. Amazing videos with amazing presentation, structure, and explanations. I'm 3 years late, but please do keep up the great work!
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
I would expect the training data to also include a bunch of random images that the network could classify as "not a number" (not hotdog 😉) That way, instead of just taking its wildest guess at which number it's detecting, it could also intelligently say "this doesn't look like a number to me"
From what i understood i think you could just program it to say that whenever all the values in the last layer are either negative or really low without changing the back propagation process. Not sure tho.
You need to be sure your "random non-number" is actually a lot of different things, otherwise the "not a number" result would have similar chances of being triggered as the number results when for example instead of alphabetical characters you present the NN with white noise it didn't train on.
I would like to take the forum to THANK YOU for your efforts to make the extremely complex topics comprehensible. I am new to the subject and found a very good starting point to start my journey towards understanding a little bit of AI. Thank you very much.
I think it'd be extremely beneficial if you were a complete lecture series for deep learning. Your way of teach is billion times better than this somewhat popular Andrew Ng lecture series, and also a lot more intuitive. Please consider this:) Best wishes
I am writing my internal assesment for school about neural networking and thank god your channel exists cuz everything about this subject is now clear to me. You explanations are perfect!
Let's just say a quick thank you to all the mathematicians who wrote the highly optimized software libraries that do all this math for us so ordinary programmers can quickly get neural nets up and running on our training data.
1.3 million views ... wow. I'm kind of blown away by how many people are interested and following this series. That's far more than I would have guessed! Great!
Oh dear! Absolutely brilliant. This is one heck of video. How would you consider creating an exhaustive ML and Neural Network video series? I'm sure there are a lot of curious folk out there, waiting for it!
Thank you so much for the simplicity in your explanations. You try to explain everything not by their definition, but with an understandable, simple way. Thank you!
"I recently created my own UA-cam channel, and I’m currently a private student. These days, I feel like I have little formal education, but through this channel, I’ve been learning a lot and gaining knowledge. It feels like I’ve entered a new world where I can study from home. This whole experience has been very inspiring for me."
I might misunderstand but maybe not. At 7:38, we want to reduce the activation of the neuron responsible for 3 right? So we should decrease the activation of the neurons that have a positive weight to 3, no? In the video, it's actually the opposite. For example, the first neuron has a positive weight with the neuron responsible for 3, so we should decrease its activation right? Can someone help me on that please?
I had the same question. It seems like the color-coding of the video at this point is inconsistent? (it applies the same procedure for adjusting the preceding layer of neurons to all the other output neurons as it does to the output neuron for 2, when really the signs in the procedure should be reversed for all the neurons besides 2, since we want their activations to decrease?)
I know what you mean, the best thing you can do to help the channel is by subscribing,liking, and sharing videos as cliche as it sounds it does help creators, not a creator myself just someone who wants to see these kinds of educational channels grow!
@@johncoleman1930 I have subscribed, I like every 3blue 1brown video I see. But, I don't know any one who is interested in this kind of stuff. So, I can't really share
I'm a senior in my computer science major and honestly I was starting to get very burnt out of it. I used to have a lot of love for math and computer science but the cumulative effect of classes rampaging through topics without giving a thorough explanation really started dimming that love. I'm in an AI class this semester and while the professor is incredible at what he does, he sadly has no idea how to teach the concepts to new students. My motivation was at an all time low and I simply went to class as routine rather than to get knowledge I care about out of it. Your video series have reinvigorated my passion for these subjects and to that I thank you. Rather than fearing and dreading what's to come I now look forward with enthusiasm.
At 7:37, shouldn't the activations associated with the digit 0 neuron (whose activation should be decreased) change the opposite direction? For example, if the neuron at the very top of the second-to-last layer increases activation, with a positive weight, the digit 0 neuron's activation would increase. Is that correct?
Hello, I'd like to start by saying your videos are amazing and beyond intersting. One thing I'm still confused about is how to update the biases through backpropagation? Thanks! and keep it up!
I had to stop seeing your video for a moment because I needed to write down my admiration for your work. It is outstanding the amount of intelligence, communication skills, careful graphic design and passion in very single frame of your videos. Congratulations and thank you for your contribution to maths education. See you in patreon!
I feel like I'm missing something with the functionality of backpropogation. So, I get that you adjust the weights between the last two rows to adjust how the second last row feeds into the last row in the optimum way, and then you change the second last way, and then you backpropogate the second row of weights away from the end, etc. But, what I don't understand is that if you change the second last weights, that's going to change the second last row of inputs, which means that the tweaked weights between the last hidden layer and the output will be wrong again. I'm trying to work out what I'm missing. Is it that all the layers of weights are tweaked simultaneously in order to be accurate? If so, I'm still not sure I understand how that works (I'll probably try and watch the video again in the morning when my brains properly awake.)
+Conor O'Neill Good question! Backpropagation is just giving the list of adjustments you will make, but you don't make them until all are computed. And of course, since this is part of (stochastic) gradient descent, that's a process you repeat multiple times.
You bring up a really good point. The next video, which will cover the actual mathematics of how backpropagation works, does take that into account. If you've taken calculus before, you've heard of the chain rule, which describes how to differentiate an expression like f(g(x)). The chain rule states that d/dx f(g(x)) = f'(g(x)) * g'(x). The chain rule is general enough that it's applicable to pretty much all situations where you can express stuff in terms of an f and g, even when f and g take multiple inputs. The important thing is that it allows you to only analyze one input at a time, making it easy to compute stuff. What happens in the case of backpropagation is that you're taking the derivative in terms of each of the weights of the cost function, and the chain rule accounts for how changes affect other changes. The math is too complex to put in a youtube comment (mainly because youtube doesn't have mathjax, so I can't write a lot of the symbols I'd need, like summations), but the next video should explain all of that!
Consider that the changes merited by each example are quite small (because each example alone doesn't carry enough weight to merit making big changes), so the changes of weights based on these values are still accurate. It doesn't matter if you do a tiny step after each example, or a larger one after a thousand examples, the over all effect of each example is still like a single tiny step in the direction it points to.
I have a masters in CS and learned quite a bit about Machine Learning in college but ironically never learned the math behind any of it. This is a brilliant video and I can’t wait to dig into the calculus of it all!
You might think that your videos are fodder for university students boning up on a subject, or mathematicians/engineers in the early stages of their careers - basically that you cater to a younger audience. I'll have to prove you wrong. I'm in my early 60's and have been involved with information technology in some form or fashion my entire career. I enjoy learning; always have. I've viewed many of your videos only because they interest me and have you subscribed on my UA-cam account so as to get notifications of updates. I find the topics about which you speak fascinating and am a bit jealous of those university grads today who now have access to this material at their fingertips. I wish with all my heart that I was able access these videos back when I was in university. It would have made life SOOO much easier for me back then. Your pedagogic skills are astounding, demonstrated by your ability to communicate difficult subjects precisely, concisely and simply. The animation format is integral with the presentation, adding to the delivery of the material. I salute you!! Please keep these videos coming.
I could not have said it better.
That is very inspiring to hear you say I am a student rn, not in university though, but I love learning and hope to continue to foster a love of learning.
Well said.
This 65 yr old ex s/w developer completely concurs!
And great thing about democratizing this learning is that kids in Africa or Europe or Asia have access to same content to what kids in America have. This makes me wonder if learning becomes easier or easily accessible, what the kids of future will be learning?
Anyone else smiling through all of his videos because you're understanding so much so well like never before?
yup
Yes!!!!!
yeah man!
Every single time.
Hell yeah
I can't claim to have understood everything from the first watch-through of this series, and I will watch these videos again with pen and paper in hand, but even this first viewing has made neural networks go from pure witchcraft and wizardry to something that actually makes sense in my head.
I can't possibly thank you enough for posting these videos.
and I thought everybody else here except me is a genius
"...has made neural networks go from pure witchcraft and wizardry to something that actually makes sense in my head"
I died coz I've never related to something so much xD D:
I thought that I was the only one who did it.
I did not understand a single word. Please help me.
@@GabrielCarvv If this video did not explain this to you then nothing will I'm afraid... Don't get me wrong - it's not about you - its a tough subject but this is by far the most approachable presentation I have seen out there. I can't even imagine how one would present it to the fellow human being in a more approachable way... ;-) PS. Have you watched the previous 3 vids from his series?
This series is totally brilliant. I am 73 years old and used to teach mathematics. I am still learning stuff and with the help of sites like yours it makes it so much easier. Have you thought of doing any videos on the really complex subject of real analysis/. Keep up the good work. Kevin Connolly
Cool! I'm mid forties and about to do a CompSci MSc. Gotta keep my own neurons going!
@@damian_smithSame! Im 31 getting my PhD in CS! Hopefully i dont need to be in a retirement home at 35.
hello, I think you are 76 years old now, if you want, you can become a Muslim and enter paradise forever in the hereafter. "But those who believe and work righteousness,- no burden do We place on any soul, but that which it can bear,- they will be Companions of the Garden, therein to dwell (for ever)". (Al-A'raf [7] :42) ( i am not a bot)
@@DarkStar-imThis is the wrong place to spread Islam.
No one is converting to Islam from a reply to their UA-cam comment.
These visualizations are spot-on. Only a few people in the entire world need to make a great explanations backed by powerful visualizations about a topic - the rest of the world just needs to discover these. So much time wasted by learners trying to locate easily-digestible information, among all the inferior presentation methods out there. Glad to have found one of the best for this topic.
It is challenging to go through books and lectres from authors and professors trying to sound smart.
I must say it. I am 100% seriously learnt English just to be able watch your videos. This is the content that will help everyone wandering grow stronger in their favourite subjects. I thank you for your work from the bottom of my heart ❤️❤️
I find it astonishing how well you convey some of the intricacies here, way better than most of ML practitioners who teach the public, whether as youtubers, online instructors or public speakers who end up on video on the internet. I very much resonate with the way you frame things, the metaphors you choose, your visualizations and of course your evident love for understanding and sharing thereof. Your work is a great gift to all of us - students, engineers, researchers, philosophers, random viewers from all walks of life. Thank you.
Great explanation, your team is awesome. "A drunk man stumbling aimlessly downhill,but taking quick steps" is the best analogy ever for Stochastic gradient descent. :-)
what the deuce?
stewie
@@youtubeviolatedme7123
Nah, it's actually not that uncommon. My material science professor even used it when discussing some Quantum processes! 😁
"A drunken sailor walking a plank" is often used to describe fixation of an allele by genetic drift in a small population. So next time you've had one too many, explain that you're feeling a little too stochastic to drive.
I disable adblock for this
+Simon van der Poel Funny, because I disabled ads for this :)
What's the first rule of adblock?
TIGuardian - Apologies, I was talking to Simon, the first rule is you never talk about adblock.
I tried, but my adblock uses neural networks. It decided it would rather re-enable itself.
Bro there's ads in the video
Note to myself:
Aditya, if you're having trouble understanding, read this.
Scroll to 06:17. Listen to what he's saying, "In a sense, the neurons that are firing while seeing a 2, get more strongly linked to those firing when thinking about a two." Now, pause the video and listen.
All this is is just a fancy way of saying: when we show our model a picture of a handwritten 2 and we tell it, "hey listen up this thing is a 2" (like in our training set we have labels for our input pictures) and then we find out the activation units that have a say in influencing the hypothesis value of the 2 label, ie, the activation units which can heavily increase or decrease the value of the output unit for the label 2, we tell those activation units "hey guys when you see something that resembles this thing, fire up the 2 label, ie increase the hypothesis value of the label for 2. Basically, we assign those activation units greater weights (or parameters) which influence the hypothesis value of the 2 label more (so that they can actually give us the right answer and say "oh look this is probably a two".
I hope this helped and didn't complicate it further. If you don't get it, go over the video a few more times and review Andrew Ng's notes on this in the ML course on the Backpropagation lecture in week 5. Cheers, bro. Love you.
Thanks dude, it really helped. I am trying to follow Ng but after a point it sounds me like an Indian language. Sometimes it might be a bit difficult to visualize it in mind, and Ng is really a bit 'bad choice' about that part. I am not saying that he is a bad teacher but sometimes I feel like I am surfing through the universe...
@@hzmuhabbet I understand. Ng gets caught up in notation quite a lot so it's difficult to follow his intuition.
I actually just noted this down for myself for future reference but I'm glad that you and other people are finding it helpful in understanding! I wish you well on your ML journey!
I finished the Andrew NG course except for the programming assignment part. I felt his explanation wasn't quite reaching my head and now watching this series has blown me away at how easy it's to understand things when they are visualized. Thanks for sharing your note snippet, pretty cool!
nice
I love that you're being so friendly and positive to yourself in these notes
Your animations are amazing!
How are the animations made? It is great!
He has built an animation "engine" in python, which you can find on his GitHub
Thanks!
Brian Evans a
github.com/3b1b/manim
3blue1brown > Indian guy on youtube >>> my CS teacher
First time I caught myself having a moment of awe while watching educational content. The production value is incredibly high... the way the connections twinkle and move to represent adjusting the weights, the small animations, the descent into various shapes, how the little arrows indicating the desired change move and change size. Beautifully put together! Thanks a lot!
I am a neuroscientist, and all your comparisons of actual neural networks to artificial neural networks seem pretty spot on to me. This is a great video series.
You are a wizard when it comes to animations and understanding
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
You would like Quaternions.
I've watched these videos 3 times and everytime I watch them, I feel a bit smarter. It starts with understanding little to progressively understanding more and more and finally seeing the big picture. Can I say I've understood everything? No. I am getting there and I'll be coming back for these again. For anyone who's feeling discouraged, I can assure you that you'll get there.
Thank you so much for creating quality content that brings the driest, most theoretical concepts to life! You're a hero.
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
Thank you for saying this, I was pretty confused by some things in this series and I'm glad to hear that it gets better the more times you watch it.
Learning some fundamental stuff like linear algebra and calculus would probably help as well, I really should get onto that
@@vgdevi5167 Hey, this might be a little late but there are some links to learning about Neural networks/Backpropagation in the description if you haven't looked at them already.
Kurzgesagt and 3Blue1Brown in the single hour! I am in heaven.
Utsav Munendra ikr?!?!?!
Utsav Munendra
I was about to comment this.
So true!
this is fav youtuber so farrrrr
haha me too
No form of words can express enough what magic you're creating! I don't know how much you actually think you impact us.. but let me tell you Grant, your effect on my life is immeasurable! And the fact that you learnt it the hard way, and made it so simple for us, so that we don't have to go through the same, makes me respect you even more and more every single day.. Thank you so much.. 3B1B is undoubtedly the best channel on UA-cam..
All of the videos shown on this channel have been written so well. Even I can understand, mathematics was a complete mystery for me in school. I have a sixth grade education and I feel really smart after watching one of these programs. Thanks so much. I'm really grateful that you have taken the time to educate the people that have a hard time understanding but still want to learn. What you're doing is just as important as any other volunteer or charity work. I'm excited I found this .
your videos have made it incredibly easier to grasp fundamental concepts of ML. Great work!!
I just want to say that I love that you're moving into the mathematics of ML. The visualizations convey the concepts so well!
Ngl man one of my Goals, once I get a good enough job. Will be to be able to live contribute to your patreon. Your videos have been so entertaining, inspiring, and helpful that I hope the move that I give allows you to continue with this channel.
Grant, you're a diamond.
no hes a human
you must be fun at the party
@@-long- I actually found it funny. Also, THE PARTY sounds dramatic, am I invited?
@@timangar9771 you gotta admit that we have weird sense of humor for watching this video for fun
no he's a pi creature
The way you teach such a complex topic is astounding
I love how you enunciate the words as if you are truly interested and into the topic. The art direction and the animations are also on point. Kudos to the animator!
bro .. the real magic is how effectively you make such a hard subject seem understandable to a layperson .. bless you
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
I've been reading the book for quite some time but your explanations using animations have pushed my understanding to new levels. So many thanks to you.
PS: Keep the background music. Holds our concentration for a long time.
Thank you for what you are doing. I have just started learning ML and I find your video really helpful. Thank you again from bottom of my heart for everything you are doing. I just wanted to let you know that you are really making changes and inspiring youngsters like me.
I am a “seasoned” systems engineer, inventor, and IT professional. IOW, I’m old (40+), lol. Like @Robert MacKinnon has said, you may think that your wonderful videos are only for those new in the field or younger folks in general. This couldn’t be further from the true. Your ability to explain things with clear diction, amazing graphics, and compelling story telling is phenomenal. They help me to not only refresh to get spun up quickly, but also serve as a point of reference for me to direct those newer in my teams; as your ability to run through this at just the right level and pacing far exceeds mine. I would go so far as to suggest that you freelance as a guest speaker or instructor for hire for larger firms having folks wanting to get spun up quickly. Apologies if this is old news and you’re already pursuing this. I simply feel compelled to tell you as we often don’t understand how awesome we are and need to be reminded by others. Consider this your reminder ;) Thanks again for your wonderful videos where you ask for nothing. I would pay for a Patron page if you get one to be able to ask for more content or explanation on topics such as the vanishing gradient problem prone to recurrent neural networks and how LSTM addresses it. I’ve been looking for a good one that does this graphically with intuitions. I haven’t found a good one yet, but I did come here first to check ;) All the best, CCM.
I'm here doing a Masters in computer science and having seen these things in my classes, this video gave me full idea of just how beautiful this whole development is. Someone was really smart
This series is better than my in total 6-hour lecture from school.
Implementing a neural network from scratch for a class project and this series helped me so much to get started. Thanks!!!
Man, the clarity and the animations make your videos masterpieces
I never understood neural nets like this before... Thanks!
* eagerly waiting for a video about convolutional neural networks *
+Michael Incog It'll be a little while. The next few videos won't be for this series, so I'll probably return back to this in a few months.
Computerphile did some videos on convolutional neural network
3Blue1Brown Please please make one soon
@@3blue1brown hasn't the few months passed? The whole community is waiting for a video on convolutional neural networks.
@@alberttamazyan Ikr :(
All I can say is thank you! Every 3b1b video that I have watched is simply incredible. The chronology of ideas introduced throughout a video makes complex topics easy to follow, and the animations are BEAUTIFUL. I have never been able to visualize math so well before. You are an amazing teacher and have contributed invaluable knowledge to society! I'm still in shock that educational resources this profound are available for free. I truly appreciate the work you are doing!
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
So far, I think this is the best intuitive intro to backprop I've seen. This channel means a lot to me. Because of its teaching style, I managed to get back to learning and grok maths while I was studying machine learning last year, 13 years since doing my undergrad in CS. It's almost the end of 2017 and I still keep hearing from the people I talked to that they fear/hate maths because they think it's a tough subject to tame. May be this example tell us why our education system (internationally) is still broken? As an aside, David Perkins, in his book "Making Learning Whole" also touch about this widespread diseases of the educational system, namely "elementitis". I think we can do better. Grant is doing great work to lower the barrier and making math more accessible to everyone. This is not an easy feat. I think we need some sort of concerted effort for encouraging more people to teaching maths or any subject through intuition. Visualization is one way to improve the teaching methodology. We can also distill intuition from stories, feelings, situations, etc. More examples, see:
distill.pub/
colah.github.io/
Cedric Chee spot on! And thanks a lot for those links.
Thanks
I am just in awe with the presentation of the concepts. The graphics, the use of animations, ... It is all very helpful in making us visualize such complex concepts. THANK YOU!!!
You are amazing!! Best explanation I've seen on the internet yet!!
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
It is stunning how much better and clearer you can explain all this than my professor at university, and you are even a lot faster!
Thank you so much for producing all those brilliant videos. Some of some help me understand what my professors are trying to tell me, and some of them are just fascinating. Keep on going!
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
Came for the knowledge, stayed for the Animations.
Here I'm thinking how it can be done in After Effects, and turned out it's custom engine!
I'm graduated in Statistics an Data Science by my local university. Study the subject for almost a decade.
For years I heard about backpropagation. No one knew how to explain it (or worse, knew how it worked). It always was "a magic algorithm that adjusts weights". This video did in 13 minutes what many professors during years couldn't do to me: explain in a simple way the logic behind it. It's just perfect: well done, free an with a relevant sponsor in the end. Best video I ever watched. Congratulations!
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
I love the statement "the most confusing part is the notation". This is so true 😂 It would be really, really nice if you did a last video on how to program this network in a comprehensive way 😄
IMO the inventors of DNN chose all those activation functions and matrice multiplications really carefully to make computation feasible. But all this beautiful simplicity is gone once these unnecessarily complicated math formulas come into place.
Honestly, you did an amazing job with your videos on DNNs and backpropagation. The first time I seemed to understand it.
Im taking a Neural networks course at my university right now and I'm finally starting to understand this stuff after watching this video series! Thank you so much!
This is a lifesaver for completing week 5 of Andrew Ng's Machine Learning course!! Been stuck at backpropagation for hours and finally found some clarity in this video. I love your channel! ❤️
Haha, I'm going through exactly the same thing!
@@Endothermia lol, high five and hang in there mate. :)
I have just found your videos. I have enjoyed watching science educators on youtube for many years, but these videos are the best examples of complex ideas being explained clearly whilst still being entertaining. I can't believe I had never heard of this channel before.
This should be art.
I don't normally comment on videos but I feel I must express my gratitude and appreciation for your amazing explanations and animations! Visualisation is so important and helpful and you have nailed it beautifully!
I totally like how your commercial sponsor is totally relevant and not just toothpaste !
This video is literal ART. Grant is my favorite artist.
All of the other commenters beat me to it, but I can't express enough how insanely high the quality of these videos is, and how much they help teach people new things. Thank you so much for making all of them and sharing some of your vast amount of knowledge!
I wish I had the option to pay my college tuition to you instead of my university. Well done mate!
I had attended 3Blue1Brown sessions on Vector and Calculus earlier before recently attending Machine Learning and Neural Networks.
It is just amazing to see a mix of great content, great description along with excellent graphic visuals. It just makes it so comprehensible that way.
I want to convey my thanks to the whole team and wish them continued success.
You have made life so much easier.
Half the time I'm in awe of the quality of the content. The other half I'm learning. Thank you so much man.
This man is the eternal fountain of knowledge straight from the heavens
As a visual learner, I can't thank you enough for this kind of video. Reading math notation is a piece of cake after seeing this!!
Sorry, just noticed a mistake and wanted to point it out. What you are calling Stochastic Gradient Decent is actually called Mini Batch Gradient Decent. SGD is actually when you take a gradient and update the parameters at every single observation, while MBGD is where we split the data into subsets and update the parameters based on each subset (what you were referring to in the video). Loves the video though, a lot was clarified for me. Keep up the good work :)
this author has an amazing talent to explain complex things in a simple intuitive visual way. that what most of academia lacks. thanks
9:00 Eureka moment. I cried.
I actually got touched for real from these videos. A great example of the beauty of the math.
Felt obvious to me 😕
aditya sai
You are so intelligent! Wow!
Hahaha I feel you mate
@Kiros37100
You're so bitter and sarcastic! Wow!
wow, I am surprise anyone knows this sophisticated concepts to explain them clearly, and yet you also accompany them by amazing visuals which I believe not easier to do. Thank you!
Hey! Just want to shout you are my HERO! I'm an AI and ML researcher and I LITERALLY cry watching those awesome explanations! YOU ARE MY HERO!
Am constantly in awe of how intuitive this is.
"This is where we were in the 80s/90s" - I did my degree in the 90s, so this is the level I learned to. The more up to date stuff would be interesting.
Normally, I never comment on youtube videos, but I'm making an exception for this one. Even though you might not see this, I want to thank you for your contribution to everyone's education. And for FREE access to it too. Amazing videos with amazing presentation, structure, and explanations. I'm 3 years late, but please do keep up the great work!
Now to watch all three parts at once!! Great video 3b1b, keep up the great work man! δ.δ
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
You're saving my presentation! I didn't know how to explain neural networks to my fellow students but now I know how to do it! Thanks mate!
I would expect the training data to also include a bunch of random images that the network could classify as "not a number" (not hotdog 😉) That way, instead of just taking its wildest guess at which number it's detecting, it could also intelligently say "this doesn't look like a number to me"
Nick V Yes, this would probably force it into more logical methods than what it turned out to be last video.
it could also intelligently say "This doesn't look like anything to me."
FTFY
That's a good idea. Also I wanna see what will happen if alphabet and numbers are trained together.
From what i understood i think you could just program it to say that whenever all the values in the last layer are either negative or really low without changing the back propagation process. Not sure tho.
You need to be sure your "random non-number" is actually a lot of different things, otherwise the "not a number" result would have similar chances of being triggered as the number results when for example instead of alphabetical characters you present the NN with white noise it didn't train on.
I would like to take the forum to THANK YOU for your efforts to make the extremely complex topics comprehensible. I am new to the subject and found a very good starting point to start my journey towards understanding a little bit of AI. Thank you very much.
I think it'd be extremely beneficial if you were a complete lecture series for deep learning. Your way of teach is billion times better than this somewhat popular Andrew Ng lecture series, and also a lot more intuitive. Please consider this:)
Best wishes
Sanderson>>> Ng
WHY IS HIS VOICE SO CALMING???
I've never clicked on a video this fast
What for did you clicked on the video?
I am writing my internal assesment for school about neural networking and thank god your channel exists cuz everything about this subject is now clear to me. You explanations are perfect!
Let's just say a quick thank you to all the mathematicians who wrote the highly optimized software libraries that do all this math for us so ordinary programmers can quickly get neural nets up and running on our training data.
1.3 million views ... wow. I'm kind of blown away by how many people are interested and following this series. That's far more than I would have guessed! Great!
Oh dear! Absolutely brilliant. This is one heck of video.
How would you consider creating an exhaustive ML and Neural Network video series? I'm sure there are a lot of curious folk out there, waiting for it!
It’s 12 midnight and I have been reading articles on this all day, thank you for helping me!
You guys just gotta stop blowing my mind ....💥
I cannot overstate how much I wish I had these videos during my machine learning course a few years back
OMG just noticed that Markus Persson (Notch - Creator of Minecraft) sponsored this video.
Oh honey.
The 3b1b comment section has been going crazy about this for years.
fossilfighters101 It's the first time I noticed it. It's nice to see famous people supporting content creators that educate people
It's a quite common Swedish name, so I wouldn't be to sure it's Notch.
NOTCH SPONSORED 3B1B OMFG
Thank you so much for the simplicity in your explanations. You try to explain everything not by their definition, but with an understandable, simple way. Thank you!
Impressive way of presenting complex information. Your teaching is really an art. Thank you!
Not drunk, just stochastic
"I recently created my own UA-cam channel, and I’m currently a private student. These days, I feel like I have little formal education, but through this channel, I’ve been learning a lot and gaining knowledge. It feels like I’ve entered a new world where I can study from home. This whole experience has been very inspiring for me."
I might misunderstand but maybe not. At 7:38, we want to reduce the activation of the neuron responsible for 3 right? So we should decrease the activation of the neurons that have a positive weight to 3, no? In the video, it's actually the opposite. For example, the first neuron has a positive weight with the neuron responsible for 3, so we should decrease its activation right?
Can someone help me on that please?
I had the same question. It seems like the color-coding of the video at this point is inconsistent? (it applies the same procedure for adjusting the preceding layer of neurons to all the other output neurons as it does to the output neuron for 2, when really the signs in the procedure should be reversed for all the neurons besides 2, since we want their activations to decrease?)
same here, i'm quite confused about this
You have no idea how useful this video is to me. I learnt deep learning at uni but honestly this has just helped me through some hard time!
Why don't you have ads on these. I have no money for patreon and I want to help you out
+TheWingDings1 I just think it's a nicer experience. If you have no money, don't worry about it, just watching is thanks enough.
I know what you mean, the best thing you can do to help the channel is by subscribing,liking, and sharing videos as cliche as it sounds it does help creators, not a creator myself just someone who wants to see these kinds of educational channels grow!
@@johncoleman1930 I have subscribed, I like every 3blue 1brown video I see. But, I don't know any one who is interested in this kind of stuff. So, I can't really share
This is such a wholesome exchange!🥺 Thank you so much Grant and I wish the best for you and the others who commented here!
I'm a senior in my computer science major and honestly I was starting to get very burnt out of it. I used to have a lot of love for math and computer science but the cumulative effect of classes rampaging through topics without giving a thorough explanation really started dimming that love. I'm in an AI class this semester and while the professor is incredible at what he does, he sadly has no idea how to teach the concepts to new students. My motivation was at an all time low and I simply went to class as routine rather than to get knowledge I care about out of it. Your video series have reinvigorated my passion for these subjects and to that I thank you. Rather than fearing and dreading what's to come I now look forward with enthusiasm.
At 7:37, shouldn't the activations associated with the digit 0 neuron (whose activation should be decreased) change the opposite direction? For example, if the neuron at the very top of the second-to-last layer increases activation, with a positive weight, the digit 0 neuron's activation would increase. Is that correct?
Same question here...Someone has an answer for that...?
After doing ur linear algebra course, it feels good to watch en episode of something you intuitively understand way more
Hello,
I'd like to start by saying your videos are amazing and beyond intersting.
One thing I'm still confused about is how to update the biases through backpropagation?
Thanks! and keep it up!
This!
I also don't get it what these "nudges" actually mean.
Do I simply add value from computed gradient to the weight?
I had to stop seeing your video for a moment because I needed to write down my admiration for your work. It is outstanding the amount of intelligence, communication skills, careful graphic design and passion in very single frame of your videos. Congratulations and thank you for your contribution to maths education. See you in patreon!
I'm just starting out with Deep learning this quarantine and man...it's a bit difficult to get my head around these concepts.
Me too man! I am doing the course by Andrew Ng and things are getting much clearer.
@@JonesDTaylor Me too 😁!
@Winston Mcgee hey..why??
@@JonesDTaylor same!
Include me too in your category...
I changed my major from cybersecurity to philosophy and mathematics because of these videos. This is much better than any college course I’ve taken.
I feel like I'm missing something with the functionality of backpropogation.
So, I get that you adjust the weights between the last two rows to adjust how the second last row feeds into the last row in the optimum way, and then you change the second last way, and then you backpropogate the second row of weights away from the end, etc. But, what I don't understand is that if you change the second last weights, that's going to change the second last row of inputs, which means that the tweaked weights between the last hidden layer and the output will be wrong again. I'm trying to work out what I'm missing. Is it that all the layers of weights are tweaked simultaneously in order to be accurate? If so, I'm still not sure I understand how that works (I'll probably try and watch the video again in the morning when my brains properly awake.)
+Conor O'Neill Good question! Backpropagation is just giving the list of adjustments you will make, but you don't make them until all are computed. And of course, since this is part of (stochastic) gradient descent, that's a process you repeat multiple times.
You bring up a really good point. The next video, which will cover the actual mathematics of how backpropagation works, does take that into account. If you've taken calculus before, you've heard of the chain rule, which describes how to differentiate an expression like f(g(x)). The chain rule states that d/dx f(g(x)) = f'(g(x)) * g'(x). The chain rule is general enough that it's applicable to pretty much all situations where you can express stuff in terms of an f and g, even when f and g take multiple inputs. The important thing is that it allows you to only analyze one input at a time, making it easy to compute stuff. What happens in the case of backpropagation is that you're taking the derivative in terms of each of the weights of the cost function, and the chain rule accounts for how changes affect other changes. The math is too complex to put in a youtube comment (mainly because youtube doesn't have mathjax, so I can't write a lot of the symbols I'd need, like summations), but the next video should explain all of that!
Thanks! That helps! Thanks for a great video - I love your work and the effort and care you put into these videos. Truly incredible. Thanks again.
Consider that the changes merited by each example are quite small (because each example alone doesn't carry enough weight to merit making big changes), so the changes of weights based on these values are still accurate. It doesn't matter if you do a tiny step after each example, or a larger one after a thousand examples, the over all effect of each example is still like a single tiny step in the direction it points to.
I love that Fermat is labeled as ,,Tease"
10:11 it means drunk people reach their destination faster.
Obviously this implies that drunk driving is the quickest and most efficient mode of transportation available.
it implies drunk people stumble down hills faster, which is probably true, but the result is less pretty
I have a masters in CS and learned quite a bit about Machine Learning in college but ironically never learned the math behind any of it. This is a brilliant video and I can’t wait to dig into the calculus of it all!
AAAAAAAAA YEEEESSSS
I'VE BEEN WAITING FOR THIS VID
This was the most life-changing of 12 minutes in my whole comp-sci university studies..
Dude how are you making these animations
he makes them in python
Dr tenma how are u