this course is so nice you are simplifying complex ideas in a really nice way, I think this series should be binge watched like a netflix series to refresh your memory on all these concepts XD
Backpropagation explained | Part 1 - The intuition ua-cam.com/video/XE3krf3CQls/v-deo.html Backpropagation explained | Part 2 - The mathematical notation ua-cam.com/video/2mSysRx-1c0/v-deo.html Backpropagation explained | Part 3 - Mathematical observations ua-cam.com/video/G5b4jRBKNxw/v-deo.html Backpropagation explained | Part 4 - Calculating the gradient ua-cam.com/video/Zr5viAZGndE/v-deo.html Backpropagation explained | Part 5 - What puts the “back” in backprop? ua-cam.com/video/xClK__CqZnQ/v-deo.html Note, at 7:44, I misspoke when I stated that the updated values we get for the weights are the the corresponding derivatives of the loss function with respect to each weight. Actually, the updated values themselves are *not* the derivatives. Rather, after calculating the derivatives, the weights are updated to their new values, which are calculated *using* the derivatives we obtain. This process of updating the weights is covered in more detail in the following video. This particular detail is mentioned at 1:26: ua-cam.com/video/_N5kpSMDf4o/v-deo.htmlm26s Machine Learning / Deep Learning Fundamentals playlist: ua-cam.com/play/PLZbbT5o_s2xq7LwI2y8_QtvuXZedL6tQU.html Keras Machine Learning / Deep Learning Tutorial playlist: ua-cam.com/play/PLZbbT5o_s2xrwRnXk_yCPtnqqo4_u2YGL.html
your videos are actually great in expressing the concepts in possibly quickest time . i havnt seen your other playlists yet. but this one really helped me so much.
Omg Thank You! All these books make it so complex and include a bunch of unnecessary equations before a noob like me can actually understand the process. I am interested in the math as well but first I have to know the concept, otherwise I often find myself understanding these complex details and be like, why exactly are we doing this? This video actually makes sense and now I can dive deeper into it. Nice animations too. Props to you guys
after watching sooooooo many videos on this topic yours was EASILY the most helpful by far. It is accurate, simple, and just all around perfect. the image is very simple. I love your voice and the explanation is so good.
I cannot thank you enough. I don't know how many videos I watched to clear this concept, nothing helped but your video. You explained it in a very detailed and clear way. Thank You so much!
Thank you very much for this video! It was fantastic that I could finally understand how SGD uses back propagation to calculate the gradients to minus off!
Thanks, cedric! The full math for backprop is covered in the videos following this one! :) They are currently #28 - 31 in this playlist: ua-cam.com/play/PLZbbT5o_s2xq7LwI2y8_QtvuXZedL6tQU.html
I already know the math behind back propagation; however, I will watch your videos in order to see how you are going to present it. Your way of giving the information is awesome. I want to see how it will work with the math complexity.
@@deeplizard To be honest, as I mentioned earlier, I already know the math behind the back propagation, yet your videos were a good refresh to me. You used the same mathematical notation and the same methodology of my teacher. he started with the last hidden layer; after that, he generalized to any hidden layer. Thanks a lot for your videos.I already finished the math videos and I will finish this list today. In the future, I am going to watch tensorFlow.js series
Great to hear, Mohamed! Thanks for letting me know. Would love to hear how your progression is going in the TensorFlow.js series as well once you start!
Yess.. Adding a specific Playlist with Maths behind all the functionality with an example (Like uh showed for max pooling which was visible to see hoe things are working with input) would be really great.. Bur still appreciate your efforts in helping students by such a great videos..
Hi, Thanks for super clean way to explaining basic concepts. Keep it up. Request you for following video series 1. Complete Maths and Stats for ML/Deep Learning. 2. Pl have additional series on calculus on backpropogation. 3. Make complete new series on ML/Deep Learning Practical projects rather than Housing Price Prediction, Titanic, iris, Hand written digits ....etc
{ "question": "Since the derivative of the loss function is calculated with respect to the weights of the model during backpropagation, what characteristics should the loss function have?", "choices": [ "Continuous", "Discontinuous", "Constant", "Zero" ], "answer": "Continuous", "creator": "Fahim Mahmud", "creationDate": "2020-02-12T04:10:35.701Z" }
At 8:00 you say that the update values of the weights are equal to their derivatives (gradient). But since we are talking about a loss function, they should be equal to de NEGATIVE gradient instead?
Thanks for the awesome video on backpropagation Yes i want to know the math behind backpropagation as the whole logic lies in and around math.So kindly make a video on the math behind the Backpropagation.
You're welcome, krishnasai! There are 4 episodes following this one that explain the math :) And they all have corresponding blog articles along with the video! The next one starts at the link below, and the following 3 are directly after. deeplizard.com/learn/video/2mSysRx-1c0
Amazing... Before asking about the math courses for the backpropagation. Is it essential to study the math behind deep learning in order to work with Keras or any other APIs? Thanks for these fruitful videos!
I have a similar question: Why can't we simply change the weights between the blue and the yellow nodes and arrive at the optimal values for the hidden layer units?
Is it the case that in order to change the nodes in each layer, we need to change BOTH the nodes in the previous layer as well as the weights between the previous layer and the current layer?
Hey richard - They are the following videos in this playlist. Check out the full series in order here: deeplizard.com/learn/playlist/PLZbbT5o_s2xq7LwI2y8_QtvuXZedL6tQU
Something isn't clear tome. Do we update the bias for each neuron or is this example without using bias? Apart from that, I love your videos and the way you explain things! Please keep up since this is a highly interesting thing. :)
i'm confused about the difference between the gradient and derivative? Is it called a gradient of the loss function when you average out all of the derivatives of the loss wrt to a certain weight? Meaning that its called a derivative when its just dloss/dweight for a single sample, and its called a gradient when its the average dloss/dweight for all of the samples? thank you for this amazing series btw it is the best on the internet
Hello :D Is the output of the activation equal to the weight ? i mean is the activation the weight? of every neuron firing? right? cause i think that's what I remember about your video on activation function so basically the output of the activation function is going to be the weight? thank you so much great tutorial ! you explain it at the simplest way :))
Love your videos, really helping me with my master thesis! :) You're talking about SGD but there are also other optimizers. I'm interested especially in Adam, and how it differs from SGD, maybe you got some paper or article recommondations? - In my understanding Adam is doing the exact same thing as SGD just using another algorithm, is that correct? - You're talking about backpropagation referring to SGD, is Adam also using backpropagation? In my understanding, backpropagation ist just the general term of changing the weights after every forward propagation.
Hey Ayan - The following two videos (in the order listed) talk about SGD: 1. ua-cam.com/video/sZAlS3_dnk0/v-deo.html 2. ua-cam.com/video/_N5kpSMDf4o/v-deo.html Note that SGD uses backpropagation during training, which is where most of the "grunt work" comes into play. So after generally understanding what SGD is doing from the above videos, this 5-part backprop series, starting with the video we're currently commenting on, gives all the details for what backprop is doing during the training process.
Hey Torbjorn - It occurs at each batch. The details for this implementation are covered in the backprop videos that come after this one in the playlist.
The math for backprop starts in the next video in the playlist! Here are the full details for the backprop series: Backpropagation explained | Part 1 - The intuition (this video) ua-cam.com/video/XE3krf3CQls/v-deo.html Backpropagation explained | Part 2 - The mathematical notation ua-cam.com/video/2mSysRx-1c0/v-deo.html Backpropagation explained | Part 3 - Mathematical observations ua-cam.com/video/G5b4jRBKNxw/v-deo.html Backpropagation explained | Part 4 - Calculating the gradient ua-cam.com/video/Zr5viAZGndE/v-deo.html Backpropagation explained | Part 5 - What puts the “back” in backprop? ua-cam.com/video/xClK__CqZnQ/v-deo.html
@@deeplizard rest all the videos are very very good. The way everything is explained is awesome!. Better that course era. One last things, are there any videos in which the programming part is explained or the dimensions of the error or Delta, and the dimensions of theta?
A video for this a video for that!! Just teach or do a brief sum of what this is, because if I want to search more I will do it from other videos not just yours.
This video deserves to be on television at dinner time.
this course is so nice you are simplifying complex ideas in a really nice way, I think this series should be binge watched like a netflix series to refresh your memory on all these concepts XD
00:00 Intro
01:23 Recap stochastic gradient descent (SGD)
04:42 What is backpropagation, the intuition
08:28 Summary
Added to the description. Thanks so much!
I have just completed watching all the 27 videos in the list. All are excellent videos!! Thanks for uploading. Please continue the good work!! Thanks!
Hey Parthasarathy - That's great to hear that you've gone through the entire playlist! I'm glad you're enjoying them!
Backpropagation explained | Part 1 - The intuition
ua-cam.com/video/XE3krf3CQls/v-deo.html
Backpropagation explained | Part 2 - The mathematical notation
ua-cam.com/video/2mSysRx-1c0/v-deo.html
Backpropagation explained | Part 3 - Mathematical observations
ua-cam.com/video/G5b4jRBKNxw/v-deo.html
Backpropagation explained | Part 4 - Calculating the gradient
ua-cam.com/video/Zr5viAZGndE/v-deo.html
Backpropagation explained | Part 5 - What puts the “back” in backprop?
ua-cam.com/video/xClK__CqZnQ/v-deo.html
Note, at 7:44, I misspoke when I stated that the updated values we get for the weights are the the corresponding derivatives of the loss function with respect to each weight.
Actually, the updated values themselves are *not* the derivatives. Rather, after calculating the derivatives, the weights are updated to their new values, which are calculated *using* the derivatives we obtain. This process of updating the weights is covered in more detail in the following video. This particular detail is mentioned at 1:26: ua-cam.com/video/_N5kpSMDf4o/v-deo.htmlm26s
Machine Learning / Deep Learning Fundamentals playlist: ua-cam.com/play/PLZbbT5o_s2xq7LwI2y8_QtvuXZedL6tQU.html
Keras Machine Learning / Deep Learning Tutorial playlist: ua-cam.com/play/PLZbbT5o_s2xrwRnXk_yCPtnqqo4_u2YGL.html
Yes i want to know on the math behind backprop
Me too!
Jan ke kia karo ge
your videos are actually great in expressing the concepts in possibly quickest time . i havnt seen your other playlists yet. but this one really helped me so much.
There is no limit to the awesomeness of her explanation!
there's a whole team behind this
she is just the voice (but she does a great job at it)
Omg Thank You!
All these books make it so complex and include a bunch of unnecessary equations before a noob like me can actually understand the process.
I am interested in the math as well but first I have to know the concept, otherwise I often find myself understanding these complex details and be like, why exactly are we doing this?
This video actually makes sense and now I can dive deeper into it. Nice animations too. Props to you guys
You're welcome, Zi! Glad you found it helpful. The math explanations are in the following videos of this playlist.
after watching sooooooo many videos on this topic yours was EASILY the most helpful by far. It is accurate, simple, and just all around perfect. the image is very simple. I love your voice and the explanation is so good.
I cannot thank you enough. I don't know how many videos I watched to clear this concept, nothing helped but your video. You explained it in a very detailed and clear way. Thank You so much!
Bless you for making the world a better place. Keep up the good work!
This is an amazing video and I understand the deep learning instantly!
Just can not THANK YOU enough for this Greatest of videos on the topic. Just Brilliant.
YES YES YES YES YES YES WE NEED MATH TOPICS
Thank you very much for this video! It was fantastic that I could finally understand how SGD uses back propagation to calculate the gradients to minus off!
This was a very clear explanation of what happens during back propagation. The next step is to provide the math. Thank you for your effort. Great Job.
Thanks, cedric! The full math for backprop is covered in the videos following this one! :)
They are currently #28 - 31 in this playlist:
ua-cam.com/play/PLZbbT5o_s2xq7LwI2y8_QtvuXZedL6tQU.html
deeplizard I will watch them thanks again
I already know the math behind back propagation; however, I will watch your videos in order to see how you are going to present it. Your way of giving the information is awesome. I want to see how it will work with the math complexity.
Thanks, Mohamed! Let me know your thoughts after you finish the following math videos.
@@deeplizard To be honest, as I mentioned earlier, I already know the math behind the back propagation, yet your videos were a good refresh to me. You used the same mathematical notation and the same methodology of my teacher. he started with the last hidden layer; after that, he generalized to any hidden layer.
Thanks a lot for your videos.I already finished the math videos and I will finish this list today. In the future, I am going to watch tensorFlow.js series
Great to hear, Mohamed! Thanks for letting me know. Would love to hear how your progression is going in the TensorFlow.js series as well once you start!
yes math too please.
Thanks you very much for making this great videos. And I want to know more about maths
Yess.. Adding a specific Playlist with Maths behind all the functionality with an example (Like uh showed for max pooling which was visible to see hoe things are working with input) would be really great.. Bur still appreciate your efforts in helping students by such a great videos..
Thanks, deepak. The next four videos in this playlist show the full math behind backpropagation. Let me know how it goes as you progress through them!
thank you very much for this clear and helpful explanation.
Words fail to express my gratitude.
You are truly a fantastic teacher :)
Hi, Thanks for super clean way to explaining basic concepts. Keep it up. Request you for following video series 1. Complete Maths and Stats for ML/Deep Learning. 2. Pl have additional series on calculus on backpropogation. 3. Make complete new series on ML/Deep Learning Practical projects rather than Housing Price Prediction, Titanic, iris, Hand written digits ....etc
great..explaination...simple and effective
Hey! What's going on everyone? :D
Great content!
Amazing, love your videos, will recommend it to all my friends
Thank you, Robert!
such an awesome super simple explanation! and how you bring up the math, it's almost like a teaser :) now I want to see the Math! =)
These are excellent videos. I do worry about biases not being discussed in later videos as these are also being updated.
Great video I wanna see the math. Thank you for uploading this one.
Great :D It's in the following episodes. Starting with this one:
deeplizard.com/learn/video/2mSysRx-1c0
Wow that was exciting Hi. Thanks for sharing these videos. And Also we want to see the maths.
{
"question": "Since the derivative of the loss function is calculated with respect to the weights of the model during backpropagation, what characteristics should the loss function have?",
"choices": [
"Continuous",
"Discontinuous",
"Constant",
"Zero"
],
"answer": "Continuous",
"creator": "Fahim Mahmud",
"creationDate": "2020-02-12T04:10:35.701Z"
}
Thanks, Fahim! Just added your question to deeplizard.com/learn/video/XE3krf3CQls :)
Thank you so much for this! Its concise and well explained.
You're very welcome, Parvez!
At 8:00 you say that the update values of the weights are equal to their derivatives (gradient).
But since we are talking about a loss function, they should be equal to de NEGATIVE gradient instead?
Never thought I'd hear the words "chain rule" outside of my old high school AP calc class
🤓
Math too please
interesting
please make a series on videos explaining math behind machine learning and deep learning in particular. Thanks for Amazing videos.
Hey Faraaz - You're welcome! Did you see the next four videos in the playlist that cover the full math for backprop?
great video - yes I want to see the math! 🙂
Yes, please explain math behind of this calculation, thanks for explaining deep neural network concepts
Great :D The math is included in the following episodes. Starting with this one:
deeplizard.com/learn/video/2mSysRx-1c0
Thanks for the awesome video on backpropagation Yes i want to know the math behind backpropagation as the whole logic lies in and around math.So kindly make a video on the math behind the Backpropagation.
You're welcome, krishnasai! There are 4 episodes following this one that explain the math :) And they all have corresponding blog articles along with the video!
The next one starts at the link below, and the following 3 are directly after.
deeplizard.com/learn/video/2mSysRx-1c0
I noticed a broken link in the article for this video, near the words:
"shown in an [earlier post]."
It looks like part of the url got duplicated.
Fixed, thanks Chris! :D
Awesome video. please a brief example of the math behind.
The math is shown in the following four videos of the series :D
great videos,,saveed,subscribed ,
Awesome, thank you!
Thank you so much
Amazing...
Before asking about the math courses for the backpropagation. Is it essential to study the math behind deep learning in order to work with Keras or any other APIs?
Thanks for these fruitful videos!
Why cant we just change the weights from pink to blue nodes since we just want activation in blue nodes to change to be efficient
I have a similar question: Why can't we simply change the weights between the blue and the yellow nodes and arrive at the optimal values for the hidden layer units?
Is it the case that in order to change the nodes in each layer, we need to change BOTH the nodes in the previous layer as well as the weights between the previous layer and the current layer?
I would greatly appreciate if you could make a series on how to take the derivative of a bunch of layers! Thank you for these videos!
What is the default height and width of a conv_2d filter ?
would this backpropagation technique, therefore, only be usable in supervised learning?
yes we need it
You got it - Math starts in the next episode :D
ua-cam.com/video/2mSysRx-1c0/v-deo.html
Any links to videos that do the complete math calculations ?
Hey richard - They are the following videos in this playlist. Check out the full series in order here:
deeplizard.com/learn/playlist/PLZbbT5o_s2xq7LwI2y8_QtvuXZedL6tQU
Something isn't clear tome.
Do we update the bias for each neuron or is this example without using bias?
Apart from that, I love your videos and the way you explain things! Please keep up since this is a highly interesting thing. :)
Thank you, Xi! I've left bias out of this example, but they get updated at the same time and in the same fashion as the weights.
Thank you for the reply! I've not seen your video to the bias which cleared my question when I commented. :)
i'm confused about the difference between the gradient and derivative? Is it called a gradient of the loss function when you average out all of the derivatives of the loss wrt to a certain weight? Meaning that its called a derivative when its just dloss/dweight for a single sample, and its called a gradient when its the average dloss/dweight for all of the samples? thank you for this amazing series btw it is the best on the internet
Hello :D Is the output of the activation equal to the weight ? i mean is the activation the weight? of every neuron firing? right? cause i think that's what I remember about your video on activation function
so basically the output of the activation function is going to be the weight? thank you so much great tutorial !
you explain it at the simplest way :))
sooo gradient descent is the one who's updating the weights? thank you soo much
Yes it is!
I can only agree with the other comments, knowing the math is neat :D Is there some sort of follow up video of yours?
Ok, I have found the videos discussing the math :D
Love your videos, really helping me with my master thesis! :)
You're talking about SGD but there are also other optimizers. I'm interested especially in Adam, and how it differs from SGD, maybe you got some paper or article recommondations?
- In my understanding Adam is doing the exact same thing as SGD just using another algorithm, is that correct?
- You're talking about backpropagation referring to SGD, is Adam also using backpropagation? In my understanding, backpropagation ist just the general term of changing the weights after every forward propagation.
hi, do you have any lecture on SGD ?
Hey Ayan - The following two videos (in the order listed) talk about SGD:
1. ua-cam.com/video/sZAlS3_dnk0/v-deo.html
2. ua-cam.com/video/_N5kpSMDf4o/v-deo.html
Note that SGD uses backpropagation during training, which is where most of the "grunt work" comes into play.
So after generally understanding what SGD is doing from the above videos, this 5-part backprop series, starting with the video we're currently commenting on, gives all the details for what backprop is doing during the training process.
In which video you were talking about SGD?
Hey Hari - These two:
deeplizard.com/learn/video/sZAlS3_dnk0
deeplizard.com/learn/video/_N5kpSMDf4o
r u bringing RNN tut too?
Hey Raju - Yes, I have RNNs on my list to cover in future videos.
Is the backpropagation being applied only once per epoch? So, if you have 20 epochs you will perform backpropagation 20 times, once per epoch ?
Hey Torbjorn - It occurs at each batch. The details for this implementation are covered in the backprop videos that come after this one in the playlist.
7:17
'ie' is pronounced as 'that is'
not i e itself
Yes please, math is fun.
All the math is in the episodes following this one :D
That is per batch per epoch
same here.
please explain the maths behind this in seperate videos
The math is shown in the next four videos after this one. Let me know what you think!
i want to know about the math
It's in the following episodes after this one :)
We want to Sep the math
The math explanation is in the following four videos of the series :D
i want to know the maths behind
Great! It's in the following episodes. Starting with this one:
deeplizard.com/learn/video/2mSysRx-1c0
And yes, I could go for some math.
The math for backprop starts in the next video in the playlist! Here are the full details for the backprop series:
Backpropagation explained | Part 1 - The intuition (this video)
ua-cam.com/video/XE3krf3CQls/v-deo.html
Backpropagation explained | Part 2 - The mathematical notation
ua-cam.com/video/2mSysRx-1c0/v-deo.html
Backpropagation explained | Part 3 - Mathematical observations
ua-cam.com/video/G5b4jRBKNxw/v-deo.html
Backpropagation explained | Part 4 - Calculating the gradient
ua-cam.com/video/Zr5viAZGndE/v-deo.html
Backpropagation explained | Part 5 - What puts the “back” in backprop?
ua-cam.com/video/xClK__CqZnQ/v-deo.html
Yeah - I found it. Awesome stuff. Thank you!
Awesome, you're welcome!
give maths pls
The math comes in the episodes that follow this one :D
i don't want to know about the math.
Warning: Don't watch parts 2 - 5 of the backprop videos 😜
math please!
Hey Wesley - The math is in the following four videos after this one in the playlist!
The background is so bad. The image and background blend
I agree the image and the background don't have enough contrast with each other. The background has changed in later videos.
@@deeplizard rest all the videos are very very good. The way everything is explained is awesome!. Better that course era.
One last things, are there any videos in which the programming part is explained or the dimensions of the error or Delta, and the dimensions of theta?
A video for this a video for that!!
Just teach or do a brief sum of what this is, because if I want to search more I will do it from other videos not just yours.
What is the default height and width of a conv_2d filter ?