How Gradient Descent Works. Simple Explanation

Video explain what is gradient descent and how gradient descent works with a simple example. Basic intuition and explanation are revealed in the video.The content is:
0:09 - What is Gradient Descent.
0:30 - Example
0:39 - Step no. 1. Start with a random point and find the gradient (derivative) of the given function.
1:28 - Step no. 2. Set learning rate to get know how big should be a step to move forward on gradient descent to the opposite direction.
1:47 - Step no. 3. Perform calculations on iterations.
1:58 - Initialize parameters.
2:20 - calculations on the 1st iteration.
3:20 - calculations on the 2nd iteration.
4:26 - till we reach global or local minimum.
I hope that this explanation on how gradient descent works is useful for beginners of deep learning or good reminder for machine learning / deep learning experts.
#deeplearning #gradientdescent #ai

[@lovekildetoft5658]

All other videos on gradient descent are atleast 20 minutes long. This one is five, and made me understand more than any of those videos. Thank you!

[@DataScienceGarage]

Thank you for watching! Hoping it was useful.

[@blendaguedes]

Sometimes we just need two loops to understand a whole. Thank you!

[@uncoded0]

Thank you! Many hours of trying to understand gradient decent, now I finally get it, thanks to this video. Thank you!

[@anamikabhowmick6322]

This is one of the best and easiest way to learn and understand gradient descent, thank you so much for this

[@DataScienceGarage]

Glad you liked it! :)

[@impzhu3088]

That’s the way to explain a concept! Example with detailed steps. Thank you so much!

[@DataScienceGarage]

Thanks for watching! :)

[@riki2404]

Thank you for such a clear explanation. Short and precise. no unnecessary talk.

[@DataScienceGarage]

Thanks for watching! Hoping it was useful :)

[@DataScienceGarage]

If you found useful in this video I highly recommend to check other related ones:
-- Calculate Convolutional Layer Volume in ConvNet (ua-cam.com/video/3uEd0ErqGzU/v-deo.html)
-- Adam. Rmsprop. Momentum. Optimization Algorithm. - Principles in Deep Learning (ua-cam.com/video/YacPECoI5SY/v-deo.html)
-- Numpy Argsort - np.argsort() - function. Simple Example (ua-cam.com/video/6W8UHvn8ckg/v-deo.html)
-- Python Regular Expression (RegEx). Extract Dates from Strings in Pandas DataFrame (ua-cam.com/video/E4avBXbNOGc/v-deo.html)

[@wtry0067]

It's very short and very useful.. I get clarity what I was looking for.
Thanks once again.

[@abdelrahmane657]

Oh my god, you are excellent. You make the difference on UA-cam. Thank you so much. 🎉🙏👏🙌👌👍✌🏼

[@DataScienceGarage]

Thanks you for such feedback, appreciate it! :)

[@hemantsah8567]

It is easy... spent my 2 days on learning gradient descent.... then I came to your video... Thanks bro

[@aalishanhunzai]

Bro thank you so much for your efforts, couldn't find a more simple explanation of gradient descent than this one.

[@DataScienceGarage]

Thanks for such feedback! :)

[@sukanya4498]

Love this video ❤️, Very simple and precise! Thank you !

[@DataScienceGarage]

Thanks for watching! :)

[@Elementiah]

Thank you so much for this! This is the perfect explanation! 😄

[@TingBie]

Thanks for this example, simple and spot-on!

[@Alex-pd5xc]

wow dude, very clearly explained and you made it simple for me to understand. cheers man

[@DataScienceGarage]

Thanks for such feedback, appreciate!

[@samvanoye]

Perfectly explained, thanks!

[@DataScienceGarage]

Thanks for watching! :)

[@alexbarq1900]

I get the idea but any reason for not doing simple math to find the local min?
dy/dx = 2(x+5)
If we want to find the min, we just do dy/dx=0.. then:
0 = 2(x+5)
x = -5

[@glenfernandes253]

how do you know, how many iterations to run before reaching the global/local minimum, what if it reaches the minimum and starts climbing on the other side ?

[@strykeregziadahmed9562]

2 hours in DL course i didnt get it
5 min made my day this is how actually learning should be

[@DataScienceGarage]

Glad it was helpful for you! :)

[@muhammadhashir7949]

Thank you so much your work was practical and I loved it alot and underestood gradient descent. Before that I spent lots of time but didn't understood it properly

[@phaniraju0456]

I bow to your for this great clarification ..loved it

[@josephsmy1994]

Awesome explanation! straight to the point

[@DataScienceGarage]

Thanks for such feedback! :)

[@eramitvajpeyee85]

Thank you so much for explaining it in short and easy way!! Please keep uploading content like this.

[@DataScienceGarage]

Thank you for watching! Glad you enjoyed! :)

[@yaminikommi5406]

We can take any number as intial parameters and learning rate

[@mattk6182]

using x as your means of showing multiplication is confusing, makes it looks like you took the derivative wrong with 2x(x+5)..maybe in future videos either leave the x out so the multiplication is implied.

[@zafarnasim9267]

You made it so simple. Great Job!

[@DataScienceGarage]

Thanks a lot! :)

[@sanurcucuyeva7040]

Hi, thanks for explanation. If our function is hard, at what point in the iteration should we stop to find the minimum point

[@murat2073]

thank you Sir! you are a HERO!!!

[@DataScienceGarage]

Thanks a lot! :)

[@twicestay6683]

Thx a lot!!! But I'd like to ask why the learning rate=0.01? is it a random number? Thx

[@michaelscott8572]

What I don't get is: When we use this method in neural net, we don't know the Errorfunction. We just have some point. So how can I build the derivative?

[@omkarkadam5715]

Thanks mate, Finally Enlightened.

[@DataScienceGarage]

Thanks for watching! I hope it was useful :)

[@smurfNA]

hey! so do we choose the learning rate? and the gradient is simply just the function right ?

[@abdellatifmarghan7521]

Thank you. grateful explanation

[@DataScienceGarage]

Glad it was useful! :)

[@mohamedelkhanche707]

ohhhhh wonderful i was chocked this is insane thank you from all my heart

[@DataScienceGarage]

Thanks a lot for such feedback, appreciate!

[@pwan3971]

Thanks a lot, really appeciate the video, this makes so much sense now

[@DataScienceGarage]

Thank you for you feedback! Glad it was useful for you :)

[@yasamannazemi6706]

It was so simple and helped me a lot :)
Thanks👍🏻

[@DataScienceGarage]

Thanks!

[@praneethcj6544]

Simple and clear ... Yet need more detailing ...!!!!

[@Slendich]

Really great and simple explanation. Thank you

[@DataScienceGarage]

Thanks for sich feedback, I'm glad it was helpful! :)

[@ajaykushwaha4233]

Best explanation ever.

[@luisurena1770]

Coñazo siempre hay un indu que me ayuda a entender todo🔥🔥🔥🔥

[@abdanettaye8217]

good starting, thank you

[@hindbelkharchiche1654]

Thank you .. the explanation is as simple as useful .

[@blinky1892]

How do we know what the y value is of the parabole at any given x?😊

[@mbogitechconpts]

Beautiful video. I have to like it.

[@DataScienceGarage]

Thanks for feedback, inspiring!

[@nawaab9275]

thanks for saving the semester

[@fmikael1]

Thanks for the great explination. everyone always complicates it

[@DataScienceGarage]

Thanks for feedback, glad it was helpful! :)

[@SuperYtc1]

This is a good video.

[@DataScienceGarage]

Thanks!

[@radhar5349]

Great explanation. Easy to get the concept

[@DataScienceGarage]

Thanks for feedback! :)

[@eliashossain4327]

Best explanation.

[@DataScienceGarage]

Thanka for such feedback! :)

[@dennisjoseph4528]

Great job of explaining this as simple as possible Sir

[@9891676610]

Awesome explanation . Thanks a lot !!

[@DataScienceGarage]

Thanks for watching! Hope it was useful!

[@basheeralwaely9658]

Well done sir, very easy to understand

[@tevinwright5109]

GREAT VIDEO

[@DataScienceGarage]

Thanks for watching this!

Perfect !

[@bharatcreations7154]

Can we compute same thing without getting into learning rate??

[@george4746]

Thanks, It was very clear and concise.

[@DataScienceGarage]

Thanks!

[@RayhanAhmedsimanto]

Amazing Practical Explanation. Great work.

[@DataScienceGarage]

Thanks Rayhan!

[@bhavikdudhrejiya4478]

Very good video. I appreciate your hard work. Keep uploading more videos.

[@DataScienceGarage]

Many thanks for such comment!

[@supantha118]

Thank you so much

[@sandipmaity2687]

Amazing Explanation :) Really simple and to the point 😀

[@colton3000]

How do we find learning rate?

[@denisplotnikov6875]

How to use this example for Stochastic Gradient Descent?

[@machinelearningid3931]

Thanks, this give me the light in darkness.

[@TheJayenz]

Thank you so much!

[@DataScienceGarage]

Thanks for watching! :)

[@Snetter]

Nice work! thanks

[@DataScienceGarage]

Thanks for feedback, glad for this! :)

[@muhammadhilmirozan1266]

thx for explanation!

[@DataScienceGarage]

Thanks for watching! :)

[@thankyouthankyou1172]

Useful thank you

[@DataScienceGarage]

Thank you for watching!

[@mastan775]

Very good explanation...thanks a lot.

[@ericklestrange6255]

didnt explain how do calculate the direction we are moving to (the minus), why the derivatives etc

[@ak-ot2wn]

That's what I am looking for already for several days and nobody mentions this. Anyways I still think, that it is trivial. If your derivative is negative, you have to "move" to the right side (in case of 2 variables). If it is positive, you have to "move" to the left.

[@debayondharchowdhury2680]

he also didn't talk about loss calculation. why do we need to calculate loss at all if we can simply use the gradient descent on the function.

[@blendaguedes]

@@debayondharchowdhury2680 Your loss function is the one pointing the difference between your output and your "y". You calculate the gradient to your loss function. At his example, he shows something that looks like a ' mean squared error' as loss function to me, and he is doing a linear regression with only one input "x".
I recommend you the Andrew Ng classes on Coursera. have a good time

[@blendaguedes]

@@ak-ot2wn I totally agree with what you are saying, the only matter is when you are programming you don't see witch direction your vector is going. So basically if the error is going down: keep going, if it starts to increase go back. You can just stop, or you can make your learning rate smaller to increase your accuracy

[@rssaiganesh]

I think the comments thread is looking for the math behind the formula for the gradient descent. Apologies if I misunderstood. But here is a link that helped me: towardsdatascience.com/understanding-the-mathematics-behind-gradient-descent-dde5dc9be06e

[@user-qj1lm1xh2z]

Well done 👏

[@DataScienceGarage]

Thanks! :)

[@arvinds7182]

On point👏

[@DataScienceGarage]

Thanks!

[@shankaks7217]

Why did we choose 0.01 as the learning rate?

[@davidbarnwell_virtual_clas6729]

How do we choose the learning rate? Good video but it's things like that I'd love to know

[@DataScienceGarage]

Hi! Choosing learning rate often is not easy task. I usually makes experiments on model performance with multiple learning rate (manual, Grid search hyperparameter tuning, Bayesian search, etc.).

[@davidbarnwell_virtual_clas6729]

@@DataScienceGarage Ahh...ok...I get you...it's very interesting.

[@gerleenjosuegoya8777]

Thank you!

[@DataScienceGarage]

Thank you for watching!

[@boniface385]

Hi, why the learning rate are 0.01? Can it be any random learning rate? For example 0.2, 0.02 or any. I appreciate it for thee fast reply, thank you😊🙏🏻🙏🏻🙏🏻

[@DataScienceGarage]

Hello! Thanks for watching this video, I'm glad it was useful for you. While modelling ML system, you can specify random Learning rate. However the good practice is to use 0.1, 0.01, 0.001, or 0.0001. Each ML model has its own architecture, and different training data, hyperparameters, etc., so learning rate can be adopted separately for each case.
Here, I used 0.01 just for demonstration purposes.

[@boniface385]

@@DataScienceGarage thank you so much for the explanation. 🫶🏻

[@dveerraju1852]

How can you know the learning rate

[@ydkmusic]

Great video! There is a typo around 3:50. The bottom equation should be x_2 = .... instead of x_1.

[@davidkayode6679]

Wonderful Video!!! Thank You!

[@DataScienceGarage]

Thanks for feedback!:)

[@kronlogic2408]

For the Iteration 2, shouldn't the second line be x2= and not x1= ?

[@karthiklogan9384]

really helpful sir.thank you so much

[@DataScienceGarage]

Happy that was useful.

[@darkman8939]

thanks, very hhelpful.

[@grinfacelaxu]

Nice!

[@DataScienceGarage]

Thanks!

[@bhavya2301]

Thankyou.

[@seathru1232]

GREAT!

[@DataScienceGarage]

Thanks!

[@emrecik9882]

Thanks

[@bernardaslabutis5098]

Ačiū, padėjo!

[@DataScienceGarage]

Džiaugiuosi! :)

[@AJ-et3vf]

Very useful! Awesome ❤️

[@DataScienceGarage]

Thanks for watching! :)

[@pearlsofwisdom2416]

Good explanation but would have been better if you elaborated its formula of why it is used to reach next step. Why is derivative multiplied by learning rate and why it is then substracted from first point value

[@blendaguedes]

The learning rate makes the "decay slow". At his first interaction, the result would be: -3 -4 = -7.
Can you see where this is going? As he goes slow he will keep dropping his "y", until he get to as close as possible to -5. Sometimes to get at the minimum you have to make you learning rate smaller while computing your weights .

[@Hasasinful]

Thanks just what i needed

[@DataScienceGarage]

Hope it was useful. Thank you!

[@diegososa5280]

Thank you very much!

[@DataScienceGarage]

Thanks! Hoping it was useful. :)

[@MuditDahiya]

Very nice explanation!!

[@DataScienceGarage]

Thanks!

[@mohsinjunaid8454]

thanks

[@govardhan3099]

Great explained...

[@DataScienceGarage]

Thanks!

[@harshithbangera7905]

How we know -5 is global minimum...is there when gradient or derivative become 0

[@explovictinischool2234]

Hello, better now than never.
Let's assume we have reached -5 at step Xn. However, we don't know that we have reached the local minimum.
We perform another step Xn+1 with the formula, which gives:
Xn+1 = Xn - (learning_rate) * (dy/dx)
Xn+1 = -5 - (0.01) * (2 * (-5+5))
Xn+1 = -5 - (0.01) * 0
Xn+1 = -5
And so, we have Xn+1 = Xn which means we can not progress anymore and which means we reached the local minimum.

[@dtakamalakirthidissanayake9770]

Thank You So Much. Great Simple Explanation!!!

[@jimyang8824]

Good explanation!

[@moazelsawaf2000]

Thanks a lot sir

[@AlfredEssa]

Good job!

[@codingtamilan]

How you draw that curve can be fixed as -5 ?
Always it is centre as -5 ?

[@blendaguedes]

First you decides witch will be his loss function. On his case it was (5+x)^2, or x^2 + 10x + 25. Nos you program the gradient descent to find the minimum of the function. It depends of your function.

[@codingtamilan]

@@blendaguedes thq... pleasure to meet you

[@MrAnindyabanerjee]

Thank you