What is great about this particular video is these concepts are explained well in many places like scattered dots , you connected the dots to paint the whole picture . an example for gradient descent included - very helpful .
You are the best. Thank you! Awesome interpretation. To anyone confused, when we should be minimizing based on slope v intercept. Here are the reasons: • Minimization based on slope happens when the angle or steepness of the line needs adjustment to better fit the data. • Minimization based on intercept happens when the line is generally in the right direction but needs to be shifted up or down. In most real-world scenarios, both slope and intercept are optimized together during each iteration of gradient descent. The cost function depends on both parameters, and gradient descent adjusts both to achieve the best fit.
Hi, Your video is helpful for beginners to understand the concept. One suggestion: In the very beginning of the video when you write the equation of your predicted line remember to mark it as y(cap) = mx(i) + c. It is not y(i) which is the actual data point.
I am a beginner and as a beginner I was struggling understand the gradient decent concept. I have seen many videos on gradient decent however all of them skipped explaining the derivative part however you explained it very well both (total and partial) with solution . Thanks!
Man, you've won my heart, you kept it so simple, best way of explaining Gradient Descent. Can you please help me in using learning rate in the equation and number of steps used in gradient descent with an example.
As far as I know, gradient descent doesn't talk about solving 'm' and 'c' directly by putting them into two equations like the way you did it here..Because sometime the expression that we get after differentiation becomes so complex especially with logistic regression, random forest and other complex model in deep learning that solving 'm' and 'c' directly becomes extremely tricky and time complexity becomes very high...So Gradient descent talks about solving 'm' and 'c' by some trial and test method. Starting with some dummy value of 'm' and 'c' and putting those values in the equation of differentiation and check if the value of differentiation(say D0) for those 'm' and 'c' becomes 0 or close to 0. If not, then subtract that differentiation value(D0) from the old value of 'm' and 'c' and get a new 'm' and 'c' and again check if the value of differentiation (say D1) for those new 'm' and 'c' comes close to 0. Continue like this till you find that old 'm' - value of differentiation is very very close to 0. That 'm' becomes your actual 'm' and similarly the same thing to be done for 'c'.
This video was an intuitive way for understanding gradient descent for beginners. Anyways appreciate your time to quote about your understanding of GD.
@@RanjiRaj18 Good explanation. I think you have shown derivation for Ordinary Least Squares method. As far as Machine Learning is concerned it has to be slightly adapted.
He solved m and c in this case because it wasn't a very complex example. Only one independent variable. In a multiple linear regression it would have been much more complex.
will the sign ( direction ) for calculating m and b at last change, from addition to subtraction if we take "y-pred - y" instead of "y - y-pred" lke you have done in cost function ? i saw at few articles where this was mentioned but it was not clear.
Hi @ranjiraj , at 21:37 u have given a wrong explanation in the partial derivative w.r.t C... d/dC (-C) will be -1 ,then why are you treating it as constant whereas in d/dc mx should be taken as constant.
That is a explanation. I have one question, where would the learning rate be actually used in computation. Like in your numerical example. We found the outputs and calculated corresponding m and c. How does the learning rate is catered. Secondly, when we multiply learning rate with derivative, what does it gives us ?
Hi one question here: First derivative: xsquare =2x Second derivative = 2 ( replaced in same location) Third derivative = 0 While applying same in mean square error formula First derivative= I understood square to 2/n(•••) Second derivative: with respective to slope It should be 2/n I=1 to n (yi -xi -c) here I applied second derivative replacing. Since -mxi converts to -xi. But In ur explanation instead of replacing, you brought second derivative to starting as below: 2/n I= 1 to n -xi(yi-mxi-c) In the same way for intercept. One more At the end , what happened to 2/n? Please correct me if I am wrong.
2/n Σi=1 to n -xi(yi-mxi-c) this comes from chain rule watch this part carefully again in the video, this (y-mxi-c) is the result of (yi-mxi-c)^2 and now since we want to differentiate wrt to slope m again you take the derivative now you treat the y and c as constants and what's left is -mxi so you get -xi that's what you get and you multiply with this(y-mxi-c). 2/n is a constant say for any number, n=5; 2/5 =constant you eventually equate it to zero so it vanishes away. Hope now you understand!
i have this video over and over again, it the most satisfying video i have seen, in as much as gradient decent is concer, but i have questions, 1 what happen to the 2 tha became the multiple of the function as chain rules implies, then what happen to the no in the cost function. i know its mean squared errored thing. in my small assumption either of the values cant be thrown away just like that mathematically. please help with explanation.
Hi Ranji Sir, i have a doubt. In 17:00 you have mentioned that we need to take derivative because we have two variables. And mentioned variables as x and c. But I think you were supposed to say m and c. later in 17:27 you mentioning about two parameters m and c. Please verify whether it correct or not. If i have pointed out wrong please apologize me.
Hi Ranji sir, I wanted to ask that if there are if our line is of the form M1*(feature1) + M2*(feature2).... Mn*(feature n) + c, do we have to follow same steps and calculate dJ/dm for all M1, M2...Mn?
If I understood your question correctly then: When? learning rate is made use for convergence, it should not be neither too large nor too low just optimal, so that your traning process is complete. How? You can use learning rate schedule or can use optimizers like ADAM.
The explanation of mathematical formula is absolutely fantastic. The explanation was about with single feature. But if we have multiple feature, what to be changed in the equation? Can you please let us know that. Thanks very much, and we will love to see this kind of videos shortly.
In case of multiple features or weights we have to conisder them individually by taking the partial derivtaive. This video is just a general idea of gradient descent. Hope it answers your question.
Sir my maths is quite weak i want to start my career in data science i know that i can make my math strong but how should i start to learn maths for data science
between the timestamp 21:39 to 21:45 you told that partial derivative of y(i)-mx-c with respect to c is 0 so only take minus sign which is wrong it will be -1 because of here c is not constant.
Thanks you so much but I have small clarification regarding differentiate. why we are differentiate with respect to m , c & why we should not differentiate with respect to x to find out the value of y..
When there are relatively larger coefficients in your model, taking total derivate would be a diffuícult task and also to estimate the optimal parameter. Partial derivatives reduces the workload by keeping one parameter as constant and determine the other.
J = 1/(2*m) * sum (h(x)-y)^2. being h(x) the hipotesis and y the accurate value... at 3:37 you got them mixed up right? damn man.. no wonder people get confused
For notes👉 github.com/ranjiGT/ML-latex-amendments
You just helped me understand hundreds of web pages that talked about topics with no order. Thank you
What is great about this particular video is these concepts are explained well in many places like scattered dots , you connected the dots to paint the whole picture . an example for gradient descent included - very helpful .
Thank You for your valuable feedback 😊
You are such a great teacher. Concepts are clearly explained beginning with the basics and slowly easing into the most advanced level. Thank you
You're very welcome!
You are the best. Thank you! Awesome interpretation. To anyone confused, when we should be minimizing based on slope v intercept. Here are the reasons:
• Minimization based on slope happens when the angle or steepness of the line needs adjustment to better fit the data.
• Minimization based on intercept happens when the line is generally in the right direction but needs to be shifted up or down.
In most real-world scenarios, both slope and intercept are optimized together during each iteration of gradient descent. The cost function depends on both parameters, and gradient descent adjusts both to achieve the best fit.
Hi, Your video is helpful for beginners to understand the concept. One suggestion: In the very beginning of the video when you write the equation of your predicted line remember to mark it as y(cap) = mx(i) + c. It is not y(i) which is the actual data point.
The best and clear explanation I've ever listened about Gradient Descent. Keep up the good work!🙌
Awesome, thank you!
this might be the most underrated explanation on youtube
My god you are perfect I think your work should reach more audience your best and clear than the renowned ML yputubers. Applause Ranji
its a very good description, The way you teach is humble and appreciatable.
Thank you very much!
I am a beginner and as a beginner I was struggling understand the gradient decent concept. I have seen many videos on gradient decent however all of them skipped explaining the derivative part however you explained it very well both (total and partial) with solution . Thanks!
Great to hear!
When my ML teacher teaching this , I felt I am learning some rocket science ,but you are teaching it felt very easy , thank you Sir😊
Glad to hear that
Your hard work made the concept very easy to grasp. Kudos.......
Everything is so easy on this channel, great work Man!
Glad you like them!
Man, you've won my heart, you kept it so simple, best way of explaining Gradient Descent. Can you please help me in using learning rate in the equation and number of steps used in gradient descent with an example.
I was unable to understand this topic tried many videos but this was the most useful video thankss
Truest your the best. You solve my long time machine learning challenge.
Hey thank you so much for this content since I started studying regression using your videos , I became huge fan of yours
Thank you for your comment. Glad you like it ;)
As far as I know, gradient descent doesn't talk about solving 'm' and 'c' directly by putting them into two equations like the way you did it here..Because sometime the expression that we get after differentiation becomes so complex especially with logistic regression, random forest and other complex model in deep learning that solving 'm' and 'c' directly becomes extremely tricky and time complexity becomes very high...So Gradient descent talks about solving 'm' and 'c' by some trial and test method. Starting with some dummy value of 'm' and 'c' and putting those values in the equation of differentiation and check if the value of differentiation(say D0) for those 'm' and 'c' becomes 0 or close to 0. If not, then subtract that differentiation value(D0) from the old value of 'm' and 'c' and get a new 'm' and 'c' and again check if the value of differentiation (say D1) for those new 'm' and 'c' comes close to 0. Continue like this till you find that old 'm' - value of differentiation is very very close to 0. That 'm' becomes your actual 'm' and similarly the same thing to be done for 'c'.
This video was an intuitive way for understanding gradient descent for beginners. Anyways appreciate your time to quote about your understanding of GD.
@@RanjiRaj18 Good explanation. I think you have shown derivation for Ordinary Least Squares method. As far as Machine Learning is concerned it has to be slightly adapted.
He solved m and c in this case because it wasn't a very complex example. Only one independent variable. In a multiple linear regression it would have been much more complex.
This is what I pay my internet bill for! Thanks a lot!
will the sign ( direction ) for calculating m and b at last change, from addition to subtraction if we take "y-pred - y" instead of "y - y-pred" lke you have done in cost function ? i saw at few articles where this was mentioned but it was not clear.
Sir thank you very much, this has been so helpful since my course will only get tougher from here onwards and u helped me understand the basics
Hi @ranjiraj , at 21:37 u have given a wrong explanation in the partial derivative w.r.t C...
d/dC (-C) will be -1 ,then why are you treating it as constant whereas in d/dc mx should be taken as constant.
Great explanation with an example.
This is the way to explain such concepts.
Thank you for the comment. Happy Learning!
perfect video for core concept understanding , amazing.. I love the explanation.. thankyou so much
Thanks for the clear explanation sir
simple and useful lecture.. thanks
You are welcome
Very good explanation of gradient descent
Very good explanation. It would've been good if you could've explained the usage of learning rate usage to find a minimum point.
Absolutely the best explanation
If I got it your last example is analytical solution, but it couldn't been done everytime, then we use iterative solution with alpha learning rate?
That is a explanation. I have one question, where would the learning rate be actually used in computation. Like in your numerical example. We found the outputs and calculated corresponding m and c. How does the learning rate is catered. Secondly, when we multiply learning rate with derivative, what does it gives us ?
can you please tell me why the curve is much sharp when you draw the graph with respect to j and c ? please tell
Really amazing, thank you so much sir, keep rocking
Always welcome
sir u explained so well
Hi one question here:
First derivative: xsquare =2x
Second derivative = 2 ( replaced in same location)
Third derivative = 0
While applying same in mean square error formula
First derivative= I understood square to 2/n(•••)
Second derivative: with respective to slope
It should be
2/n I=1 to n (yi -xi -c) here I applied second derivative replacing.
Since -mxi converts to -xi.
But In ur explanation instead of replacing, you brought second derivative to starting as below:
2/n I= 1 to n -xi(yi-mxi-c)
In the same way for intercept.
One more
At the end , what happened to 2/n?
Please correct me if I am wrong.
2/n Σi=1 to n -xi(yi-mxi-c) this comes from chain rule watch this part carefully again in the video, this (y-mxi-c) is the result of (yi-mxi-c)^2 and now since we want to differentiate wrt to slope m again you take the derivative now you treat the y and c as constants and what's left is -mxi so you get -xi that's what you get and you multiply with this(y-mxi-c). 2/n is a constant say for any number, n=5; 2/5 =constant you eventually equate it to zero so it vanishes away. Hope now you understand!
i have this video over and over again, it the most satisfying video i have seen, in as much as gradient decent is concer, but i have questions, 1 what happen to the 2 tha became the multiple of the function as chain rules implies, then what happen to the no in the cost function. i know its mean squared errored thing. in my small assumption either of the values cant be thrown away just like that mathematically. please help with explanation.
Hi Ranji Sir, i have a doubt. In 17:00 you have mentioned that we need to take derivative because we have two variables. And mentioned variables as x and c. But I think you were supposed to say m and c. later in 17:27 you mentioning about two parameters m and c. Please verify whether it correct or not. If i have pointed out wrong please apologize me.
Yes you are correct we have to take derivative wrt m and c
Sir linear regression is not used for classification as you said in satrting of video while explaining
Great explanation, thank you.
Thank you so much, I got a clear picture of the topic now.
You are welcome!
killer explanation !! amazingly amazing !! thank you bro
Well explained bro ❤ just bring another video for statistics and linear algebra 🎉
explanation is simply awesome.....
really nice approch to teach
thank you sirji
Thankyou sir ,
Make more videos on machine learning concepts .
you earned a subscriber my man
very nicely explained Bro
Glad you liked it
Great video man. Loved it.
Thank you! Wish me luck on exam about it!
Good Luck 👍
Hi Ranji sir, I wanted to ask that if there are if our line is of the form M1*(feature1) + M2*(feature2).... Mn*(feature n) + c, do we have to follow same steps and calculate dJ/dm for all M1, M2...Mn?
Yes, one by one
Thank you so much..quick question, when/how do we use the learning rate in this regard?
If I understood your question correctly then:
When? learning rate is made use for convergence, it should not be neither too large nor too low just optimal, so that your traning process is complete.
How? You can use learning rate schedule or can use optimizers like ADAM.
@@RanjiRaj18 Yes, Alright thank you so much. Your vid was really helpful.
Thanks so much for the video. It helped me a lot.
At 20.37, you discarded n/2. I would like to know why?
if 2x = 0 , then it simplifies to x = 0/2 . i.e. x = 0
Thanks for this tutorial sir..made it very easy and simple.
Thanks a lot sir! It was really helpful. Excellent explaination.
What should i do if i am to apply learning rate of some value?
great explainnation
The explanation of mathematical formula is absolutely fantastic. The explanation was about with single feature. But if we have multiple feature, what to be changed in the equation? Can you please let us know that. Thanks very much, and we will love to see this kind of videos shortly.
In case of multiple features or weights we have to conisder them individually by taking the partial derivtaive. This video is just a general idea of gradient descent. Hope it answers your question.
hi, sir do you have the python code using tensor flow or do you have any recordings of ML using TF
Sir my maths is quite weak i want to start my career in data science i know that i can make my math strong but how should i start to learn maths for data science
Hello Maria, you can refer to websites like www.mathsisfun.com/ to learn the basics. Hope it helps!
@@RanjiRaj18 thanks sir
@@RanjiRaj18 Sir its very low level mathematics I am in sybsc it
Thanks ...very well explained
Lovely explanation
between the timestamp 21:39 to 21:45 you told that partial derivative of y(i)-mx-c with respect to c is 0 so only take minus sign which is wrong it will be -1 because of here c is not constant.
Thanks bro.
Well explained
can you please explain me why we are squaring the step at 4.36 . everything is clear to me except this one squaring step I cant able to understand..
your videos are really nice, good content and presentation...keep it up sir.
one of the best
Nicely explained.
Best explanation sir
Thanks you so much but I have small clarification regarding differentiate. why we are differentiate with respect to m , c & why we should not differentiate with respect to x to find out the value of y..
Because m, c are the weights that we want to determine which will give the best equation for curve fitting.
@@RanjiRaj18 thanks you so much for spending time to respond to my comment..
why do we set the derivative equal to 0? i mean the gradient at the minima might not be equal to zero for all curves
What type is it batch gradient?
excellent video
can this method work for an equation with multiple slopes?
very nice explanation
great explaination
Thank you so much sir for such a perfect explanation....🙏🙏👏👏👏
what about when someone is working with the learning rate?
thank you so much sir
dear sir,
I still cannot connected what is the purpose we do m = m - lambda * dJ/dm and c = c - lambda * dJ/dC
Yess .. exactly....plz explain the proof of these 2 equations
Thankyou sir 😊
So gradient descent is our cost function to calculate the error
Amazing lecture. x^n, will not have a 3rd order derivate to be 0, it will be n+1 order derivate.
How did the 2/n gone from equation when dJ/dm and dj/dc was assigbed to 0
2/n is a constant say you take n =5 so it becomes 2/5 so it's derivative is zero
@@RanjiRaj18 Thank you Ranji
Why do we need partial derivative when we have the total derivative?
When there are relatively larger coefficients in your model, taking total derivate would be a diffuícult task and also to estimate the optimal parameter. Partial derivatives reduces the workload by keeping
one parameter as constant and determine the other.
is it a sweet, in the middle of a hyperplane?
keep up the good work !
bro please ans this question why we are taking summation of c in one equation and not in other ---> one is 5C why
Plase ans
Thanks Buddy :)
Linear regression for classfication at 0:20???
Sir can you recommend a book for machine learning with mathematical background please
You can refer the book by `Tom Mitchell`
@@RanjiRaj18 sir please what is book name and if you share pdf link it will be better
@@shafiqahmad9057 You can check on google it is open source
@@RanjiRaj18 thank you for very fast respomse
Which level of maths is required 11th and 12th or degree level mathematics ?
Personally both Derivatives, differential equations, Matrices and vector concepts
@@RanjiRaj18 thanks
thank you!!!
It was very helpful, in writing a thesis. Could you also indicate some bibliography for citations
Excellent...
sir explane krne ke baad thoda side ho jaya kijiye screenshot lena rehta hai
J = 1/(2*m) * sum (h(x)-y)^2. being h(x) the hipotesis and y the accurate value... at 3:37 you got them mixed up right? damn man.. no wonder people get confused
why at the end multiply the 2 nd equation with 5
To make equation balance on both sides for cancellation. Those are basic algebraic rules.
Thanks Bro :)
🙂👍 nice like that