This was exactly what I needed, thank you! after learning Newton's method for finding the x-intercept, I was confused at first on how it was being used for minimization problems
Thanks for posting these videos. They are quite helpful. So, to ensure that we minimize and not maximize, is it sufficient to ensure that the newton step has the same sign (goes towards the same direction) as the gradient? Is it ok to just change the sign of the step if that's not the case? (my experiments seem to indicate its not, but what should be done then?)
So here is the thing ... My function y = (guess^2 - x ) Now i want to minimize y by approximating guess so i use first order with guess = guess - (guess^2 - x)/2guess .... which is xt+1 = xt - f(x)/f'(x) but if i do on more derivative then xt+1 = xt - f ' (x)/f " (x) which is guess = guess - (2guess)/2 = guess - guess = 0 What to do the function is finding square root by newtons method
Hii @Alex N ,as per my knowledge this methods are used for Machine learning, where gradient descent is a classical algorithm to find minimum of a function( not always zero), If you know basics about ML then you will be familiar with loss function , so we have to minimize that function, for that we need its derivative to be zero, for finding that we use gradient as direction where the change in function is maximum.Now we have the direction but the we dont have the magnitude , for that we use a learning rate as a constant which is what 1st order does.In 2nd order we would use the magnitude which gives us the magnitude for which the point where derivative of function is 0 can be reached in less iterations.Thus 3rd order will ultimately result in finding the minimum of dervative of the loss function , but we need to find minimum of the loss function so ,it will be useless. Hope this was helpul
Video makes sense up until the point where the "pictorial" representation of the 2nd order method comes in. That to me makes absolutely no sense whatsoever, the "pictorial" should not be the function itself but you rather the 1st derivative of the function and you apply Newton's method to that.
I think that the visualization makes sense if we think about approximating the function f(x) by its second order Taylor expansion around x_t. Taking the derivative of the second order Taylor expansion and setting it equal to zero leads us to the formula of the Newton's method for optimization. This operation is the same as minimizing the second order approximation of the function at x_t as depicted in the video.
The way you explain this is so helpful - love the comparison to the linear approximation. Thank you!
Wonderful video for clearing optimization of newtons method for finding minima of function in machine learning
man, perfect explanation. clear and intuitive!
Illuminating! Thank you
This was exactly what I needed, thank you!
after learning Newton's method for finding the x-intercept, I was confused at first on how it was being used for minimization problems
It was one of the best explanations, so informative and helpful. Thank you!
Your videos are awesome!
Sir your way of explaining is really good.
This was actually quite helpful :)
Thank you for the video!
really appreciate your work :)
Thanks so much for posting!!
Thanks for posting these videos. They are quite helpful. So, to ensure that we minimize and not maximize, is it sufficient to ensure that the newton step has the same sign (goes towards the same direction) as the gradient? Is it ok to just change the sign of the step if that's not the case? (my experiments seem to indicate its not, but what should be done then?)
Great video. Thank you
Can you please make a video on levenberg method. Since there is no lecture available on this topic
thanks , very informative
Amazing! Thanks
I am a PhD student and I will be using optimization methods in my research.
this is so good man
monk
cool! ;D
damn good!
THANK YOU
very good. thank you
ua-cam.com/video/kxftUHk7NDk/v-deo.html
So here is the thing ... My function y = (guess^2 - x )
Now i want to minimize y by approximating guess
so i use first order with guess = guess - (guess^2 - x)/2guess ....
which is xt+1 = xt - f(x)/f'(x)
but if i do on more derivative
then xt+1 = xt - f ' (x)/f " (x) which is guess = guess - (2guess)/2 = guess - guess = 0
What to do
the function is finding square root by newtons method
Thank you u god among men
ua-cam.com/video/kxftUHk7NDk/v-deo.html
Soooo, if 2nd order is faster than 1st order, why not try 3rd order too?
Hii @Alex N ,as per my knowledge this methods are used for Machine learning, where gradient descent is a classical algorithm to find minimum of a function( not always zero), If you know basics about ML then you will be familiar with loss function , so we have to minimize that function, for that we need its derivative to be zero, for finding that we use gradient as direction where the change in function is maximum.Now we have the direction but the we dont have the magnitude , for that we use a learning rate as a constant which is what 1st order does.In 2nd order we would use the magnitude which gives us the magnitude for which the point where derivative of function is 0 can be reached in less iterations.Thus 3rd order will ultimately result in finding the minimum of dervative of the loss function , but we need to find minimum of the loss function so ,it will be useless. Hope this was helpul
Video makes sense up until the point where the "pictorial" representation of the 2nd order method comes in. That to me makes absolutely no sense whatsoever, the "pictorial" should not be the function itself but you rather the 1st derivative of the function and you apply Newton's method to that.
I think that the visualization makes sense if we think about approximating the function f(x) by its second order Taylor expansion around x_t. Taking the derivative of the second order Taylor expansion and setting it equal to zero leads us to the formula of the Newton's method for optimization. This operation is the same as minimizing the second order approximation of the function at x_t as depicted in the video.
#IntuitiveAlgorithm finding where zero of a function
Shreyas Rane Now I see where you study from.
Busted
comparing to Andrew Ng's explanation, this one is hard to understand
thanks for the video. could you please check your inbox, I have some further questions, thanks!!!
chonssdw seso