AE501 Took me some time to figure out how to create a function in MATLAB utilizing this armijo rule, but it was very satisfying once it worked. I never had much experience programming and really appreciated the basic breakdown of how to write the code.
AE 501: Jesse Perez - Good explanation of the Armijo Rule which appears to be similar to line minimization but more efficient on the points being used along the linear direction.
In case it is helpful, all my optimization videos in a single playlist are located at ua-cam.com/play/PLxdnSsBqCrrHo2EYb_sMctU959D-iPybT.html. You can support this channel via Patreon at www.patreon.com/christopherwlum. Please let me know what you think in the comments. Thanks for watching!
AE501: This method really makes leaps in efficiency and requires some tuning. It looks useful as a tool to explore model behavior as well as solution space for those unfamiliar cost functions. It probably would help to have a deeper understanding of the specific forms of cost functions and how they behave physically...a little like studying PDE's...Gotta make sure I hop in the right hole
Hi Max, Thanks for the kind words, I'm glad you enjoyed the video. If the find the these videos to be helpful, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum. Given your interest in this topic, I'd love to have you a as a Patron as I'm able to talk/interact personally with all Patrons. Thanks for watching! -Chris
Hi Sandeep, Thanks for the kind words, I'm glad you enjoyed the video. If you find these videos helpful, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum or via the 'Thanks' button underneath the video. Given your interest in this topic, I'd love to have you a as a Patron as I'm able to talk/interact personally with all Patrons. I can also answer any questions and provide code/downloads on Patreon. Thanks for watching! -Chris
Hi Justin, Thanks for the kind words, I'm glad you enjoyed the video. If you find these videos helpful, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum or via the 'Thanks' button underneath the video. Given your interest in this topic, I'd love to have you a as a Patron as I'm able to talk/interact personally with all Patrons. I can also answer any questions, provide code, notes, downloads, etc. on Patreon. I do indeed still teach a few classes in the AA dept at UW. Thanks for watching! -Chris
Ae 501: I'm having trouble interpreting how the sigma value affects the convergence of the armijo rule. It seems like it either converges or doesnt, but speed doesnt change.
Hi, is it there any lecture that you'd recommend to study this implementation but in the case where it is too costly to evaluate the function to check the condition. I was wonder if it could be used the Taylor's series to have an approximation of this Thanks so much for all the work done to teach these lessons, I've learned so much from them! I really appreciate it :)
Hi Giselt, Thanks for reaching out, I'm glad you enjoyed the video. Unfortunately I'm unable to respond to questions on UA-cam due to the sheer volume of inquiries that I receive. That being said, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum or via the 'Thanks' button underneath the video as I'll be able to answer questions there. Given your interest in the topic, I'd love to have you as a Patron as I'm able to talk/interact personally with Patrons. Thanks for watching! -Chris
thank you very much for your video Mr. Lum, , I learned a lot :) I would like to know where I can find your code, I have a doubt on how to get your graphs, as I find it difficult to replicate them. kind regards, Alan.
Hi Alan, Thanks for the kind words, I'm glad you enjoyed the video. If you find these videos helpful, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum or via the 'Thanks' button underneath the video. Given your interest in this topic, I'd love to have you a as a Patron as I'm able to talk/interact personally with all Patrons. I can also answer any questions you might have about any of the videos on Patreon. Patrons also have access to code and other downloads. Thanks for watching! -Chris
Hi, Thanks for the kind words, I'm glad you enjoyed the video. If you find these videos helpful, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum or via the 'Thanks' button underneath the video. Given your interest in this topic, I'd love to have you a as a Patron as I'm able to talk/interact personally with all Patrons. Thanks for watching! -Chris
Hi, Thanks for reaching out, I'm glad you enjoyed the video. Unfortunately I'm unable to respond to questions on UA-cam due to the sheer volume of inquiries that I receive. That being said, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum as I'll be able to answer questions there. Given your interest in the topic, I'd love to have you as a Patron as I'm able to talk/interact personally with Patrons. Thanks for watching! -Chris
I will try this. I have tried many different versions of GD. None of them are very good when the "terrain" gets rough or the path to them minimum is very crooked. I didn't like the first part of the video because there was too much time wasted writing on a white board. I also think it would be faster to simply make a small step and keep stepping in the same direction as long as the MSE keep dropping. I have some test data that is very difficult for GD to minimize. There are 5 parameters, and the "terrain" is more like the Colorado river cutting and winding its way through the Grand Canyon. The path is very narrow and always changing. I don't have high hope for the this Armijo algorithm. I have found the brute force line search along with the coordinate search work best. All methods work well if the "terrain" is like a bowl. Nelder-Mead would be MUCH faster than the GD with Armijo. Levenberg-Marquardt is faster yet even with the Grand Canyon like terrain of my test data.
![[CFD] Conjugate Gradient for CFD (Part 1): Background and Steepest Descent](/img/n.gif)
AE501: The introduction for the Armijo Rule really helped me understand the rest of the video. Thank you for the great content!
AE501: This is my first exposure to Armijo Rule. Very interesting application and theory. Thank you for your in-depth explanation.
AE501
Took me some time to figure out how to create a function in MATLAB utilizing this armijo rule, but it was very satisfying once it worked. I never had much experience programming and really appreciated the basic breakdown of how to write the code.
AE 501: Jesse Perez - Good explanation of the Armijo Rule which appears to be similar to line minimization but more efficient on the points being used along the linear direction.
AE 501 This is something that I had not been exposed too yet. Thank you for your explanation
AE501: The introduction for the Armijo Rule really helped
AE501 The Armijo lecture was pretty confusing at first but this video definitely helped a lot. Thank you professor.
In case it is helpful, all my optimization videos in a single playlist are located at ua-cam.com/play/PLxdnSsBqCrrHo2EYb_sMctU959D-iPybT.html. You can support this channel via Patreon at www.patreon.com/christopherwlum. Please let me know what you think in the comments. Thanks for watching!
AE501: This method really makes leaps in efficiency and requires some tuning. It looks useful as a tool to explore model behavior as well as solution space for those unfamiliar cost functions. It probably would help to have a deeper understanding of the specific forms of cost functions and how they behave physically...a little like studying PDE's...Gotta make sure I hop in the right hole
AE501: The Armijo rule seems pretty powerful. Would be interested in seeing its implementation when a non-analytical solution exists
Hello sir
I like your way of teaching
Love from India 😘😘😍♥️♥️♥️
Hi Max,
Thanks for the kind words, I'm glad you enjoyed the video. If the find the these videos to be helpful, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum. Given your interest in this topic, I'd love to have you a as a Patron as I'm able to talk/interact personally with all Patrons. Thanks for watching!
-Chris
thank you for this awesome lecture.
Hi Sandeep,
Thanks for the kind words, I'm glad you enjoyed the video. If you find these videos helpful, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum or via the 'Thanks' button underneath the video. Given your interest in this topic, I'd love to have you a as a Patron as I'm able to talk/interact personally with all Patrons. I can also answer any questions and provide code/downloads on Patreon. Thanks for watching!
-Chris
AE501: Isn't gradient dotted with the direction just gradient squared (assuming we are using gradient descent)?
I am getting my math degree at UW right now, do you still teach there? I love your lecturing style!
Hi Justin,
Thanks for the kind words, I'm glad you enjoyed the video. If you find these videos helpful, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum or via the 'Thanks' button underneath the video. Given your interest in this topic, I'd love to have you a as a Patron as I'm able to talk/interact personally with all Patrons. I can also answer any questions, provide code, notes, downloads, etc. on Patreon. I do indeed still teach a few classes in the AA dept at UW. Thanks for watching!
-Chris
Ae 501: I'm having trouble interpreting how the sigma value affects the convergence of the armijo rule. It seems like it either converges or doesnt, but speed doesnt change.
Hi, is it there any lecture that you'd recommend to study this implementation but in the case where it is too costly to evaluate the function to check the condition. I was wonder if it could be used the Taylor's series to have an approximation of this
Thanks so much for all the work done to teach these lessons, I've learned so much from them! I really appreciate it :)
Hi Giselt,
Thanks for reaching out, I'm glad you enjoyed the video. Unfortunately I'm unable to respond to questions on UA-cam due to the sheer volume of inquiries that I receive. That being said, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum or via the 'Thanks' button underneath the video as I'll be able to answer questions there. Given your interest in the topic, I'd love to have you as a Patron as I'm able to talk/interact personally with Patrons. Thanks for watching!
-Chris
thank you very much for your video Mr. Lum, , I learned a lot :)
I would like to know where I can find your code, I have a doubt on how to get your graphs, as I find it difficult to replicate them.
kind regards, Alan.
Hi Alan,
Thanks for the kind words, I'm glad you enjoyed the video. If you find these videos helpful, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum or via the 'Thanks' button underneath the video. Given your interest in this topic, I'd love to have you a as a Patron as I'm able to talk/interact personally with all Patrons. I can also answer any questions you might have about any of the videos on Patreon. Patrons also have access to code and other downloads. Thanks for watching!
-Chris
Great lecture. Does dk, the direction, change with xk or does it remain constant?
Good question, you need to compute a new dk at each xk. This is because the function can be completely different at a different location xk.
thanks
Hi,
Thanks for the kind words, I'm glad you enjoyed the video. If you find these videos helpful, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum or via the 'Thanks' button underneath the video. Given your interest in this topic, I'd love to have you a as a Patron as I'm able to talk/interact personally with all Patrons. Thanks for watching!
-Chris
Interesting lecture! Can I use the same way to deal with maximum?
Hi,
Thanks for reaching out, I'm glad you enjoyed the video. Unfortunately I'm unable to respond to questions on UA-cam due to the sheer volume of inquiries that I receive. That being said, I hope you'll consider supporting the channel via Patreon at www.patreon.com/christopherwlum as I'll be able to answer questions there. Given your interest in the topic, I'd love to have you as a Patron as I'm able to talk/interact personally with Patrons. Thanks for watching!
-Chris
Yes maximum max(f)=-min(-f)
I will try this. I have tried many different versions of GD. None of them are very good when the "terrain" gets rough or the path to them minimum is very crooked. I didn't like the first part of the video because there was too much time wasted writing on a white board. I also think it would be faster to simply make a small step and keep stepping in the same direction as long as the MSE keep dropping.
I have some test data that is very difficult for GD to minimize. There are 5 parameters, and the "terrain" is more like the Colorado river cutting and winding its way through the Grand Canyon. The path is very narrow and always changing. I don't have high hope for the this Armijo algorithm. I have found the brute force line search along with the coordinate search work best. All methods work well if the "terrain" is like a bowl. Nelder-Mead would be MUCH faster than the GD with Armijo. Levenberg-Marquardt is faster yet even with the Grand Canyon like terrain of my test data.