Hello, thank you very much for your content, it is very entertaining and educational. I have a question, why do you say that the solution to Lasso is in the corners? I have read in the literature that it is a common case, but not forced in any way. Can you give more details on this?
Why do the edges in L1, and the cusp in elastic net have a higher probabilty of hitting the level curve? I dont get it. Can someone give me a reference. Imaginging a loss landscape as a function of all the parameters, so what if a level set hits the edge? what does that represent?
I probably under-explained it in the video but this image is nice to get a sense of it: miro.medium.com/v2/resize:fit:761/1*nrWncnoJ4V_BkzEf1pd4MA.png Basically the "sharp corners" of the diamond in L1 (and to a lesser extent in elastic-net) will tend to first hit the level curves of the objective function (which are actually ellipses in 2d)
Hello, thank you very much for your content, it is very entertaining and educational. I have a question, why do you say that the solution to Lasso is in the corners? I have read in the literature that it is a common case, but not forced in any way. Can you give more details on this?
Why do the edges in L1, and the cusp in elastic net have a higher probabilty of hitting the level curve? I dont get it. Can someone give me a reference. Imaginging a loss landscape as a function of all the parameters, so what if a level set hits the edge? what does that represent?
I probably under-explained it in the video but this image is nice to get a sense of it:
miro.medium.com/v2/resize:fit:761/1*nrWncnoJ4V_BkzEf1pd4MA.png
Basically the "sharp corners" of the diamond in L1 (and to a lesser extent in elastic-net) will tend to first hit the level curves of the objective function (which are actually ellipses in 2d)