When Should You Use L1/L2 Regularization

Overfitting is one of the main problems we face when building neural networks. Before jumping into trying out fixes for over or underfitting, it is important to understand what it means, why it happens and what problems it causes for our neural networks. In this video, we will look into L1 and L2 regularization. We will learn how these regularization techniques work and when to use them.
Previous lesson: • Why Regularization Low...
Next lesson: • What is Dropout Regula...
📙 Here is a lesson notes booklet that summarizes everything you learn in this course in diagrams and visualizations. You can get it here 👉 misraturp.gumroad.com/l/fdl
📕 NNs hyperparameters cheat sheet: www.soyouwanttobeadatascienti...
👩‍💻 You can get access to all the code I develop in this course here: github.com/misraturp/Deep-lea...
❓To get the most out of the course, don't forget to answer the end of module questions:
fishy-dessert-4fc.notion.site...
👉 You can find the answers here:
fishy-dessert-4fc.notion.site...
RESOURCES:
🏃‍♀️ Data Science Kick-starter mini-course: www.misraturp.com/courses/dat...
🐼 Pandas cheat sheet: misraturp.gumroad.com/l/pandascs
📥 Streamlit template (updated in 2023, now for $5): misraturp.gumroad.com/l/stemp
📝 NNs hyperparameters cheat sheet: www.misraturp.com/nn-hyperpar...
📙 Fundamentals of Deep Learning in 25 pages: misraturp.gumroad.com/l/fdl
COURSES:
👩‍💻 Hands-on Data Science: Complete your first portfolio project: www.misraturp.com/hods
🌎 Website - misraturp.com/
🐥 Twitter - / misraturp

[@hanipmanju]

Wow thank you for this video!!! this 8min video was better than my instructor's 8hour class on the same topic.

[@ranjanpal7217]

Amazing explanation

[@NisseOhlsen]

The goal of regularization is to spread out the transfer from one layer to the next to as many connections as possible, thereby forcing the network to consider many aspects of the connection between the input and output. This is done by penalizing 'tunnelling' through few connections. And that is exactly that penalizing large weights does.

[@nasioulicious]

Thank you Misra.Great content!

[@misraturp]

You're very welcome!

[@oo_wais]

the model that lets you use both L1 and L2 regularization techniques is called Elastic Net. it has an extra parameter which takes values in range of 0 and 1. I just read about it yesterday in a book. Anyway, thanks for this great series, i am a complete beginner to NNs, and this series is helping me a lot in understanding the big picture and all the basic concepts and procedures of NNs.

[@bay-bicerdover]

not in range of 0 and 1 but 0 and pos. inf.

[@wut4100]

L1 was solid, I wish L2 was explained as well as L1.

[@grekiki]

Why can't I use alpha>1? Also doesn't this fail for networks with batchnorm for example?

[@kylmaz5782]

Mısra Hanım merhabalar. Örneklem sayısı az olan bir veri seti ile Ridge regresyon yöntemini kullanarak bir model oluşturmak istiyorum. Ancak modeli oluştururken çözümü el ile yapacağım. Bu konuda yardımcı olabilir misiniz?

[@andresg3110]

Ok I subscribed! Like I'm a simple NN I see talent I converge to my optimum solution

[@bay-bicerdover]

5:40'da parametrenin adini "alpha" diye belirtmissiniz, λ degil mi dogrusu?