Loss or Cost Function | Deep Learning Tutorial 11 (Tensorflow Tutorial, Keras & Python)
Вставка
- Опубліковано 13 сер 2020
- Loss or a cost function is an important concept we need to understand if you want to grasp how a neural network trains itself. We will go over various loss functions in this video such as mean absolute error (a.k.a MAE), mean squared error (a.k.a MSE), log loss or binary cross entropy. After going through theory we will implement these loss functions in python. It is important to go through this implementation as it might be useful during your interviews (if you are targeting a role of a data scientist or a machine learning engineer)
Code: github.com/codebasics/deep-le...
Exercise: Go at the end of the above notebook to see the exercise
Do you want to learn technology from me? Check codebasics.io/ for my affordable video courses.
🔖 Hashtags 🔖
#lossfunction #costfunction #costfunctionneuralnetwork #lossfunctionneuralnetwork #costfunctiondeeplearning #lossfunctiondeeplearning
Why not MSE for logistic regression:
towardsdatascience.com/why-no...
Next video: • Gradient Descent For N...
Previous video: • Matrix Basics | Deep L...
Deep learning playlist: • Deep Learning With Ten...
Machine learning playlist : ua-cam.com/users/playlist?list...
Prerequisites for this series:
1: Python tutorials (first 16 videos): ua-cam.com/users/playlist?list...
2: Pandas tutorials(first 8 videos): • Pandas Tutorial (Data ...
3: Machine learning playlist (first 16 videos): ua-cam.com/users/playlist?list...
Website: codebasics.io/
Facebook: / codebasicshub
Twitter: / codebasicshub
Do you want to learn technology from me? Check codebasics.io/ for my affordable video courses.
Loving this series. You're so good at looking at everything from the learners perspective and clarifying everything! Much better than all the other tutorials I've tried
Beautiful explaination ❤️
That log loss vs MSE article was a BOOM!
Man, I love this series.
Glad you enjoyed it
Feedback: You can create chapter of the tutorials, will be helpful to navigate
This video cannot have chapters since he's teaching the concept as a whole and not discrete things in the video that could be bookmarked like in other videos. 🙄
When changing y_predicted to avoid errors with log(0) it is possible to do [ abs( y - epsilon ) for y in y_predicted ], which will work for both y=1 and y=0 cases
if earlier, some y was epsilon(which is fine), then now it will become 0 from your function(which is a problem).
it's very easy exercise i just replace the absolute with square and it done...thanks for this playlist
Thank you so much for this course, sir
Great man does a great job. Thank you so much.
@Devin Preston ???
These series are the best
very benificial videos, can u upload videos of some online or offline job oppurtunities of machine, deep learning
Thanks for the presentation.
Small question: whereas y-hat is actual predicted result (fractional values) y seems to be from Step Function (ie, 0 or 1, not fractional value). Is this correct ?
Sir in binary cross entropy the loss function should be c=-- 1/n sum (y*log(p) + (1-y)log(1-p))
Where p is the probability of the sample for the positive class.
By the way nice intuitive lectures ,I love you way of teaching 🙏
yes it comes from bernoullie's theorem P(x)=p^x*(i-p)^n-x; after that if we take log on both side.
Amazing tutorial :)
Thank you so for the detailed explanation. Could you please explain about SGD, ADAM optimizations.
I have a video coming up on SGD, batch and mini batch. It will be live today.i will add video on Adam too in future
@@codebasics Thank you so Much 🙇
Simply great💌
The tutorials are very helpful. Can we get ppt slides of the tutorials plz?
You are excellent boss. Allah bless you!
Am just stuck with the series.. It's super interesting
Great.Thank you very much
thanks for the video
Thank you for this very clear lesson. Is it possible that the "y_predicted" and "y_true" are confused?
I would expect y_true to be in {0,1}, and y_predicted represent a probability in [0,1]
Do you have tutorial like this but for NCE Loss function? Thank you
You are rock🙏
Nice Explanation. Do we need to use Binary crossentropy in case of Binary classification only or can be used in Multi class classification?
Also kindly explain about the sparse categorical entropy.
In multi class classification we need to use categorical cross entropy or sparse categorical entropy. Yes I will cover difference among them in future videos
@@codebasics Hi Sir, This is a nice explanation about MSE and MAE. Have you made video on Categorical cross entropy? Looking for this video. Thanks.
Dhaval ji it would be better if you also explain if one get 'nan' error due to log(1-y_pred) part in log loss equation when (1-y_pred) value becomes 'negative' which is invalid value encountered in log. And How to resolve this issue?
Pls add more videos of this Playlist sir
U should not hesitate to mention Andrew NG 's name. We all learn from somebody. Andrew NG has his own expertise and u have ur own.
It is always perfectly okay to share from sources we learn.
Of course. I think pretty much all the people who know ML have learnt from Andrew NG at some point. I have a great admiration for him 👍🏼
hi , i have a problem understanding the example in the sklearn documentation
from sklearn.metrics import log_loss
>>> log_loss(["spam", "ham", "ham", "spam"],
... [[.1, .9], [.9, .1], [.8, .2], [.35, .65]])
0.21616...
can you please explain this and extend the log_loss idea to multiclass classification. Where they take np.argmax of true_values and subtract the predicted.
can you please make a video and expain this concept. Thanks a lot for making this great video
Thank you so much Sir.
Most welcome
Have a question, in 7:50 you mentioned that MAE is the cost function, but why in 1:05, you said it is loss function? Besides, in TF's parameter, should not the parameter "loss" should be "cost"? Because it is cost function not loss function?
thank you
Awesome content thanks a lot
Glad you enjoyed it.
Thank you for the video
👍😊
Very nice video tutorial.. keep on doing this!!
if predicted value is always 0 and 1 then following is easy right?
y_predicted_new = [ abs(i-1e-15) for i in y_predicted ]
very nicely explained
👍😊
hi sir,when we are doing log loss function in that we are taking -np.mean(log function) but we didn't take n value hear -1/n ? at 22:37
for real i really performed all those code without using your notebook sir because of the virus warning.😅😅
Any tutorials on multiclass-multioutput regression or classification
you mean classification?
Sir please make a course on deep learning and AI🙏🙏.please reply
RuntimeWarning: divide by zero encountered in log
min(i, 1 - epsilon) puts all values below even if it is above 1
you are such a great man 👨
why don't you change to value of y_true with epsilon when doing log loss function ?
look at equation 16:42 there's no log for y_true. so no need to change it with epsilon
Sir, in Binary cross entropy ... won't the true value (y) create a problem if it is 1 ?.... because it makes the equation 0 as 1-1 = 0
Yeah, I think y_true will make problem because if it's zero all the left portion will be ignored as it's doesn't exist and if it's 1 all the right portion will be ignored . Will that cause a problem I don't know but I think we lost a valuable information.
😅😄you are cool, thank you
Naively explained
If log loss outputs the range from 0 to 1, why is your model giving values outside this range?
We need to upgrade to read the full article that you are referring to in the video...
at 17:32 the formula needs a parentheses to begin before yi and end at the end of expression.
Here is my intuition for the scary looking Log loss formula
Case 1: We predict perfectly
As per formula if y is actually 0 and you predict 0 then the loss is 0. if y is actually 1 and you predict 1 then loss is 0
Case 2: We predict with some errors
If y was actually 0 then the first term is 0.
The Second term is putting a very high penalty (exponentially higher) if you predict value closer to 1.
The penalty would be infinite if your algorithm predicted a value of 1 when the actual was value was 0
Vice versa also holds
Guessing, is chance so on bases on chances the error should have been calculated
Hello sir.....I'm a beginner machine learning learner...I started learning it from your channel...but one of my friend suggested me to do the course of Machine Learning by andrew ng from coursera...but in that course..after linear and logistic regression, he has started directly teaching neural networks while in your playlist first all ml concepts are suggested as an prequisite. So what should I do...hope you clear my doubts asap....thnx
Yes please follow my entire machine learning playlist. You need to have good understanding on basic machine learning models, train test split, k fold, GridsearchCV etc before you start deep learning. And I have covered all of that in my ML playlist
Also he teaches in very deep...like mathematics and all as compared to you...so are your videos sufficient to do various projects?
@@sarveshamrute2959 without ML u can't move smoothly in DL. If you dont have enough knowledge in ML algorithms you probably stuck everywhere in DL. Codebasics gives you an amazing grip in ML with awesome explanation as this video so you can implement any project smoothly. go to ml playlist now. i'm saying this with experience. and also he has discord server u can join there and get instant help from experts in ml and dl whenever u stuck. good luck
When i use tf.keras.metrics.binary_crossentropy, the result is different. please comment.
Where can I get this PPT for my revision
why both np.sum(np.abs(y_predicted-y_true)) and np.mean(np.sum(np.abs(y_predicted-y_true))) have same result when i calculate on my own
I have accidentally opened the solution file ... Now my computer is feeling fever and headache 😂 I think you were right... Now I need to get a vaccination for my computer, kindly make a tutorial for that too.... 😂 😂 😂
Pfizer is still inventing the vaccination for the computer :)
@codebasics but I want you to make a vaccine for my computer.... 😂😂 because you send this coronavirus in my computer or send me NVIDIA GeForce RTX 3070 ( Vaccine for my pc ) 😂😂 😂😂😂😂😂😂
sir what is the difference between (math.e)**(-15) and 1e-15
By math.e you call the Euler's number from math module in python so your example is (2.718281)**-15 whereas by 1e-15 you mean the scientific notation so it's 1 * 10**(-15).
@@mikeminotakis9092 thank you anna
Reflecting back, seems like many people clicked on the solution link without trying to solve for the answer themselves.
6:29 Great content. sir, how can be like that y=0?
it was the actual value, have_insurance=y=0
Nobel prize
i am implimenting mini batch graddesc but y_predicted is very far from y_true..no matter how many epochs i try.....
sir please register a company and give us internship in a project. ready to pay you anything!!It will be an honour and privilege to work with you.
poor corona virus :(
MSE is 0.366
Your Virus came to my computer
Lmfao coronavirus solution! Just a quick question -> On the mean square error, what does n represent? is it the number of output nodes or the number of training samples in the dataset?
The second one. If your dataset has 30 records, so it will be 1/30*Loss
point nineninenineninenineninenine
dcmm
Sir, can we have more clarification on the terminiologies we use?initially you said individual error is called loss and cumulative error is called cost function. which means MSE and MEA should be called as cost function and not loss but later in the model you put loss = 'mean-squared-error'. Shouldn't it be costfunction = 'mean-squared-error'
Obviously 'loss' must be a keyword used by tensorflow, so should we assume that every thing(absolute error, MAE, MSE, log loss) can be called as 'loss' or 'cost' or 'loss function' or 'cost function' interchangeably?? if so then why should we differentiate between loss and cost function at all?? @codebasics
Loss and cost are used interchangeably in machine learning
np.absy_predicted - y_true)
----> 4 np.absolute(y_predicted - y_true)
ValueError: operands could not be broadcast together with shapes (5,) (6,)
Amazing tutorial :)