NOTE: This StatQuest is sponsored by JADBIO. Just Add Data, and their automatic machine learning algorithms will do all of the work for you. For more details, see: bit.ly/3bxtheb BAM! Corrections: 3:42 I said 10 grams of popcorn, but I should have said 20 grams of popcorn given that they love Troll 2. Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
4 weeks back I had no idea what is machine learning, but your videos have really made a difference in my life, they are all so clearly explained and fun to watch, I just got a job and I mentioned some of the learnings I had from your channel, I am grateful for your contribution in my life.
Im at the point where my syllabus does not require me to look into all of this but im just having too much fun learning with you. Im glad i took this course up to find your videos
This is by far my favorite educational UA-cam channel. Everything is explained in a simple, practical and fun way. The videos are full of positive vibes just from the beginning with the silly song entry. I love the catch phrases. Statquest is addictive!
Thank you Josh. You deserve all the praises. I have been struggling with a lot of the concepts on traditional classic text books as they tend to "jump" quite a lot. You channel brings all of them to life vividly. This is my go to reference source now.
I am a beginner in Machine Learning field, and your channel helped me alot, almost went through all the videos, very nice way of explaining. Really appreciate you for making these videos and helping everyone. You just saved me ... Thank you very much...
WOOOOOOW. I watched every video of yours, recommended in the description of this video, and now this video. Everything makes much more sense now. It helped me a lot to undersand the Gaussian Naive Bayes algorithm implemented and available from scikit-learn for applications in machine learning. Just awesome. Thank you!!!
Hi, Josh. Thank you so much for all the exceptional content from your channel. Your work is amazing. I'm a professor in Brazil of Computer Science and ML and your videos have been supporting me a lot. You're an inspiration for me. Best.
It's amazing! Thank you so much ! Our professor let us self-teach the Gaussian naive bayes and I absolutely don't understand her slides with many many math equations. Thanks again for your vivid videos !!
This is crazy I went to school for Applied Mathematics and it never crossed my mind that what I learned was machine learning as chatgpt came into the lime light I started looking into it and almost everything I've learned so far is basically everything I've learned before but in a different context. My mind is just blown that I was assuming ML was something unattainable for me and it turns out I've been doing it for years
Thank you for the prompt response. I’m fairly new to Stats. But this video prompted me to do a lot more research and I’m finally confident on how you got to the result. Thank you for your videos. They are so helpful
😅😅😅😅It's the "Shameless Self Promotion" for me... Thank you very much for this channel. Your videos are gold. The way you just know how to explain these hard concepts in a way that 5-year-olds can understand... To think that I just discovered this goldmine this week. God bless you😇
In Stats Playlist, we used following notation for P( Data | Model ) for probability & L(Model | Data) for likelihood; Here we are writing likelihood as L(popcorn=20 | Loves) which I guess L( Data | Model );
Unfortunately the notation is somewhat flexible and inconsistent - not just in my videos, but in the the field in general. The important thing is to know that likelihoods are always the y-axis values, and probabilities are the areas.
@@statquest understood; somewhere in the playlist you mentioned that likelihood is relative probability; and I guess this neatly summaries how likelihood and probability
Daym, your videos are so good at explaining complicated ideas!! Like holy shoot, I am going to use this, multiple predictors ideas to figure out the ending of inception, Was it dream, or was it not a dream!
@@statquest I think you should explain some formula briefly. Like in Naive Bayes algorithm, you'd better explain why P(N)*P(Dear|N)*P(Friend|N)=P(N|Dear,Friend). I use GPT to finally understand it.
@@CyberGimen I've got a whole video about that here: ua-cam.com/video/9wCnvr7Xw4E/v-deo.html However, the reason I don't mention it in this video is that it's actually not critical to using the method.
contents are excellent and also i love your intro quite a lot (its super impressive for me) btw. thanking for doing this at the fisrt place as a beginner some concepts are literally hard to understand but after watching your videos things are a lot better than before. Thanks :)
Glad you liked it. It's a new feature that UA-cam just rolled out so I've spent the past day (and will spend the next few days) adding it to my videos.
Hey Josh I hope you are having a wonderful day, I was searching for a video on " Gaussian mixture model " on your channel but couldn't find one, I have a request for that video since the concept is a bit complicated elsewhere Also btw your videos enabled to get one of the highest scores in the test conducted recently in my college, all thanks to you Josh, you are awesome
Thanks for the great video! I would just like to point out that in my opinion if you are talking about log() when the base is e, it is easier (and more correct) to write ln().
Great style of teaching & also thank you so much for such a great video (Note : I have bought your book "The StatQuest illustrated guide to machine learning") 😃
Josh. I love you your videos. I've been following your channel for a while. Your videos are absolutely great! Would you consider covering more of Bayesian statistics in the future?
A really comprehensive video. Thank you! Sir, I have some questions about the conditions when applying this algo: 1. Is it compulsory that all features contain continuous value? 2. What happens if a feature doesn't have gaussian distribution? Is it worth to apply this algo? 3. If that, I will find a function that makes that feature have gaussian distribution. Can it work? And also, Do u plan to do a video about Bernoulli Naive Bayes?
1. No - you can mix things up. I illustrate this in my book. 2. You can use other distributions 3. No need, just use the other distribution. 4. Not in the short term.
These videos are extremely valuable, thank you for sharing them. I feel that they really help to illuminate the material. Quick question though: where do you get the different probabilities, like for popcorn, soda pop, and candy? How do we calculate those in this context? Do you use the soda a person drinks and divide it by the total soda, and same with popcorn, and candy?
What time point are you asking about (in minutes and seconds). The only probabilities we use in this video are if someone loves or doesn't love troll 2. Everything else is a likelihood, which is just a y-axis coordinate.
Hi - another great explanation! I wonder what would be the result if you normalise the probabilies of the 3 values. - Would it affect the outcome of the example in this video? - Which areas of values are affected: different outcomes with non-normalised and normalised distributions (=probability or likelihood here)?
@@statquest Hi, that only make sense with real data. Without that, only juggling with equations and abstract parameters, the thing is not enough 'visual', IMO. Though, could run through the calculations with e.g. 2x scale, 10x scale and 100x scale... Maybe, when I have free few hours.
Josh, a question about the formulation of Bayes' Theorem, especially considering the likelihood. For Naive Bayes, the formula is: P(class | X) = P(class) * P(X | class), in which the last term. is the likelihood In your video, you represented the likelihood as L, so that, apparently, the formula would be: P(No Love | X) = P(No Love) * L(X | No Love) (1) Is my assumption correct? Is it just a change of letters to mean the same thing? (2) Or is there any other math under the hoods? For example, something like: P(X | class) = L(No Love | X) Thanks in advance.
When I use the notation "L(something)" for "likelihood", I mean that we want the corresponding y-axis coordinate for that something. However, not everyone uses that notation. Some put p(something) and you have to figure out from the context whether or not they are talking about a likelihood (y-axis coordinate) or, potentially, a probability (since "p" often refers to "probability"). So, if you use my notation, then you are correct, you get: P(No Love | X) = P(No Love) * L(X | No Love)
Thanks for this super clear explanation. Why would we prefer this method for classification over a gradient boosting algorithm? When we have too few samples?
Hey JOSH Thanks for making such amazing video. Keep up the work. I just have a quick question if you don't mind. I can't understand how you got the likelihood eg: L(soda = 500 | LOVES) how you calculating that value.
We plugged the mean and standard deviation of soda pot for people that loved Troll2 into the equation for a normal curve and then determined the y-axis coordinate when the x-axis value = 500.
Dear Mr. Josh, I have taken another course have the following equation for the probability of Naive Bayes, P( Loves Troll 2 | new data ) = [ P( new data | Loves Troll 2 ) * P( Loves Troll 2 ) ] / P( new data) P( new data ) called marginal likelihood, and P( new data | Loves Troll 2 ) called likelihood And then, the way to calculate marginal likelihood and likelihood is to calculate the probability nearby the data at a certain distance, and the distance is adjustable while you are building the algorithm. For instance, there is a circle in which the center is new data and you can adjust the radius if your data is 2-D data. After watching both courses, I am wondering how can these two equations be equivalent? I deeply appreciate your time for answering my question. Sincerely, Gavin
The marginal likelihood is often omitted because both p(Loves Troll 2 | data) and p(Does not love Troll 2 | data) are divided by it. In other words, the only thing that makes p(Loves Troll 2 | data) different from p(Does not love Troll 2 | data) is what is in the numerator. And because it is usually really hard to calculate the marginal likelihood, we just omit it because it will not change the results.
No, probability is the area under the curve and those are defined such that the total area under the curve is always 1. For details, see: ua-cam.com/video/pYxNSUDSFH4/v-deo.html
@@statquest Dear Dr. Starmer, Thank you for your reply. I have another follow-up question regarding the calculation of probabilities for continuous random variables (i.e. what this video is about). From my understanding, when we have discrete random variables, the probability of a given outcome P(Y=y|X1,X2,..Xn) is proportional to the product of the probabilities of the individual variables given the outcome, times the prior probability (assuming conditional independence). i.e. P(Y=y) * the product of P(Xi=xi | Y=y) This makes sense to me, because the result is a probability value between 0 and 1. However, in the case of continuous random variables, the probability of a given outcome is zero, so we instead calculate the likelihood of the outcome. This means that the product of the individual likelihoods is no longer a probability value between 0 and 1. Is this correct? What I mean is: P(Y=y) * the product of L(Xi=xi | Y=y) is not guaranteed to be a value between 0 and 1. Thank you for your expertise and for being such a valuable educator. 💖
@@kirilblazevski8329 That's correct, with the continuous version, we do not end up with probabilities. However, if you saw my video on the discrete version of Naive Bayes ( ua-cam.com/video/O2L2Uv9pdDA/v-deo.html ) you'll notice that I call the results "scores" instead of probabilities. The reason for this is that in both cases (discrete and continuous), to get the correct probabilities for the results, you need to divide the results (what I call "scores") by the sum of the scores for the two possibilities. By doing this, you normalize the scores for the two possibilities so that they will add up to 1.
A nice video on Gaussian Naive Bayes Classification model. Well done! But I have a quick question for you, Josh. I only understand that Lim ln(x) as x approaches o is negative infinity. How is the Natural log of a really small unknown number very close to zero assumed to be equal to -115 and -33.6 as in the case of L(candy=25|Love Troll 2) and L(popcorn=20|does not Love Troll 2) respectively? What measure was used to determine these values?
Hi, Josh. Thanks for this clear explanation. Since this Naive Bayes could be applied to Gaussian distribution, I guess it could also be applied to other distributions like Poisson distribution, right? Then a question is: how to determine the distribution of a feature? I believe this will be quite important to build a reasonable model. Thanks again for the nice video.
One day (hopefully not too long from now), I'm going to cover the different distributions, and that should help people decide which distributions to use with their data.
Great work ! In 8:11 How can we use cross validation with Gaussian Naive Bayes? I have watched the Cross validation video but I still can't figure out how to employ cross validation to know that candy can make the best classification.
to apply cross validation, we divide the training data into different groups - then we use all of the groups, minus 1, to create a gaussian naive bayes model. Then we use that model to make predictions based on the last group. Then we repeat, each time using a different group to test the model.
Hello! Does it matter if the data in one of the columns (say popcorn) is not normally distributed? Or should the assumption be that we will have a large enough sample size to use the central limit theorem? Thanks for all of your videos! I love them and can’t wait for your book to be delivered (just ordered it yesterday).
It doesn't matter how the data are distributed. As long as we can calculate the likelihoods, we are good to go. BAM! :) And thank you so much for supporting StatQuest!!! TRIPLE BAM!!! :)
Can we also say that this person can be an outlier? Because of having very high likelihood of popcorn and soda pop scores given that he likes troll 2 and only but high variance according to 3rd category we can also say consider him under the outlier category, can't we? Can you clear this doubt for me, please! And also thanks a lot for your effort and work..
Maybe. It depends on how much data we have in the training dataset - because that will define how confident we are that we have correctly modeled the two categories.
@@statquest Yes! If the training dataset contains a good enough number of data then we can calculate the margin of error too at various confidence levels with the given sample size and present our output. Thank you!
3:38, shouldn’t the notation be L(Loves | popcorn=20), since we’re given that he eats 20g of popcorn, how likely is that sample generated from the Loves distribution. Isn’t that right?
Error in Video ("Probably") Hey Josh, I think you should be saying "Probability" when you are saying Likelihood. I know the distinction can be tough, but they are VERY different, so it is important to get right. And I believe you explain it correctly in your other video, so I'm surprised to see it used incorrectly here. When you already have a known distribution (not changing), and you calculate a value for an event based on that (known) distribution, that is a Probability. "Likelihood" occurs when you have some real data, but you're not sure what the "true" distribution is that the data came from. So you take a stab and pick one possible distribution, and then calculate the "likelihood" that the data came from that distribution that you guessed. And seriously, I think that the difference between "probability" and "likelihood" is as big as consequential as the difference between differentiation and integration, so it's really important to clearly distinguish the two concepts. Or am I wrong?
With continuous distributions, the probability of an individual event is always 0, this is because probability is the area under the curve between two points. If we only have 1 point, then the width of the area is 0 and thus, the probability is 0. So, in Naive Bayes we do not use probability, instead we use the y-axis coordinate, which I call the likelihood because that is what the y-axis coordinates are called when we do "maximum likelihood". I know that statisticians only call it a likelihood when we are specifically changing parameters and holding the data as fixed, but I believe we can also call it a likelihood since the values are equal to each other and they are calculated the same way. In other words, if x = a number that is calculated the same way as a likelihood and is the same value as a likelihood, then I'll say x is equal to a likelihood.
@@statquest Hmmm. Well, in the beginning, you take training data, and just ASSUME that the distributions of the data in each variable is normal, when you actually have no idea whether the distributions of each variable are actually "normal"/gaussian. All of the values that I suggested should be called "probabilities" are calculated based on that assumption that the distributions are normal. But since we don't know that they are normal, I suppose they truly are "likelihoods". They are the likelihoods of the normal distribution assumption being correct given the actual data. So maybe that is why they are called "Likelihoods" in Naive Bayes.
@@adamkinsey3139 In statistics, when we have a continuous distribution like the normal distribution, likelihoods are distinct from probabilities (for details, see: ua-cam.com/video/pYxNSUDSFH4/v-deo.html ). That said, we do assume that the distribution is normal, however, that is not a requirement. We can assume that the data come from different distributions and the method will work (in a technical sense) just as well. Whether or not the results will be improved, however, depends on the actual underlying distributions.
@@atravellingstudent PDF is just the curve for a continuous distribution if we integrate that curve between two points, we get the probability of something happening between those two points. If we look at the y-axis value for a specific point, we get the likelihood. For details, see: ua-cam.com/video/pYxNSUDSFH4/v-deo.html
Thanks for these great videos! Quick question: In other resources the likelihood is actually the probability of the data given the hypothesis rather than the likelihood of the data given the hypothesis. Which one would be correct, or is it fine to use either?
Generally speaking, you use the likelihoods of the data. However, we can normalize them to be probabilities. This does not offer an advantages and takes longer to do, so people usually omit that step and just use the likelihoods.
NOTE: This StatQuest is sponsored by JADBIO. Just Add Data, and their automatic machine learning algorithms will do all of the work for you. For more details, see: bit.ly/3bxtheb BAM!
Corrections:
3:42 I said 10 grams of popcorn, but I should have said 20 grams of popcorn given that they love Troll 2.
Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
website not working?
@@phildegreat Thanks! The site is back up.
8:15 There's a minor error in the slide 'help use decide' .
You really are a great teacher.Wish I could Meet you in person some day.
4 weeks back I had no idea what is machine learning, but your videos have really made a difference in my life, they are all so clearly explained and fun to watch, I just got a job and I mentioned some of the learnings I had from your channel, I am grateful for your contribution in my life.
Happy to help!
Congratulations!!
That is a HUGE help my friend, congrats.. !!
Im at the point where my syllabus does not require me to look into all of this but im just having too much fun learning with you. Im glad i took this course up to find your videos
Hooray! :)
Following your channel for over 6 months now sir, your explanations are truly amazing..
Thank you very much! :)
This is by far my favorite educational UA-cam channel.
Everything is explained in a simple, practical and fun way.
The videos are full of positive vibes just from the beginning with the silly song entry. I love the catch phrases.
Statquest is addictive!
Thank you very much! :)
My little knowledge about machine learning could not be derived without your tutorials. Thank you very much
Glad I could help!
If I remember all the best educator's name on UA-cam, you always come at the beginning! You are a flawless genius!
Thank you! 😃
I have watched over 2-3 hours of lecture about Gaussian Naive Bayes. Now is when I feel my understanding is complete.
Hooray!
Thank you Josh. You deserve all the praises. I have been struggling with a lot of the concepts on traditional classic text books as they tend to "jump" quite a lot. You channel brings all of them to life vividly. This is my go to reference source now.
Awesome! I'm glad my videos are helpful.
I am a beginner in Machine Learning field, and your channel helped me alot, almost went through all the videos, very nice way of explaining. Really appreciate you for making these videos and helping everyone. You just saved me ... Thank you very much...
Thank you very much! :)
One of the best channel for learners that the world can offer..
Thank you!
WOOOOOOW. I watched every video of yours, recommended in the description of this video, and now this video. Everything makes much more sense now. It helped me a lot to undersand the Gaussian Naive Bayes algorithm implemented and available from scikit-learn for applications in machine learning. Just awesome. Thank you!!!
Wow, thanks!
amazing kowledge with incredible communication skills..world will change if every student has such great teacher
Thank you!
Hi, Josh.
Thank you so much for all the exceptional content from your channel.
Your work is amazing.
I'm a professor in Brazil of Computer Science and ML and your videos have been supporting me a lot.
You're an inspiration for me.
Best.
Muito obrigado!
It's amazing! Thank you so much !
Our professor let us self-teach the Gaussian naive bayes and I absolutely don't understand her slides with many many math equations. Thanks again for your vivid videos !!
Glad it was helpful!
This is the only lecture that makes me feel not stupid...
:)
This is crazy I went to school for Applied Mathematics and it never crossed my mind that what I learned was machine learning as chatgpt came into the lime light I started looking into it and almost everything I've learned so far is basically everything I've learned before but in a different context. My mind is just blown that I was assuming ML was something unattainable for me and it turns out I've been doing it for years
bam!
same applied math undergraduate student who switched to AI field as a postgraduate student now🙂
Literally the best video ever on this.
Thank you!
Thank you for the prompt response. I’m fairly new to Stats. But this video prompted me to do a lot more research and I’m finally confident on how you got to the result. Thank you for your videos. They are so helpful
Glad it was helpful!
Sir, this playlist is a one-stop solution for quick interview preparations. Thanks a lot sir.
Good luck with your interviews! :)
this was the best explanation i've ever seen in my life, (i'm not even a english native speaker, i'm brazilian lol)
Muito obrigado! :)
Great video! If people are willing to spend time on videos like this rather than Tiktok, the wold would be a much better place.
Thank you very much! :)
😅😅😅😅It's the "Shameless Self Promotion" for me... Thank you very much for this channel. Your videos are gold. The way you just know how to explain these hard concepts in a way that 5-year-olds can understand... To think that I just discovered this goldmine this week.
God bless you😇
Thank you very much! :)
This channel has helped me so much during my studies 🎉
Happy to hear that!
you explained much clearer than my lecturer in ML lecture.
Thanks!
These videos are amazing !!! Truly a survival pack for my DS class👍
Bam! :)
In Stats Playlist, we used following notation for P( Data | Model ) for probability & L(Model | Data) for likelihood;
Here we are writing likelihood as L(popcorn=20 | Loves) which I guess L( Data | Model );
Unfortunately the notation is somewhat flexible and inconsistent - not just in my videos, but in the the field in general. The important thing is to know that likelihoods are always the y-axis values, and probabilities are the areas.
@@statquest understood; somewhere in the playlist you mentioned that likelihood is relative probability; and I guess this neatly summaries how likelihood and probability
I just had the exact same question when I started writing the expression in my notebook. I am more acquainted with the L(Model | Data) notation.
This channel should have 2.74M subscribers instead of 274K.
One day I hope that happens! :)
Your videos and voice make ML and statistics fun to learn. :)
Glad you like them!
This video on Gaussian Naive Bayes has been very well explained. Thanks a lot.😊
Most welcome 😊
These gloriously wierd examples really are needed to understand a concept
Thanks!
Daym, your videos are so good at explaining complicated ideas!! Like holy shoot, I am going to use this, multiple predictors ideas to figure out the ending of inception, Was it dream, or was it not a dream!
BAM! :)
superb cool explanation. I am big fan of your explanation. Once I went through your explanation, I don't want any further reference for that topic.
Thanks!
This series is helping me so much with my dissertation, thank you!!
Awesome and good luck with your disertation!
Thank you, You have made the theory concrete and visible!
Thanks!
Bam! I love your teaching style!!!
Thanks!
@@statquest I think you should explain some formula briefly. Like in Naive Bayes algorithm, you'd better explain why P(N)*P(Dear|N)*P(Friend|N)=P(N|Dear,Friend). I use GPT to finally understand it.
@@CyberGimen I've got a whole video about that here: ua-cam.com/video/9wCnvr7Xw4E/v-deo.html However, the reason I don't mention it in this video is that it's actually not critical to using the method.
How do people come up with these crazy ideas? it's amazing, thanks a lot for another fantastic video
Thank you again!
contents are excellent and also i love your intro quite a lot (its super impressive for me) btw. thanking for doing this at the fisrt place as a beginner some concepts are literally hard to understand but after watching your videos things are a lot better than before. Thanks :)
I'm glad my videos are helpful! :)
The world needs more Joshuas!
Thanks! :)
You have really helped me a lot. Thanks Sir. May you prosper more and keep helping students who cant afford paid content :)
Thank you! :)
The demarcation of topics in the seek bar is useful and helpful. Nice addition.
Glad you liked it. It's a new feature that UA-cam just rolled out so I've spent the past day (and will spend the next few days) adding it to my videos.
@@statquest We really appreciate all your dedication into the channel!
It's 100% awesomeness :)
@@anitapallenberg690 Hooray! Thank you! :)
i promise i will join the membership and buy your products when i get a job... BAM!!!
Hooray! Thank you very much for your support!
+5000 for using an example as obscure and as obscene as Troll 2.
:)
Hey Josh I hope you are having a wonderful day, I was searching for a video on " Gaussian mixture model " on your channel but couldn't find one, I have a request for that video since the concept is a bit complicated elsewhere
Also btw your videos enabled to get one of the highest scores in the test conducted recently in my college, all thanks to you Josh, you are awesome
Thanks! I'll keep that topic in mind.
Why the fuck does this video make it look so easy and makes 100 percent sense?
I'm Having great time watching Ur videos ❤️
Thanks!
can't wait for your channel to BAAM! going worldwide!!
Me too!!
Thank you for another excellent Statquest !~
Bam! :)
Thanks for the great video!
I would just like to point out that in my opinion if you are talking about log() when the base is e, it is easier (and more correct) to write ln().
In statistics, programming and machine learning, "ln()" is written "log()", so I'm just following the conventions used in the field.
Great style of teaching & also thank you so much for such a great video (Note : I have bought your book "The StatQuest illustrated guide to machine learning") 😃
Thank you so much for supporting StatQuest!
I'm a simple man, I watch statquests in the nights, leave a like and go chat about it with chatgpt.That's it.
bam! :)
Josh. I love you your videos. I've been following your channel for a while. Your videos are absolutely great!
Would you consider covering more of Bayesian statistics in the future?
I'll keep it in mind.
Thank you Josh for another great video! Also, this (and other vids) makes think I should watch Troll 2, just to tick that box.
Ha! Let me know what you think!
Your videos are really great !! my prof made it way harder!!
Thanks!
Thanks for the video !! it was very helpful and easy to understand
Glad it was helpful!
A really comprehensive video. Thank you!
Sir, I have some questions about the conditions when applying this algo:
1. Is it compulsory that all features contain continuous value?
2. What happens if a feature doesn't have gaussian distribution? Is it worth to apply this algo?
3. If that, I will find a function that makes that feature have gaussian distribution. Can it work?
And also, Do u plan to do a video about Bernoulli Naive Bayes?
1. No - you can mix things up. I illustrate this in my book.
2. You can use other distributions
3. No need, just use the other distribution.
4. Not in the short term.
Best video i have ever seen
:)
Another great tutorial, thank you!
Thanks!
These videos are extremely valuable, thank you for sharing them. I feel that they really help to illuminate the material.
Quick question though: where do you get the different probabilities, like for popcorn, soda pop, and candy? How do we calculate those in this context? Do you use the soda a person drinks and divide it by the total soda, and same with popcorn, and candy?
What time point are you asking about (in minutes and seconds). The only probabilities we use in this video are if someone loves or doesn't love troll 2. Everything else is a likelihood, which is just a y-axis coordinate.
Troll 2 is an awesome classic, and should not be up for debate. =)
Ha! :)
Your video just helped me a lot !
Glad it helped!
Tqsm Sir for the Very Valuable Information
Thanks! :)
Hi - another great explanation!
I wonder what would be the result if you normalise the probabilies of the 3 values.
- Would it affect the outcome of the example in this video?
- Which areas of values are affected: different outcomes with non-normalised and normalised distributions (=probability or likelihood here)?
Interesting questions! You should try it out and see what you get.
@@statquest Hi, that only make sense with real data. Without that, only juggling with equations and abstract parameters, the thing is not enough 'visual', IMO. Though, could run through the calculations with e.g. 2x scale, 10x scale and 100x scale... Maybe, when I have free few hours.
Hi, Josh. Troll 2 is a good movie... Thanks
bam!
Super awesome, thank you. Useful for my Intro to Artificial Intelligence course.
Glad it was helpful!
Hey Josh, Thank you for making these amazing videos. Please make a video on the "Bayesian Networks" too.
I'll keep it in mind.
Excellent explanation. Any NLP series coming up ? Struggling to find good resources.
I'm working on Neural Networks right now.
@@statquest it's going to be BAM!!
Can you talk about Kernel estimation in the future?? Bam!
I will consider it.
BAM! Someone is going to pass the exam this semester .
Hooray!
Finally worked up to the Gaussian Naive Bayes. BAM! "If you are not familiar with
..." :(
You can do it! :)
StatQuest made me lose my anxiety for statistics. It's truly brilliant, just start with the next video!
BAM! :)
Josh, a question about the formulation of Bayes' Theorem, especially considering the likelihood.
For Naive Bayes, the formula is:
P(class | X) = P(class) * P(X | class), in which the last term. is the likelihood
In your video, you represented the likelihood as L, so that, apparently, the formula would be:
P(No Love | X) = P(No Love) * L(X | No Love)
(1) Is my assumption correct? Is it just a change of letters to mean the same thing?
(2) Or is there any other math under the hoods?
For example, something like: P(X | class) = L(No Love | X)
Thanks in advance.
When I use the notation "L(something)" for "likelihood", I mean that we want the corresponding y-axis coordinate for that something. However, not everyone uses that notation. Some put p(something) and you have to figure out from the context whether or not they are talking about a likelihood (y-axis coordinate) or, potentially, a probability (since "p" often refers to "probability"). So, if you use my notation, then you are correct, you get: P(No Love | X) = P(No Love) * L(X | No Love)
So great, this video so helpful
Glad it was helpful!
I dont even know why there is people disliking this video!!
It's always a mystery. :)
Thanks for this super clear explanation. Why would we prefer this method for classification over a gradient boosting algorithm? When we have too few samples?
With relatively small datasets it's simple and fast and super lightweight.
Love the explaination BAM!
BAM! :)
BAM! thanks, Josh! It would be amazing if you can make a StatQuest concerning A/B testing :)
It's on the to-do list. :)
Hey JOSH Thanks for making such amazing video. Keep up the work. I just have a quick question if you don't mind.
I can't understand how you got the likelihood eg: L(soda = 500 | LOVES) how you calculating that value.
We plugged the mean and standard deviation of soda pot for people that loved Troll2 into the equation for a normal curve and then determined the y-axis coordinate when the x-axis value = 500.
Love your channel
Thanks!
Dear Mr. Josh,
I have taken another course have the following equation for the probability of Naive Bayes,
P( Loves Troll 2 | new data ) = [ P( new data | Loves Troll 2 ) * P( Loves Troll 2 ) ] / P( new data)
P( new data ) called marginal likelihood,
and P( new data | Loves Troll 2 ) called likelihood
And then, the way to calculate marginal likelihood and likelihood is to calculate the probability nearby the data at a certain distance, and the distance is adjustable while you are building the algorithm. For instance, there is a circle in which the center is new data and you can adjust the radius if your data is 2-D data.
After watching both courses, I am wondering how can these two equations be equivalent?
I deeply appreciate your time for answering my question.
Sincerely,
Gavin
The marginal likelihood is often omitted because both p(Loves Troll 2 | data) and p(Does not love Troll 2 | data) are divided by it. In other words, the only thing that makes p(Loves Troll 2 | data) different from p(Does not love Troll 2 | data) is what is in the numerator. And because it is usually really hard to calculate the marginal likelihood, we just omit it because it will not change the results.
Awesome as always
Thanks again! :)
Looks like I have to check out the quests before getting to this one😂
:)
Amazing videos. The beep boop sound reminds me of squid games
Maybe they got the sound from my video! :)
So we use Gaussian when ALL our features are continuous and multinomial when ALL our features are categorical?
You can mix them by multiplying the different likelihoods. For more details, see: sebastianraschka.com/faq/docs/naive-bayes-vartypes.html
Great video!
Thanks!
Since the likelihood can be greater than 1, doesn't that mean that we could get probability that is greater than 1?
No, probability is the area under the curve and those are defined such that the total area under the curve is always 1. For details, see: ua-cam.com/video/pYxNSUDSFH4/v-deo.html
@@statquest
Dear Dr. Starmer,
Thank you for your reply. I have another follow-up question regarding the calculation of probabilities for continuous random variables (i.e. what this video is about).
From my understanding, when we have discrete random variables, the probability of a given outcome P(Y=y|X1,X2,..Xn) is proportional to the product of the probabilities of the individual variables given the outcome, times the prior probability (assuming conditional independence).
i.e. P(Y=y) * the product of P(Xi=xi | Y=y)
This makes sense to me, because the result is a probability value between 0 and 1.
However, in the case of continuous random variables, the probability of a given outcome is zero, so we instead calculate the likelihood of the outcome. This means that the product of the individual likelihoods is no longer a probability value between 0 and 1. Is this correct?
What I mean is: P(Y=y) * the product of L(Xi=xi | Y=y) is not guaranteed to be a value between 0 and 1.
Thank you for your expertise and for being such a valuable educator. 💖
@@kirilblazevski8329 That's correct, with the continuous version, we do not end up with probabilities. However, if you saw my video on the discrete version of Naive Bayes ( ua-cam.com/video/O2L2Uv9pdDA/v-deo.html ) you'll notice that I call the results "scores" instead of probabilities. The reason for this is that in both cases (discrete and continuous), to get the correct probabilities for the results, you need to divide the results (what I call "scores") by the sum of the scores for the two possibilities. By doing this, you normalize the scores for the two possibilities so that they will add up to 1.
@@statquest Now I understand what I was missing. Thank you for clarifying, I really appreciate it!!
Thanks for the awesome explanation. But I've a question. Is GNB can be used for sentiment analysis?
Presumably you could use GNB, but I also know that normal NB (aka multinomial naive bayes) is used for sentiment analysis.
A nice video on Gaussian Naive Bayes Classification model. Well done! But I have a quick question for you, Josh. I only understand that Lim ln(x) as x approaches o is negative infinity. How is the Natural log of a really small unknown number very close to zero assumed to be equal to -115 and -33.6 as in the case of L(candy=25|Love Troll 2) and L(popcorn=20|does not Love Troll 2) respectively? What measure was used to determine these values?
log(1.1*10^-50) = -115 and log(2.5*10^-15) = -33.6
Hi, Josh. Thanks for this clear explanation. Since this Naive Bayes could be applied to Gaussian distribution, I guess it could also be applied to other distributions like Poisson distribution, right? Then a question is: how to determine the distribution of a feature? I believe this will be quite important to build a reasonable model.
Thanks again for the nice video.
One day (hopefully not too long from now), I'm going to cover the different distributions, and that should help people decide which distributions to use with their data.
Great work ! In 8:11 How can we use cross validation with Gaussian Naive Bayes? I have watched the Cross validation video but I still can't figure out how to employ cross validation to know that candy can make the best classification.
to apply cross validation, we divide the training data into different groups - then we use all of the groups, minus 1, to create a gaussian naive bayes model. Then we use that model to make predictions based on the last group. Then we repeat, each time using a different group to test the model.
Tough being a ML teacher these days with you around
bam!
People who don't like this video are obviously trolls
Ha! :)
Hello! Does it matter if the data in one of the columns (say popcorn) is not normally distributed? Or should the assumption be that we will have a large enough sample size to use the central limit theorem?
Thanks for all of your videos! I love them and can’t wait for your book to be delivered (just ordered it yesterday).
It doesn't matter how the data are distributed. As long as we can calculate the likelihoods, we are good to go. BAM! :) And thank you so much for supporting StatQuest!!! TRIPLE BAM!!! :)
Could you please make a video on Time Series Analysis (Arima model)?
One day I'll do that.
Can we also say that this person can be an outlier? Because of having very high likelihood of popcorn and soda pop scores given that he likes troll 2 and only but high variance according to 3rd category we can also say consider him under the outlier category, can't we? Can you clear this doubt for me, please! And also thanks a lot for your effort and work..
Maybe. It depends on how much data we have in the training dataset - because that will define how confident we are that we have correctly modeled the two categories.
@@statquest
Yes!
If the training dataset contains a good enough number of data then we can calculate the margin of error too at various confidence levels with the given sample size and present our output.
Thank you!
3:38, shouldn’t the notation be L(Loves | popcorn=20), since we’re given that he eats 20g of popcorn, how likely is that sample generated from the Loves distribution. Isn’t that right?
The notation in the video is most common, however, the notation doesn't really matter as long as it is clear that we want the y-axis coordinate.
Thank you josh your videos are amazing! HoW to buy study guides from statquest
See: statquest.gumroad.com/
awesome stuff for real
Thank you!
Error in Video ("Probably")
Hey Josh, I think you should be saying "Probability" when you are saying Likelihood.
I know the distinction can be tough, but they are VERY different, so it is important to get right. And I believe you explain it correctly in your other video, so I'm surprised to see it used incorrectly here.
When you already have a known distribution (not changing), and you calculate a value for an event based on that (known) distribution, that is a Probability.
"Likelihood" occurs when you have some real data, but you're not sure what the "true" distribution is that the data came from. So you take a stab and pick one possible distribution, and then calculate the "likelihood" that the data came from that distribution that you guessed.
And seriously, I think that the difference between "probability" and "likelihood" is as big as consequential as the difference between differentiation and integration, so it's really important to clearly distinguish the two concepts.
Or am I wrong?
With continuous distributions, the probability of an individual event is always 0, this is because probability is the area under the curve between two points. If we only have 1 point, then the width of the area is 0 and thus, the probability is 0. So, in Naive Bayes we do not use probability, instead we use the y-axis coordinate, which I call the likelihood because that is what the y-axis coordinates are called when we do "maximum likelihood". I know that statisticians only call it a likelihood when we are specifically changing parameters and holding the data as fixed, but I believe we can also call it a likelihood since the values are equal to each other and they are calculated the same way. In other words, if x = a number that is calculated the same way as a likelihood and is the same value as a likelihood, then I'll say x is equal to a likelihood.
@@statquest Hmmm. Well, in the beginning, you take training data, and just ASSUME that the distributions of the data in each variable is normal, when you actually have no idea whether the distributions of each variable are actually "normal"/gaussian. All of the values that I suggested should be called "probabilities" are calculated based on that assumption that the distributions are normal. But since we don't know that they are normal, I suppose they truly are "likelihoods". They are the likelihoods of the normal distribution assumption being correct given the actual data. So maybe that is why they are called "Likelihoods" in Naive Bayes.
@@adamkinsey3139 In statistics, when we have a continuous distribution like the normal distribution, likelihoods are distinct from probabilities (for details, see: ua-cam.com/video/pYxNSUDSFH4/v-deo.html ). That said, we do assume that the distribution is normal, however, that is not a requirement. We can assume that the data come from different distributions and the method will work (in a technical sense) just as well. Whether or not the results will be improved, however, depends on the actual underlying distributions.
Hi Josh, as always thanks so much for the very informative video!!! Quick question, how did you calculate for the likelihoods? :D
I plugged the x-axis coordinate, the mean and the standard deviation into the dnorm() function in R.
@@statquest I am confused, I thought pdf != likelihood, but videos suggest otherwise.
@@atravellingstudent PDF is just the curve for a continuous distribution if we integrate that curve between two points, we get the probability of something happening between those two points. If we look at the y-axis value for a specific point, we get the likelihood. For details, see: ua-cam.com/video/pYxNSUDSFH4/v-deo.html
Thanks for these great videos! Quick question: In other resources the likelihood is actually the probability of the data given the hypothesis rather than the likelihood of the data given the hypothesis. Which one would be correct, or is it fine to use either?
Generally speaking, you use the likelihoods of the data. However, we can normalize them to be probabilities. This does not offer an advantages and takes longer to do, so people usually omit that step and just use the likelihoods.