suggestion at time 6:45 minutes, the y values aren't the probabilities of the x values, intuitively the probability for a single point on the gaussian will be 0.
3:44 Intro, Gaussian Distribution, Probability Density Function (PDF) 7:38 GMM Intro 9:08 Covariance matrix 10:15 GMM Definition, K Gaussians 11:30 How to apply GMM for classification 12:30 Problem statement, Fitting a GMM model, Maximum Likelihood Estimate (MLE) 13:58 Similarity to Kmeans clustering algorithm 16:13 Expectation maximization (EM) algorithm and difference to Gradient Descent 18:15 When to apply GMM, anomaly detection, clustering, object tracking 19:30 Coding example with Python 25:10 EM algorithm workflow in practice, Log Likelihood 27:54 EM algorithm visual / walkthrough 36:30 Summary great video, many Thanks :)
I have the problem with the gaussian mixture models, I don't know how generate outliers uniformly in the p-parallelotope defined by the coordinate-wise maxima and minima of the ‘regular’ observations in R?
I got pretty confused around 33:33 with the E step. You've computed wp1 and wp2, which is cool, and then normalised them so their sum is 1 [wp1/(wp1+wp2) + wp2/(wp1+wp2) = (wp1+wp2)/(wp1+wp2) = 1], which makes sense. You then add the log of this sum to self.loglike. But the log of 1 is 0... Which is where you lost me.
I have some questions: 1. In the end, what we achieved: probability distribution of people whether they keep playing the game? 2. May it cause overfitting if we set too many gaussian distributions? Regards.
Hi. Great again Siraj. You're the best on that online apparently. Should we have a video about non-parametric estimation or Higher Order statistics, perhaps ICA?
I keep getting this error : AttributeError Traceback (most recent call last) in 10 try: 11 mix.iterate() ---> 12 if mix.loglike > best_loglike: 13 best_loglike = mix.loglike 14 best_mix = mix AttributeError: 'GaussianMixture' object has no attribute 'loglike' I am not sure what to do in this case. Any ideas? Thank you
He makes mistakes... If only that was the only one... Referring to Variance as Variation... Doesn't know how a Standard Deviation is calculated... omg.
Would be nice with timestamps, since it is quite impossible to find the bit of information about Gaussian mixture models that I was actually looking for...
Hi, Im following this channel for a while now and love that you create different series. can you make a small series of basic examples next, so it's easier to learn and get started. With one of your first videos I've just created an sklearn programm that had 50 examples of fruit and car names and with KNN I've got pretty good results. but they are not perfect. now I want to use deep learning for that and would love to see a series where you give different simple examples like this to compare and get started using the different libaries and algorithms. And yes you created some beautiful similar content before but it's not exactly that. Best Wishes
I am trying to use your notebook and getting this error -- any ideas?? I am getting an error for #checking the fitting process AttributeError: 'GaussianMixture' object has no attribute 'loglike'
Siraj I have a question/problem. I have two data inputs which is to be comparatively trained by a learning model. It's not a multiple set of data but only one. It's a set of pair of inputs. I have been reading pairwise svm. How do I do that? Is there a better model.
Another great tutorial, Siraj. By the way, if anyone gets an error seaborn deprecation and being unable to plot histograms, I resolved it by going into the linked 3rd party lib seaborn/distrubutions.py and changing "normed" to "density"
warning: when he finger styles his hair, get ready for hardcore info dump. PS: 3blue1brown series on linear algebra has THE BEST vid on eigen vectors/value pairs, no joking.
In case you have bad results using Gaussian mixtures, keep in mind the EM optimization only has local convergence properties, just like gradient descent: it can get stuck. Restarting the the density estimation with other initial parameters might solve it ! :)
Siraj, I think it would of been helpful if you showed the resulting clusters that you get from the gaussian mixture model approach in your data. You showed how to model your data using the gaussian mixture, but I am unclear on how we get the specific clusters (say 2 clusters) from that?
Hi all! I've fixed some bugs from the original .ipynb so you are welcome to try my one: github.com/ivanpastukhov/Gaussian_Mixture_Models/blob/master/intro_to_gmm_%26_em.ipynb Siraj, thanks a lot for your brilliant videos!
Great video, I tried running your code on my terminal and it's giving the error that 'GaussianMixture' object has no attribute 'loglike', would you happen to know why an error like would occur, or anyone by that matter. Thank you so much
I watch 4-5 vídeos of you per day. I'm Learning generative models for drug Design Siraj. Watch your videos not only motivates me, also makes my life & study fun and cool.
So... 38 minutes to predict something and he just forgets about the prediction part? I'm sorry, but the justification on 36:02 is not enough for my satisfaction.
Thank you! Your videos helped me a lot... I was so lost and confused about this topic that I was on the verge of giving up. Checked out your tutorials that gave a lot of useful information and insights. Thanks a tonne! :) :D Keep up the good stuff
Hi Siraj, I appreciate your videos and I love your content. I' am working on a project on cross-matching using active learning, what advice would you have for me? I' am trying to build something scalable but not so computationally intense.
hey siraj ! EM is a heuristic with no guarantees for global convergence. there have been recent algorithms based on method of moments, random projections etc. which provably recover the gmm under some assumptions
You guess a theta ( model params) , then that gives you a probability distribution of the hidden variables. With that known, you maximize the joint probability distribution of X and the hidden variables. That gives you a new theta. Repeat the 2 steps above: use the new theta model params instead of your guess.
Hey Siraj! Just found your channel and it doesn't cease to amaze. I am learning a lot about AI and ML with your vibrant and enthusiastic expression. My 2 cents would be to talk a tiny bit slower but it is up to you. Congrats and Keep up the Good Work!
From a muddy blur to crystal clear in 30 min, thank you very much for this video Siraj
I love how passionate you are about this
suggestion at time 6:45 minutes, the y values aren't the probabilities of the x values, intuitively the probability for a single point on the gaussian will be 0.
3:44 Intro, Gaussian Distribution, Probability Density Function (PDF)
7:38 GMM Intro
9:08 Covariance matrix
10:15 GMM Definition, K Gaussians
11:30 How to apply GMM for classification
12:30 Problem statement, Fitting a GMM model, Maximum Likelihood Estimate (MLE)
13:58 Similarity to Kmeans clustering algorithm
16:13 Expectation maximization (EM) algorithm and difference to Gradient Descent
18:15 When to apply GMM, anomaly detection, clustering, object tracking
19:30 Coding example with Python
25:10 EM algorithm workflow in practice, Log Likelihood
27:54 EM algorithm visual / walkthrough
36:30 Summary
great video, many Thanks :)
the iteration function is empty, which makes the current code completely random, it should be "mix.Mstep(mix.Estep())" inside that function
Like he understands that
The butt kissing ends at 3:40
Thanks. Haha
Your accent reminds me of Mitchell from Modern Family(fav character) :')
Also great video thanks!!
I have the problem with the gaussian mixture models, I don't know how generate outliers uniformly in the p-parallelotope defined by the
coordinate-wise maxima and minima of the ‘regular’ observations in R?
I got pretty confused around 33:33 with the E step. You've computed wp1 and wp2, which is cool, and then normalised them so their sum is 1 [wp1/(wp1+wp2) + wp2/(wp1+wp2) = (wp1+wp2)/(wp1+wp2) = 1], which makes sense. You then add the log of this sum to self.loglike. But the log of 1 is 0... Which is where you lost me.
You are right! Siraj should check and fix that with UA-cam annotations.
Agree
I have some questions:
1. In the end, what we achieved: probability distribution of people whether they keep playing the game?
2. May it cause overfitting if we set too many gaussian distributions?
Regards.
you are the best source of ML... thanks for your attention(s) and love to AI!!!!!
Hi. Great again Siraj. You're the best on that online apparently. Should we have a video about non-parametric estimation or Higher Order statistics, perhaps ICA?
I keep getting this error :
AttributeError Traceback (most recent call last)
in
10 try:
11 mix.iterate()
---> 12 if mix.loglike > best_loglike:
13 best_loglike = mix.loglike
14 best_mix = mix
AttributeError: 'GaussianMixture' object has no attribute 'loglike'
I am not sure what to do in this case. Any ideas?
Thank you
whether wp1 + wp2 = 1 always...so self.loglike += log(wp1 + wp2) will be zero ????
Is it true ?? whether my assumption is wrong ??
Kindly explain...
He makes mistakes... If only that was the only one... Referring to Variance as Variation... Doesn't know how a Standard Deviation is calculated... omg.
Would be nice with timestamps, since it is quite impossible to find the bit of information about Gaussian mixture models that I was actually looking for...
Hi, Im following this channel for a while now and love that you create different series. can you make a small series of basic examples next, so it's easier to learn and get started. With one of your first videos I've just created an sklearn programm that had 50 examples of fruit and car names and with KNN I've got pretty good results. but they are not perfect. now I want to use deep learning for that and would love to see a series where you give different simple examples like this to compare and get started using the different libaries and algorithms. And yes you created some beautiful similar content before but it's not exactly that. Best Wishes
Code doesn't work, There is a problem in GaussianMixture class.
You can visit us in Uruguay! Everyone is welcome in Uruguay and especially, people who motivate the world to be better, like you @siraj!
@Siraj
, why do you change the formula at 29:54? instead of sigma^2 you are using abs(sigma).
Hi, how to change the variance and average Gaussian function in matlab? Can you show an example of what the code looks like?
pdf does not give the probability. It gives the probability density at that x
Hey @siraj where are you going to be in India would love to catch up
Hey Siraj thank you. If you ever come to México, you'll have a room, a meal, a beer and a friend :)
I am trying to use your notebook and getting this error -- any ideas??
I am getting an error for #checking the fitting process
AttributeError: 'GaussianMixture' object has no attribute 'loglike'
Siraj I have a question/problem. I have two data inputs which is to be comparatively trained by a learning model. It's not a multiple set of data but only one. It's a set of pair of inputs. I have been reading pairwise svm. How do I do that? Is there a better model.
could you please show an example on 3d data (XYZ - points) ?
Siraj, any plans on coming to Germany with in the future?
Hey Siraj! Come down to Mumbai for some beers and nerding out!
You is amazing! Siraj!
Great! u solved smartly my doubts... thanks man =)
Is there a guide in how to set up jupyter notebook?
If you aren't already using Python, use the anaconda distribution. www.continuum.io. It will also include the most useful libraries.
what if we had more than one column to train?? anyone ??
Should we do same processing twice or combine dataset into one row?
Great presentation and really well explained! Are you using AWS Sagemaker for this?
Thank you. Very helpful video. :)
I still can't understand why the bell curve turns into circle, could someone provide me with an explanation,please .
It's top view of 3d hill shaped bell curve....which looks like a circle with its centre being the pick of curve
why you did not explain the code
6:45 y is not the probability. y is the "likelihood" because the probability function is a pdf.
yeah probability is the area under the curve
Another great tutorial, Siraj. By the way, if anyone gets an error seaborn deprecation and being unable to plot histograms, I resolved it by going into the linked 3rd party lib seaborn/distrubutions.py and changing "normed" to "density"
Hi siraj iam working on helmet detection model can u hlp me out!
I believe, the objective is to maximize the likelihood of observed data, not the observed data and the hidden variables.
That was an amazing intro! Great videos man!
subscribed :)
Awesome !!!!!!
you are awesome!
you are amazing (y)
please explain Expectation Maximization.. i em not understanding and crying please explain.
Holy shit you're fkkin awesome
Thats reminds me of the firefly algorythm
Made no sence. Better try comming up with more and more examples rather than just reading slides.
First 3 mins is him sucking up the subscribers
Siraj for the president
Thanks NS Hippie!
And then ppl like you and me...
come to china
Ridiculous
Bhai hadd ho gyi...kitna style marta hai ye...aur upar se itni bakwaass krta hai...
NPTEL dekhiye aap.
R u a Gay... Bcz u r reacting like that... Plzz... Stop overacting and try to speak normally.. Like a civilian not like karan johar
hahah
warning: when he finger styles his hair, get ready for hardcore info dump.
PS: 3blue1brown series on linear algebra has THE BEST vid on eigen vectors/value pairs, no joking.
In case you have bad results using Gaussian mixtures, keep in mind the EM optimization only has local convergence properties, just like gradient descent: it can get stuck. Restarting the the density estimation with other initial parameters might solve it ! :)
thanks Jason!
Siraj, I think it would of been helpful if you showed the resulting clusters that you get from the gaussian mixture model approach in your data. You showed how to model your data using the gaussian mixture, but I am unclear on how we get the specific clusters (say 2 clusters) from that?
Hi all! I've fixed some bugs from the original .ipynb so you are welcome to try my one: github.com/ivanpastukhov/Gaussian_Mixture_Models/blob/master/intro_to_gmm_%26_em.ipynb
Siraj, thanks a lot for your brilliant videos!
Thank you so much Ivan!
Where do I get the dataset? It is not mentioned anywhere and is not in Github repository either
Dataset can be found at: raw.githubusercontent.com/brianspiering/gaussian_mixture_models/master/bimodal_example.csv
Checkout Siraj's India visit info >>pydata.org/delhi2017/
very excited
Can you please control your moving hands data points? too much distraction.
Great video, I tried running your code on my terminal and it's giving the error that 'GaussianMixture' object has no attribute 'loglike', would you happen to know why an error like would occur, or anyone by that matter. Thank you so much
I watch 4-5 vídeos of you per day. I'm Learning generative models for drug Design Siraj. Watch your videos not only motivates me, also makes my life & study fun and cool.
So... 38 minutes to predict something and he just forgets about the prediction part?
I'm sorry, but the justification on 36:02 is not enough for my satisfaction.
1:30 to 3:40 - talk trash
Thank you! Your videos helped me a lot... I was so lost and confused about this topic that I was on the verge of giving up. Checked out your tutorials that gave a lot of useful information and insights. Thanks a tonne! :) :D Keep up the good stuff
Great series!!!! even helps me in my AI learning curve at Udacity. Thanks for it. rgds tibor
I keep getting the following error though:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
in
8 #train!
9 mix.iterate(verbose=True)
---> 10 if mix.loglike > best_loglike:
11 best_loglike = mix.loglike
12 best_mix = mix
AttributeError: 'GaussianMixture' object has no attribute 'loglike'
25:22 EM model
Hi Siraj, I appreciate your videos and I love your content. I' am working on a project on cross-matching using active learning, what advice would you have for me? I' am trying to build something scalable but not so computationally intense.
Hi Siraj, wonderful video! I am wandering what is the difference between Gaussian mixture model and least square method in the data fitting' view?
hey siraj ! EM is a heuristic with no guarantees for global convergence. there have been recent algorithms based on method of moments, random projections etc. which provably recover the gmm under some assumptions
Thank you very much! Your explication is very good and educative! I'm recommending your channel to my friends too.
Hi, your videos are great!. Please cover VGG, Alexnet, and others sometime.
thanks Aamir!
Wow! Finally I got my head around this subject. Well done and amazing teaching skills 👏🏻
Andre
6:45 "y values are the probabilities for the x values." NO!! y values are decidedly NOT probabilities.
7:39 "Sometimes the data has multiple distributions". WTF?! This video is deeply misleading. Please take it down.
33:30
wp1/(wp1+wp2) + wp2/(wp1+wp2) = 1
log(wp1 + wp2) = log(1) = 0
How is his model being trained?
You guess a theta ( model params) , then that gives you a probability distribution of the hidden variables. With that known, you maximize the joint probability distribution of X and the hidden variables. That gives you a new theta. Repeat the 2 steps above: use the new theta model params instead of your guess.
We actually try to get the value of log(wp1 + wp2) =1 not (wp1 + wp2) to be 1.
Here, x1, x2... are the vecors or are the data points of a vector x?
Apple sends their hinge prototypes to this guy for testing. If this guy won't wear out hinges, who will?
Really thanks man, your video helped me a lot in my Hyperspectral Images classification project's
when I do your codes couldnot find data file error? Why? how can find it?
8:30 "x is the number of data points"? What are you talking about?!
Great Video! Really helpful for Data scence students..
The hand gestures as always are a little bit distracting to me by the way..
Hey Siraj!
Just found your channel and it doesn't cease to amaze. I am learning a lot about AI and ML with your vibrant and enthusiastic expression. My 2 cents would be to talk a tiny bit slower but it is up to you. Congrats and Keep up the Good Work!
thanks Kashyap!
where is the mathematical explanation?
Where I can get the blog he is following?
Handsome. I like you teaching
Does anyone know what the mix value is ?
Thanks for the vid. Owe ya one big broth
OKOK I will subscribe you ; >
We love you Siraj
Video starts at 03:40
Super tutorial! Thank you so much!
Thanks for reading theory to me. Couldn't do that by myself
I know you're being sarcastic, but honestly, I'm looking for people to do just that for me, I HATE reading technical material.
good job!
Siraj. The depth and range of your knowledge still continues to amaze me.
thanks Antony!
Okay