I seriously hope he teaches a lot more machine learning and those lectures get published here. He is the only teacher I found who actually dives into the math behind machine learning.
He is certainly at a level that makes you understand. Best teacher. No show off , no hand waving. Genuine teacher. Goal is to teach . Thanks you thank you
This professor is amazing. I'm italian so it's more difficult to follow a lesson in english than in my language. Well, it was much much easier to understand PCA here for me than in any other PCA lesson or paper in italian. And not only, but he gave a more rigorous explanation too! Outstanding, really...
I agree. Watch 21:00. The student repating the answer is the eigen value and the eignen vector, and the intructor says Ok this is correct but why!? In most videos on youtube I have seen, people who pretend to be expert do not know (or do not say) the logic behind their claims.
This is what i've been looking for! Every explanation out there just contains more question. "Get the covariance" What for ? "Do decomposition" Why ? "Use eigen vectors" Huh ??. Thank you for explaining every question I have!
Wow. This IS what I'm looking for!! Thank you SO much! BTW the explanation for 20:21 is simple if you already have some experience of manipulating with linear algebra. Just decompose the matrix S into EΛ(E^-1) and the sum will turn into sum of ratios of eigenvalues, with the ratios sum up to 1. (assume that the data are already standardized, which is crucial.) Thus you have to put the ratio of corresponding eigenvector to 1 to get the max sum, which is the maximum eigenvalue.
Thank you professor. This lecture explains exactly what I was looking for - why principal components are the eigenvectors of the sample covariance matrix.
thank you so much! I tried to see the same topic in other videos and was impossible to understand, this is so clear, ordered and intuitively explain, awesome lecturer!
Dear Prof, at 28:46 you say that tangent of f and tangent of g are parallel to each other - possibly you meant to say that gradient ie normal of f and normal of g are parallel to each other. Anyways it effectively means the same thing. Excellent video!
Good lecture. PCA tries to find the direction in the space, namely a vector, that maximises the variance of the projected points or observations on that vector. Once the above method finds the 1st principal component, the second component is the vector orthogonal to the first component.
At time 1:03:26, [U, D, V] = svd (X). Question: shall we do svd(X - E(X)), since X contain pixel values in [0, 255] and the data points X is not centered to E(X)?
Sir I have one question @1:01:50. If I had only one face image with each pixel distribution independent of other but mean corresponds to original face value at that pixel. I think first, second and so on PCs are noise dominant and we are still able to see the face.?
Great lecture! Tip to the camera person: there's no need to zoom in on the powerpoint. The slides were perfectly readable even when they were at 50% of the video area but it is much better to see the lecturer and the slide at the same time. Personally, it makes me feel more engaged with the lecture than just seeing a full-screen slide and hearing the lecturer's voice.
Is it important to show 95% confidence ellipse in PCA? why it is so? If my data is not drawing it then what should i do ? can i used PCA score graph without 95% confidence ellipse?
1:14 Be careful that this example is not that proper. Note that PCA is basically a system for axis rotation and hence it usually does not have good applications for those data with "donut" (or, swiss roll) structure. A better way is either to use kernel PCA or MVU (maximum variance unfold).
I think this is not about PCA but the fact that distributions in higher dimensions can be projected to lower dimensions such that there is one to one correspondence between higher dimensional and lower dimension counterparts as much as possible.
S is just a notation. S is covariance matrix of original matrix X , u1 is constant(We can say) . In variance constant become square but in the case of vectors form you write u1 u1_transpose. final expresssion is u1 X u1_transpose
t=transpose ^2= square This function is quadratic because of u and ut: Quadratic function for one variable has the following form : ax^2 + bx + c Quadratic function for two variables has the following form ax^2 + bxy + cy^2 + dx + ey + g Let's consider an example: 1- Suppose vector u=[x1] [x2] then ut = [x1 x2] matrix S=[1/2 -1] [-1/2 1] 2- ut S gives us the following vector: ut S = [ 1/2*x1-1/2*x2] [ -x1 + x2 ] 3- ut S u gives the following function which will be a scalar if the vector u is known: ut S u = 1/2 * x1^2 + x2^2 -3/2 * x1 * x2 ut S u is quadratic
t= transpose ^2= squared In ordrer to demonstrate that Var(ut X) = ut S u I will use - the könig form of the variance Var(X)= E(X^2) - E^2(X) - and this covariance matrix form COV(X)= E(X Xt) - E(X) [E(X)]t So let's start: We will use the köning form to define the variance: 1- Var(ut X) = E((ut X)^2) - E^2(ut X) * We know that (ut X)^2=(ut X) [(ut X)]t so the first quantity becomes: E((ut X)^2) = E( (ut X) [(ut X)]t ) The second quantity becomes: E^2(ut X)=E(ut X) [E(ut X)]t And we get: 2- Var(ut X)= E( (ut X) [(ut X)]t ) - E(ut X) [E(ut X)]t * We know that [ut]t = u and [(ut X)]t= (Xt u) (notice that the transpose has changed the multiplication order) so the first quantity will change like this: E( (ut X) [(ut X)]t ) = E( (ut X) (Xt u) ) And we get: 3-Var(ut X) = E( (ut X) (Xt u) ) - E(ut X) [E(ut X)]t * We know that Expectancy of a vector(or matrix) filled with scalars gives the same vector(or matrix) and Expectancy of a vector(or matrix) filled with random variables gives Expectancy of that vector(or matrix) In others words: E(u)=u, E(ut)= ut , E(X Xt)=E(X Xt) and E(X)=E(X) So the first quantity becomes E( (ut X Xt u) ) = E(ut) E(X Xt) E(u) = ut E(X Xt) u and the second quantity becomes E(ut X) [E(ut X)]t = E(ut) E(X) [E(ut) E(X)]t = ut E(X) [ut E(X)]t = ut E(X) [E(X)]t u And we get: 4-Var(ut X) = ut E(X Xt) u - ut E(X) [E(X)]t u * let's factorize by ut And we get: 5-Var(ut X) = ut [ E(X Xt) u - E(X) [E(X)]t u ] * let's factorize by u And we get: 6 -Var(ut X) = ut [ E(X Xt) - E(X) [E(X)]t ] u * We know that COV(X)= E(X Xt) - E(X) [E(X)]t And we get: 7-Var(ut X) = ut COV(X) u * Here S = COV(X) And finally, we have: 8-Var(ut X) = ut S u
WITH THE HELP OF GOD WE ADVANCE IN A STRAIGHT LINE THINKING SPEAKING BEHAVIOR ACTIONS LIFE TO THE HIGHEST STATE OF PERFECTION GOODNESS RIGHTEOUSNESS GOD'S HOLINESS EXACTLY AS WRITTEN IN THOSE 10 LAWS
I seriously hope he teaches a lot more machine learning and those lectures get published here. He is the only teacher I found who actually dives into the math behind machine learning.
Agreed
Tou directly Statistics k professors k lecture bhi le sakty ho.
This guy owns Databricks now, the biggest ai start up in the world. He isn’t coming back anytime soon sadly 😂
@@ElKora1998 Not the same person
@@andrewmills7292 you sure? He looks identical!
He is certainly at a level that makes you understand. Best teacher. No show off , no hand waving. Genuine teacher. Goal is to teach . Thanks you thank you
This professor is amazing. I'm italian so it's more difficult to follow a lesson in english than in my language. Well, it was much much easier to understand PCA here for me than in any other PCA lesson or paper in italian. And not only, but he gave a more rigorous explanation too! Outstanding, really...
No words to describe the greatness of this professor!
If you were to take my opinion, his videos are best one on ml and deep learning on youtube
I agree. Watch 21:00. The student repating the answer is the eigen value and the eignen vector, and the intructor says Ok this is correct but why!?
In most videos on youtube I have seen, people who pretend to be expert do not know (or do not say) the logic behind their claims.
@@VahidOnTheMove exactly
I agree!
Thanks, this is by far the most detailed explanations of PCA
This is what i've been looking for! Every explanation out there just contains more question. "Get the covariance" What for ? "Do decomposition" Why ? "Use eigen vectors" Huh ??. Thank you for explaining every question I have!
I'd won a Fields medal if I had a professor like this guy in my undergrad.
i think you can do that right now... age is just a number when it comes to learning and creating :D
Crazy || ME :) only if you are not older than 40...fields medal is given for
@@pubudukumarage3545 oh thanks for the information.... I didn't knew that.... Guess I really am kinda crazy huh?? XD
Yeah, a Garfield the cat medal I'm sure you can win. Just give him Lasagne.
Clearly you would not have won a medal for your English ability.
Wow. This IS what I'm looking for!! Thank you SO much!
BTW the explanation for 20:21 is simple if you already have some experience of manipulating with linear algebra.
Just decompose the matrix S into EΛ(E^-1) and the sum will turn into sum of ratios of eigenvalues, with the ratios sum up to 1. (assume that the data are already standardized, which is crucial.)
Thus you have to put the ratio of corresponding eigenvector to 1 to get the max sum, which is the maximum eigenvalue.
This is the best video about this subject, including the math behind it, that I've found so far.
Thank you professor. This lecture explains exactly what I was looking for - why principal components are the eigenvectors of the sample covariance matrix.
Iranian Professors are fantastic! and Prof. Ali Ghodsi is one of them
Teachers like him make more people fall in love with the topic.
3:30 We can always reduce the dimensionality of the features by projecting them onto an optimal subspace.
thank you so much! I tried to see the same topic in other videos and was impossible to understand, this is so clear, ordered and intuitively explain, awesome lecturer!
43:03 The entries in Σ are not eigenvalues of A transpose A, but square roots of eigenvalues of A transpose A.
Dear Prof, at 28:46 you say that tangent of f and tangent of g are parallel to each other - possibly you meant to say that gradient ie normal of f and normal of g are parallel to each other. Anyways it effectively means the same thing. Excellent video!
Good lecture. PCA tries to find the direction in the space, namely a vector, that maximises the variance of the projected points or observations on that vector. Once the above method finds the 1st principal component, the second component is the vector orthogonal to the first component.
By far, the best video on PCA
This lecture is amazing, your student are extremely lucky ...
At time 1:03:26, [U, D, V] = svd (X). Question: shall we do svd(X - E(X)), since X contain pixel values in [0, 255] and the data points X is not centered to E(X)?
This is the best pca explanation I've ever seen!! 👍👍
know we see teacher with clear and open mind
Sir I have one question @1:01:50. If I had only one face image with each pixel distribution independent of other but mean corresponds to original face value at that pixel. I think first, second and so on PCs are noise dominant and we are still able to see the face.?
Around 44:50 , u explained M as set of mean values of x_i data points, shouldn’t mean (xi) = 1/d rather than 1/n *sum of xi over i = 1 to d
Amazing lecture. I really enjoyed every single second ....
That was an amazing lecture Sir! Thank you!
Great lecture! Tip to the camera person: there's no need to zoom in on the powerpoint. The slides were perfectly readable even when they were at 50% of the video area but it is much better to see the lecturer and the slide at the same time. Personally, it makes me feel more engaged with the lecture than just seeing a full-screen slide and hearing the lecturer's voice.
At 17:25 sigma should be sigma squared to call it variance if not we say standard deviation.
Wonderful lecture thats both intutive as well as mathematically excellent.
This is a great tutorial video, I could grasp the idea behind PCA with easy and clear thoughts.
Best lecture on PCA!
not a fan of learning through youtube videos , but this was an excellent lecture
43:20, aren't they the square roots of eigenvalues of XTX or XXT?
This series of videos are so great!
Man, this is pure gold!!
everything is explained crystal clear. thanks!
There is more mathematics in this video than a data science curriculum.
Thank you for this lecture. Where can we find the dataset used for the noisy faces?
He's an astute mathematician with virtuoso teaching skills.
Great lecture Dr. Ali...Thanks a lot
thanks for this explanition.please how can i contact with you?i have inquiry
Amazing lecture , Fabulous
best video about pca math thanks
it's amazing and it really made me understand clearly!!!!!!!!!
Great video! Really nice explanation
Thank you prof Ghodsi, very helpful
so using SVD is it correct to say that columns of U are similar to PC loadings (eigenvalue scaled eigenvectors) and V is the scores matrix?
Is it important to show 95% confidence ellipse in PCA? why it is so? If my data is not drawing it then what should i do ? can i used PCA score graph without 95% confidence ellipse?
concise and clear explanation
I got answers to almost every"WHY?" that I had while reading books.
Amazing lecture
at 18:03 I think "a square times sigma square" not sigma?
Does anyone know the proof for the second pc ?
دم شما گرم. عالی بود.
1:14 Be careful that this example is not that proper. Note that PCA is basically a system for axis rotation and hence it usually does not have good applications for those data with "donut" (or, swiss roll) structure. A better way is either to use kernel PCA or MVU (maximum variance unfold).
I think this is not about PCA but the fact that distributions in higher dimensions can be projected to lower dimensions such that there is one to one correspondence between higher dimensional and lower dimension counterparts as much as possible.
He already mentions that, the assumption is that the data is aligned close to a plane like a paper
Why has lecture 3 been deleted? How do we watch it?
Can I find the slides online ?
We need code and application areas of PCA?
wow....great lecture
This is Gold.
mind blown in the first two minutes
Dear Prof, Thanks for the lecture. Is it possible to share the lecture materials? Thank you!
Thats great , can we have access to that noisy dataset ?
Lecture is great but that struggle to find image size though.
Oh what an explanation!!
Amazing lecture!
Amazing lecture..
Thank you it’s fist time I understand PC but I am studying master financial mathematics sorry I just want to do every step manual is that possible
Good lecture. Thank you.
he is great
the best ever, thanks!
at 19:32 why is var (u_1 transpose x)=u__1 transpose s u)
S is just a notation.
S is covariance matrix of original matrix X ,
u1 is constant(We can say) . In variance constant become square but in the case of vectors form you write u1 u1_transpose.
final expresssion is u1 X u1_transpose
Great lecture, thank you sir
Around 22:00 can someone explain why the function is quadratic?
t=transpose
^2= square
This function is quadratic because of u and ut:
Quadratic function for one variable has the following form : ax^2 + bx + c
Quadratic function for two variables has the following form ax^2 + bxy + cy^2 + dx + ey + g
Let's consider an example:
1- Suppose vector u=[x1]
[x2]
then ut = [x1 x2]
matrix S=[1/2 -1]
[-1/2 1]
2- ut S gives us the following vector:
ut S = [ 1/2*x1-1/2*x2]
[ -x1 + x2 ]
3- ut S u gives the following function which will be a scalar if the vector u is known:
ut S u = 1/2 * x1^2 + x2^2 -3/2 * x1 * x2
ut S u is quadratic
Definition?
ALL Thanks Dr.Ali
how??? How is the variance of the projected data = u^(T)SU
Got it
I am not clear on this .can you explain
@@venkatk1591 you can see the explanation here - ua-cam.com/video/WpYoKsWKS7w/v-deo.html
best video oh PCA
Jump @17:20 then @29:30 @42:10
English subtitles please!
great lecture thanks!!!
excellent lecture
AMAZING!
Awesome!
بالتوفيق ان شاء الله
I'm still lost :(
Ye lecture kis bachhe ke liye h
This gave me a moment of epiphany.
Excellent.
why is var (u_1 transpose x)=u__1 transpose s x)
t= transpose
^2= squared
In ordrer to demonstrate that Var(ut X) = ut S u I will use
- the könig form of the variance Var(X)= E(X^2) - E^2(X)
- and this covariance matrix form COV(X)= E(X Xt) - E(X) [E(X)]t
So let's start:
We will use the köning form to define the variance:
1- Var(ut X) = E((ut X)^2) - E^2(ut X)
* We know that (ut X)^2=(ut X) [(ut X)]t so the first quantity becomes: E((ut X)^2) = E( (ut X) [(ut X)]t )
The second quantity becomes: E^2(ut X)=E(ut X) [E(ut X)]t
And we get:
2- Var(ut X)= E( (ut X) [(ut X)]t ) - E(ut X) [E(ut X)]t
* We know that [ut]t = u and [(ut X)]t= (Xt u) (notice that the transpose has changed the multiplication order)
so the first quantity will change like this: E( (ut X) [(ut X)]t ) = E( (ut X) (Xt u) )
And we get:
3-Var(ut X) = E( (ut X) (Xt u) ) - E(ut X) [E(ut X)]t
* We know that Expectancy of a vector(or matrix) filled with scalars gives the same vector(or matrix)
and Expectancy of a vector(or matrix) filled with random variables gives Expectancy of that vector(or matrix)
In others words: E(u)=u, E(ut)= ut , E(X Xt)=E(X Xt) and E(X)=E(X)
So the first quantity becomes E( (ut X Xt u) ) = E(ut) E(X Xt) E(u)
= ut E(X Xt) u
and the second quantity becomes E(ut X) [E(ut X)]t = E(ut) E(X) [E(ut) E(X)]t
= ut E(X) [ut E(X)]t
= ut E(X) [E(X)]t u
And we get:
4-Var(ut X) = ut E(X Xt) u - ut E(X) [E(X)]t u
* let's factorize by ut
And we get:
5-Var(ut X) = ut [ E(X Xt) u - E(X) [E(X)]t u ]
* let's factorize by u
And we get:
6 -Var(ut X) = ut [ E(X Xt) - E(X) [E(X)]t ] u
* We know that COV(X)= E(X Xt) - E(X) [E(X)]t
And we get:
7-Var(ut X) = ut COV(X) u
* Here S = COV(X)
And finally, we have:
8-Var(ut X) = ut S u
c'est un grand
WITH THE HELP OF GOD WE ADVANCE IN A STRAIGHT LINE THINKING SPEAKING BEHAVIOR ACTIONS LIFE TO THE HIGHEST STATE OF PERFECTION GOODNESS RIGHTEOUSNESS GOD'S HOLINESS EXACTLY AS WRITTEN IN THOSE 10 LAWS
tanks.
👏👏👏👏👏👏👏👏👏👏👏👏👏
perfect :)
ADAMSIN
太神奇了！
2
viva Iran
worst lecture