There's a timeless beauty to the cadence , passion, and clarity of your lecture series. Glad I could find it again. Different goals now, but same appreciation.
Thanks à lot for your videos. I am à french Student in mathematics and your way of explaining things is so different from here but it all makes more sense. Look forward to the next videos.
The formula I haven see uses the complex conjugate transpose instead of the plain transpose. For real values the two are equivalent but I would make a note of this.
Oh I think I get it now. Since xTATb = is a 1x1 matrix, and since the transpose of a 1x1 must be the same, then you can have the big transpose or ignore it, it doesn't matter in this case!
@@matthewjames7513 AtA is invertible since it is positive definite, therefore, all of its eigenvalues are greater than zero(and real since its symmetric). Note that AtA is symmetric, then consider the magnitude square of Ax, expand it as a dot product and take x to be an eigenvector of A.
great work very good lecture , but i wished there was an order or number of the lecture so i can visit the previous ones if i dont understand an certain concept
Yeah, me too at first. On the left of the board there's something previously covered about "Quadratic Form Minimization", and that operation parallels differentiation in calculus. Note that to find the x that minimizes rᵀr, you take the derivative WRT x and solve for the x that makes that derivative equal to zero. It looks like he compares the xᵀAx to ax² and illustrates how they become 2Ax and 2ax respectively. So he goes from rᵀr = 2(½xᵀAᵀAx - xᵀAᵀb) + bᵀb to 0=2(AᵀAx - Aᵀb) to AᵀAx = Aᵀb to x = (AᵀA)⁻¹ Aᵀb
@@MichaelStangeland Thanks, this was the missing step in the video. Would you agree to say that we differentiate w.r.t xᵀ? Maybe it truly doesn't matter, one can always rearrange. And we use the fact that AᵀA is symmetric when we work with differentiation this way.
Slow until the audio cut out, after which the interesting part takes place. This could have been really good, but the audio problem was not handled well. "What a beautiful equation" for 20 seconds the end immediately after the solution was some frustrating editing. Keep it up, but that's some constructive criticism
what if ATA is not positive definite (that leads to the rTr does not have minimum value because we can choose x to make rTr as small as possible ) and if ATA is not invertible ? . Please help me .
So ATA is always positive semi-definite . if A is invertibe so that ATA is positive definite and we can find a global minimum of rTr . If A is singular , ATA will not invertible and the equation resulting from derivative ATAx = ATb can have multiple x to satisfy . As a result rTr will have multiple local optima . That 's what my thoughts . Am I correct ? . Thank you very much for quick response .
You are correct. More specifically, it will be one minimum value, but it will attained at a whole subspaces of locations. Sort of like the function f(x,y)=x^2.
I understand your example of f(x,y) = x^2 . The minimum subspaces in this example is a line x = 0(y whatever ) . If we randomly choose location of x and apply gradient descent , do we always certainly get to the minimum subspaces ? Does all optimization problem that can be expressed in quadratic form always have just one minimum value ? (I think yes but I still want a verification ) For other cases that are not a quadratic form , the function will have multiple local optima and the value of function of those local optima are different . If I use gradient descent with randomly choosing the starting point of x , it will lead to different local optima ( in case of luck , we get to the global optima ). So the position of starting x is important . Is there any way that I surely can come to global optima instead of getting stuck of local optima . My idea is choosing multiple starting position x and the get the x that have f(x) minimum . But this way also depends on luck and not solve the situation completely . Can you come up with a good solution for this problem ?
![Lp. Сердце Вселенной #60 РОЖДЕНИЕ ЛОЛОЛОШКИ [Финал] • Майнкрафт](http://i.ytimg.com/vi/YoR0pAV9FVQ/mqdefault.jpg)
Go to LEM.MA/LA for videos, exercises, and to ask us questions directly.
It would be great to have the second part of this with an audio recording of the explanation. Everything was crystal clear until the mic died
There's a timeless beauty to the cadence , passion, and clarity of your lecture series. Glad I could find it again. Different goals now, but same appreciation.
Sir, I love the intuition you inject into these topics, something most other teachers (and UA-camrs for that matter) fail to provide.
An energetic math teacher. I've never met one in the wild. Thanks!
And damn handsome, too!
Thanks à lot for your videos. I am à french Student in mathematics and your way of explaining things is so different from here but it all makes more sense. Look forward to the next videos.
the most important part is missing
What is that part?
@@vangrails battery😂
Even with the mic batteries dead, this was a great lecture.
..................................................
I wish there were more professors like you sir. You are truly genius. Thank you so much.
this video is marvelous from start to finish, and the mute subtitles did really help
Glad to hear that!
Tank you very much, You solved my least squares problem.
It's the least I can do
Please record the next video with sound.
The formula I haven see uses the complex conjugate transpose instead of the plain transpose. For real values the two are equivalent but I would make a note of this.
Why are they equal because they are transposes of each other?
You should have done the second half of the video in sepia. A moustache would have been a nice touch too.
What about the error?
What an epic way do deal with a microphone problem XXDDXXDXD (also good explaining, thanks!)
yeah i loved the idea!
4 years later he is still busy hahahahaha. Good class except fot the last 2:30 where magic happens
at 8:41 can somone help me undersatnd why xTATb = bTAx ? I can understand why xTATb = (bTAx)T, but where did the big transpose go?
Oh I think I get it now. Since xTATb = is a 1x1 matrix, and since the transpose of a 1x1 must be the same, then you can have the big transpose or ignore it, it doesn't matter in this case!
But why at 9:43 is ATA guaranteed to be invertible ? :O
@@matthewjames7513 AtA is invertible since it is positive definite, therefore, all of its eigenvalues are greater than zero(and real since its symmetric). Note that AtA is symmetric, then consider the magnitude square of Ax, expand it as a dot product and take x to be an eigenvector of A.
Yes please re-record the audio, it was a brilliant explanation up until then
great work very good lecture , but i wished there was an order or number of the lecture so i can visit the previous ones if i dont understand an certain concept
where is the re-recorded version
Sorry, never happened!
Sir
As we express a length as xTx
How can express a volume in matrix entry
Determinant
Interesting way of explaining the subject.Could I get a pdf file of the story?Jozef
as geodesy and geomatics engineering student i always using the least square adjustment to solve problems :D thank your sir for the lecture he he he
wonderful!
Can someone point to me where are the rest of the videos of this course available ?
Great video, but I cant understand the end without knowing what the lecturer is talking...
I was lost from the 'A', 'b' part... like we discovered a similar structure but how did we directly got the value of x...?
Yeah, me too at first. On the left of the board there's something previously covered about "Quadratic Form Minimization", and that operation parallels differentiation in calculus. Note that to find the x that minimizes rᵀr, you take the derivative WRT x and solve for the x that makes that derivative equal to zero. It looks like he compares the xᵀAx to ax² and illustrates how they become 2Ax and 2ax respectively.
So he goes from rᵀr = 2(½xᵀAᵀAx - xᵀAᵀb) + bᵀb
to 0=2(AᵀAx - Aᵀb)
to AᵀAx = Aᵀb
to x = (AᵀA)⁻¹ Aᵀb
ua-cam.com/video/oaiiyIsbNdI/v-deo.html
@@MichaelStangeland Thanks, this is helpful
@@MichaelStangeland Thanks, this was the missing step in the video. Would you agree to say that we differentiate w.r.t xᵀ? Maybe it truly doesn't matter, one can always rearrange.
And we use the fact that AᵀA is symmetric when we work with differentiation this way.
Slow until the audio cut out, after which the interesting part takes place. This could have been really good, but the audio problem was not handled well. "What a beautiful equation" for 20 seconds the end immediately after the solution was some frustrating editing.
Keep it up, but that's some constructive criticism
This feels like a Charlie Chaplin movie after mic died.
Thanks
Thank you very much, yur videos are great, they give me a pretty good insight of the subject, please upload this video again :D
didn't get the final step o f the proof. anyone found the video?
modern problems require modern solutions
Ah Noice! A Math lesson in Bioshock style xD
ty sire
what if ATA is not positive definite (that leads to the rTr does not have minimum value because we can choose x to make rTr as small as possible ) and if ATA is not invertible ? . Please help me .
Please see this video: ua-cam.com/video/bp38BKP-xh4/v-deo.html
So ATA is always positive semi-definite . if A is invertibe so that ATA is positive definite and we can find a global minimum of rTr . If A is singular , ATA will not invertible and the equation resulting from derivative ATAx = ATb can have multiple x to satisfy . As a result rTr will have multiple local optima . That 's what my thoughts . Am I correct ? .
Thank you very much for quick response .
You are correct. More specifically, it will be one minimum value, but it will attained at a whole subspaces of locations. Sort of like the function f(x,y)=x^2.
I understand your example of f(x,y) = x^2 . The minimum subspaces in this example is a line x = 0(y whatever ) . If we randomly choose location of x and apply gradient descent , do we always certainly get to the minimum subspaces ? Does all optimization problem that can be expressed in quadratic form always have just one minimum value ? (I think yes but I still want a verification )
For other cases that are not a quadratic form , the function will have multiple local optima and the value of function of those local optima are different . If I use gradient descent with randomly choosing the starting point of x , it will lead to different local optima ( in case of luck , we get to the global optima ). So the position of starting x is important . Is there any way that I surely can come to global optima instead of getting stuck of local optima . My idea is choosing multiple starting position x and the get the x that have f(x) minimum . But this way also depends on luck and not solve the situation completely . Can you come up with a good solution for this problem ?
I think the word you were looking for was "stamp"
That's right!
very nice!
What is everyone here studying? I just came here because of an equation I saw for Linear Regression XD
@Max Kirkman linear algebra
I’m actually here for NLP
Computer Vision
@@flightrisk7566 NLP x2
regression
Great !
very funny! Tks a lot for such a great lecture
feels like the battery understood what he said just earlier, hearing wont help, read out it yourself to better understand , LOL
yes the battery is in charge
NOOOO !!! This is horrible!!! This was shaping up to answer all my nagging questions about least squares :( :(
Give us one of those nagging questions.
MathTheBeautiful in the last step did you take the first derivative and set it to zero to find the minimum of r^Tr?
Please rerecord or talk over!!!! Please!!!
Damn the mic dying!!!!
Gave you a dislike because of the sound towards the end.
but it is not his mistake. u should be respectful. if u dont like go other channels. this man explains in best way.
like it
stump it out
Math is super fun, but when we listen to about one xillion detail at every corner? It becomes so boring.
I think it's the one zillion first detail that one too many!
i was enjoying it until the CHALK SCREECHED AGAINST THE CHALKBOARD UNGHGHHGH IM SO UNCOMFORTABLE NOW
Geez my guy, for every 5 seconds of math, there's like 2 minutes of you talking about "stamping it out". It's painful to watch, sorry.
Is this a joke?
The guy talks too much.
Agreed. His wife must be sick of him .