Least Squares Approximation
Вставка
- Опубліковано 7 лют 2025
- MIT 18.06SC Linear Algebra, Fall 2011
View the complete course: ocw.mit.edu/18...
Instructor: Ben Harris
A teaching assistant works through a problem on least squares approximation.
License: Creative Commons BY-NC-SA
More information at ocw.mit.edu/terms
More courses at ocw.mit.edu
"i'm gonna rush through this because you should know how to multiply by now"
*does it wrong*
LOL
rofl
XD
Yep
cut him some slack. it make me feel good sometimes that MIT students and professors makes silly mistakes which is normal by the way!!. the most important part is the learning process
Thanks for this problem solving and the methodology presented, but there is an error in the solution during multiplication of A_transpose with A. This error accumulates over time in the solution and results in the wrong answer. The correct answer should be y= 5/22 x^2 + 41/22 x
goat comment
Computation is something that everybody is supposed to do on their own. What one really cares about are the ideas implemented to solve a problem. The remaining things seems to be of second grade.
Thanks for leaving the comment! I had a little different A and b (points in another order) and it's good to know i got to the right answer!
Hello everyone! The answer to A^T A is 18 at the 2,2 entry
yes
It's still a nice example, I've never thought concepts of linear algebra could come together like this
Where is the 10 coming from? Shouldn't it be 18?
Yes it should. The correct solution is then y = (41/22)*c + (5/22)*d^2. You can also see this if you plot both equations against the given points.
@@Rhinozekon y=(81/22)*c + (-25/22)*d^2
I was not really doubting myself but I did not like it. This guy should know, right?! Anyway, answers I got were : C = 41/22 & D = 5/22.
Thank you all!!!!!
Yep
This is a nice example. But it's a pity that there seems to an calculation error In the derivation of the projection matrix There should be an 18 instead of 10 in the matrix. And C and D should be 41/22 and 5/22, leading to (e1, e2, e3)=(-12/11, 4/11, -4/11) if my calculation is correct. If, as calculated in the video, C and D were indeed 11/2 and -5/2, then you would have (e1, e2, e3)=(-2, 4, 6). That didn't look like a "least mean square error".
Correct! Right answer should be b = 5/22 x^2 + 41/22
y = 2.3*x - 0.1*x^2 fits much more correctly.
True, is it the best ?
A'A would result in (6 8; 8 18).
Correct me if I'm wrong, but isn't an equation of the form ax^2 + bx + c = 0, isn't y = ax^2 + bx + c just a quadratic function. I know it should be obvious, but it threw me off a bit, I wrote the equation and realized quickly that only two values of x were even possible :D
This quadratic function goes through the origin, so c must be zero (there is no shift).
@@ИльяЛомоносов-ю3м Why? Linear has a non-zero intercept, why not quadratic? the three points fits perfectly(0 error)
@@faizanmansoori The definition of the problem says "find a quadratic equation *through the origin*"
@@maxim_ml with no intercept it still goes through the origin at t =0, the wording should be clearer
I got the same answer, he made an error accidentally
There is a computation error in one of the steps. Final answer should be: (82/44, 10/44).
To verify:
import numpy as np
from numpy.linalg import lstsq
# Given points
points = np.array([(1, 1), (2, 5), (-1, -2)])
# Separate x and y values
x = points[:, 0]
y = points[:, 1]
# Set up the design matrix for a quadratic fit y = ax^2 + bx
X = np.vstack([x, x**2]).T
# Perform least squares fitting
coefficients, _, _, _ = lstsq(X, y, rcond=None)
print(coefficients)
print(np.round([82/44, 10/44], 8))
Output:
[1.86363636 0.22727273]
[1.86363636 0.22727273]
What 3 points is he talking about at the end?
I am still not able to get why we took square of 1st column as the 2nd column in A
because the equation was cx+dx^2 so for matrix a second column will be all values of first column squared to satisfy c and d.
Problem13 . If e components of e=b−Ax averages to zero, then so does ( A^T * A)^(−1) * A^T * e. Why?
why not make it a quadratic in the form of ax^2 + bx + c?
instead of coefficients of only the 2nd and 1st power
because the curve is through the origin thus C the y-intercept is zero
UA-cam comments to the rescue. I thought I was going insane.
Will be always take square of first elements of A?
shouldn't have rushed A transpose A
Isn't it supposed to be x and not x-hat at 3:35? My reasoning is cause x-hat is the best answer we can get and x is the perfect answer.
@CoeusQuantitative why so serious
@ 5:17, it should be 18, not 10.
Ehren Mann
yes, 10 should be 18, and the correct formula is y=(81/22)*c + (-25/22)*d^2
I think it's more like
[6 8; 8 18] [c; d] = [13; 19]
then c = 41/22
d = 5/22
y = (41/22)*t + (5/22)*t^2
@@chotirawee how did you get that 41 please explain
@@RaselAhmed-ix5ee it's taken as
Eq1 and eq 2 for c and d variables.
Where it gets the answer by using substitution.
@@shironoyami7002 thanks for the quick reply, I am father now not a student anymore
@@RaselAhmed-ix5ee bro I just saw it by chance now.
I searched it up because tomorrow is my exam.I tried to search answer too because I didn't understand like how you were back then.
And I leave an answer simply for someone else to read it instead of the purpose of replying to your question....
Congrats man.👏
Why did he choose the quadratic function?
how to choose what kind of equation that is?
quadratic means y=ax²+bx+c
through the origin means y(0)=0 so c=0
MIT lectures >>>>>> anyone else tryna teach
He reminds me a person that i used to bully.
And yet, he now has a Ph.D. from MIT and is now a professor of at Cornell, and you're... not.
@@nicksaba3882 i'm happy XD
Haha Eduardo trying to be hard on youtube while this lecturer is now a Prof at a prestigious university. Giad you're happy though Eddy x
@@jonpol770 if he has now phd in his field that just makes more to bully not less lol
You are pathetic
n3rd