What is Least Squares Estimation?
Вставка
- Опубліковано 1 жов 2024
- Explains Least Squares (LS) Estimation with two examples: 1. line-fitting a data set, and 2. digital communications. Derives the LS equation and shows how it can be viewed as a pseudo inverse.
Related videos: (see iaincollings.com)
• What is Fisher Information? • What is Fisher Informa...
• What is an Adaptive Step Size in Parameter Estimation? • What is an Adaptive St...
• What is the Kalman Filter? • What is the Kalman Fil...
• How are Matched Filter (MF), Zero Forcing (ZF), and MMSE Related? • How are Matched Filter...
• MIMO Communications • MIMO Communications
• What is Intersymbol Interference ISI? • What is Intersymbol In...
• Signal Model for MIMO and CDMA • Signal Model for MIMO ...
• What is a Decision Feedback Equalizer (DFE)? • What is a Decision Fee...
For a full list of Videos and Summary Sheets, goto: iaincollings.com
What I love about this channel is the consistency. Every week there's a new good video. Keep it up Iain!
Thanks. I'm glad you're liking the videos each week. I'm enjoying making them.
Have you considered offering a full fleged communication systems MOOC on edX or Coursera, etc? I know of only one offered on communication systems (on edX) so far and I don't think it has been offered again for a long time. You're a really good teacher!
Thanks for the suggestion. It's occurred to me, but I haven't looked into it. I might give it some thought. I'm glad you like the videos.
Hello, I am Muhammad.
I am studying a master’s degree in communications engineering, specializing in radio and mobile communications systems. Can you provide me with suggested titles for the master’s thesis !
With best wishes
Will the sum of residuals for a best fit line for the given data equal zero?
Great question! I've never actually thought about that before, but I'm sure it's true. I can't think of how to prove it right now, but I tried running some examples in Matlab, and they all give the sum of residuals in the order of 10^(-14), so that's as close to zero as numerical accuracy goes as far as I'm concerned.
@@iain_explains I see, thanks!
Thank you for the Least Squares Estimation explanation in this video. I need to work with Adaptative Digital Beamforming systems and this video will help me a lot!. Can I suggest if you could please share information in a video about the IQ modulation/demodulation ? Thanks!
Thanks for the suggestion. I've got it on my "to do" list. In the meantime you might like to watch these videos that are on the topic: "What is a Constellation Diagram?" ua-cam.com/video/kfJeL4LQ43s/v-deo.html and "Is the Imaginary Part of QAM Real?" ua-cam.com/video/6asDtzaVjbQ/v-deo.html
Hey I see some weighted least square problems are solved through iterations…
Please explain what is the condition when we need to solve it iteratively…?
There are lots of different versions and related algorithms that involve iterations, so there's not one single answer to your question. However, one important aspect is that it is computationally expensive to calculate inverses of large matrices (eg. (H^TH)^(-1) in this case), so sometimes it is a good idea to formulate the problem as "repeated smaller experiments", each with fewer measurements (shorter "y" vector), and then iterate.
Thank you
Hi Iain, just wanted to say that your teaching is exceptional - so calm, clear and concise. Thank you so much for the effort you put in, which I am sure is not insignificant. It is massively appreciated!
Thanks Betsy, that's so great to hear. I'm really glad you like the style and content of my videos.
Can you help me in non-linear least squares estimator for trajectory estimation of a vehicle?
Is that possible to get rid of the noise if number of the equations equals to the number of variables? Seems analogues to the situation where we have just two points and would like to estimate a function across them - no chanse to eliminate residual?
It's not getting rid of the noise, it's estimating the parameters in the presence of noise.
Of course it's just estimation, so the niose is always there, but i wonder if this method is able to minimize the noise if the H is squared MxM matrix ?
Love you! You solved tons of doubts. What you’re explaining it’s highly correlated with my university course of Satellite Navigation.
That's great to hear.
Why do one always take the square? Why cannot we maximize the simple term (argminI(y-Hx)I)?
It's easier to calculate the derivative of the square function to find the optimal points, compared to the modulus function.
In the LMS alg. is there always a straight line the model (y=beta1*x + beta2), or can that also be another graph (e.g. parabel)?
Thank You.
You can estimate higher order lines/curves, yes. For example, considering the model on the top right hand side of the page, you could add a column to the D matrix with elements d_i^2, and extend the beta vector by one (to include beta_2). I've already included the code for this in the accompanying file. See www.iaincollings.com/probability-and-random-variables#h.5vsqt9fvre40 (under the heading "Estimation and Hypothesis Testing").
Your videos are brilliant, thank you. In this example, it is clear what the data measurements could be (e.g. temperature). But, with respect to a MIMO system where you want adjust the antenna elements, what is there the meaaurement? Perhaps you can take the meaaurement from the received signal and then calculate the optimal weights for the last step. Perhaps these calculated weights could be the measurements.
It all depends on the equation you're dealing with. Often in communication systems there is a training period, where training data is sent which is known/expected at the receiver. Then the input is known, and the received signal is known (measured), and the unknown variables are the channel path gains to each antenna element. See: "Channel Estimation for Mobile Communications" ua-cam.com/video/ZsLh01nlRzY/v-deo.html
谢谢！My course design is adaptive algorithms for beamforming and this video may help
Great. I hope it helped. Let me know if there are other specific topics you'd like me to cover.
Pulled this up as I'm learning about adaptive filters. It's almost a weekly occurrence, learning a new topic, struggling to wrap my head around it, wondering "Hm, has Dr. Collings covered this?" and finding the answer is yes, of course he has
That's great to hear. I'm so glad you're finding the videos helpful.
These videos might also be helpful for your current topic:
- "What is an Adaptive Step Size in Parameter Estimation?" ua-cam.com/video/Nwm1cngRta8/v-deo.html
- "What is the Kalman Filter?" ua-cam.com/video/OiUS2926nQM/v-deo.html
- "How does a Radar Track Manoeuvring Targets?" ua-cam.com/video/ibvlKTGQ4zQ/v-deo.html
Nice video, i have one question. What does the "arg" stands for in the formula: x^=arg min Sigma "error^2"..... ?
It means the argument (ie. the variable value) that minimises the function. In other words, it's not the minimum value of the function, but it is the value of x for which the function is a minimum.
@@iain_explains Thansk, that made it very clear to me :)
Hi sir, can you please make a video to teach us how the EM waves are working, who they are traveling to, what the RF signal looks like, and which way we can imagine it and its modulation? I really cannot imagine the whole process. Until now, I am saving without understanding the entire process between the sender and the receiver. Thank you very much! I really appreciate your working .
Thanks for the suggestion. I've added it to my "to do" list.
Very Good video! Could you talk more on the OFTS, that is very interesting
I'm assuming you've seen my OTFS video already? ua-cam.com/video/MvK3zhPrGkk/v-deo.html
It's ongoing research, but I'll add it to my "to do" list to do a follow up video at some point.
Thank you for the lecture, it helped me a lot to understand LS estimation.
Glad it helped!
I like your videos in general, but in this case, seems like you can get the same formula at the end without going through any the calculations in the middle. x = y/H is the same as x = (tran(H)*y) / (tran(H)*H). What am I missing?
Your formula doesn't work when the matrix H is not invertible. And it has numerical problems when H is "close" to not being invertible. In contrast, H'H is always invertible.
@@iain_explains Thanks for answering my question. Is it correct that equation -2y'H+2x'H'H = 0 and x = y/H achive the same goal but the second one is not always solvable?
Well, first of all x=y/H doesn't make sense because H is a matrix. I guess you mean x=inv(H)y , and if so, then if you substitute that into the first equation, you will see that you'll get 0 = 0 , so yes, they achieve the same when H is invertible.
@@iain_explains Thank you.
Can’t believe I get this awesome teacher for free (with ads)!
Glad you like the channel.
Thanks for the derivation of pseudo inverse.
Glad you liked it.
Thanks Iain for this detailed explanation.
Glad it was helpful!
Pseudo inverse of Moore penrose
Yes, exactly.
You are Amazing!
Thanks so much for your nice comment. I'm glad you like the videos.