Standard Deviation and Standard Error of the Mean
Вставка
- Опубліковано 2 жов 2024
- This video describes the differences between standard deviation and the standard error of the mean, and how they can be used to interpret data when the values are plotted as error bars on a graph.
Thank you very much for this video! I can't imagine a more clear explanation
Thanks Lena, I am glad you found it helpful!
Kevin Piers what I don't understand is where does the number 6.33 at 4:18 come from?
Standard deviation of a list of red-colored numbers (averages from 12 experiments with Jack, on 3:40), that is of [100, 101, 99, 114, 103, 101, 95, 99, 95, 105, 100, 93], is not 6.33 but 5.25. I've checked with Python with numpy, by hand, and using online SD calculators. All give population SD as 5.25 and sample SD as 5.48.
same calc results. asking too.
it's because he has messed up a lot in this video. the s.d. of the first 5 trials listed for jack is 14.14 (not "16" as he claims later). and the SEM of *that* set is 6.32. so i assume that's what he was referring to with "6.33." ..as for the rest of the video, it's also not a very good explanation of a simple concept: i.e. that you can have quite variable data (say flipping a coin and calling heads "0" and tails "1") which could have a large s.d. no matter how many times you measure it (for the coin example it will approach 0.5 as the number of random flips grows), but nevertheless the uncertainty in the mean itself (represented by the SEM) will tend toward zero as you make more and more measurements, irrespective of how noisy those data are.
the video could've used a better example (for instance, a person running anywhere is not a good example of independent data, as there could be trends in either direction -- faster or slower -- depending on the time between trials; thus these data are not even appropriately lumped together like this) and as you point out, he could've at least double checked his math before posting.
Piers Support
Hello sir,
I am having airborne dust concentrations data as PM10, PM2.5, PM 1 .
These data was taken before and during dust producing work in a civil construction site.
N=5
How can i compare these before and during operations data ?
It seems that there is percent variation in dust concentrations in atmosphere between before and during operation data based on particle size.
Before operation:
PM 10 ( particle size less than 10 microns) is sharing 40% of total airborne dust, and PM 2.5 ( particle size less than 2. 5 micron) shares 10% of total airborne dust.
During machine operation:
PM 10 shares 60% and PM 2.5 10% only.
It seems that PM 10 share is increased due to that machine operation?
Which test is suitable for analysing these type similar data for discussion ?
How to use statistics?
Any comparison among these particle sizes?
thank u.
Four semesters of graduate level statistics, and you accomplished in 10 minutes what four different profs failed to accomplish at all. Thank you!
idk why but I laughed at the example of examining the data of Jack and jill who ran up the hill.
No better explanation than this one. Can't thank you enough Dr. Piers. You're an exceptional instructor with the ability to clearly, precisely and simply explain very complex statistics topics. God bless you. Much appreciation from Uganda - East Africa.
This is thee single best video on this subject on YT, thank you Dr. Piers.
I used R to calculate the standard deviation of the averages and got 5.484828 (instead of 6.33 that you show). Note that the sd() function is R uses denominator n - 1. Why the difference?
Thank you for the clear explanation!
This is a brilliant video because it makes clear what the SEM is. first class! thanks.
he is explaining very well , thank you
The Mean of the 12 means = 100.42 and SD = 5.48.
The SEM = SD/ sqrt(12) = 1.58, not 6.33?
please help me, really dont know how and where to take in the class for Applied Data Science Module that i was accepted from, when i login the worldquant university portal than it only prompts for the Master degree registration
Hi Kelvin, are you able to show how you calculate SEM=6.33? I tried to compute but my answer is about 5.5.
HI Ben. By golly it appears you are right. I put the values into the spreadsheet and got 5.48. Thanks for picking this up and sorry for any confusion!
Thank you very much. This video is really friendly even for beginner who knows nothing. You really know understand the beginner difficulty. Thumbs up for you. :]
THANK GOD SOMEONE ASKED THIS, I was going crazy. I too copied the averages of those 12 trials into Excel, calculated STDEV.S and got 5.48.
Excellent breakdown, thank you for your service.
the standard deviation of the 12 sample means is not 6.33. it is 5.48.
I always explain it this way to students. SD is a measure of how big the spread is between samples. SEM is a measure how exact we know the average (or mean). It starts to get interesting when SD is relatively big. At that point one could argue how valid the test actually was. Because it's perfectly possible that the SD is very big (individual samples are widely spread) but SEM is quite small at the same time.
Thank you for explaining this concept so clearly and with great humor. Very helpful and enjoyable!
Nice video thanks! Would you say in general that when a single person does the same test repeated times (person running a mile 5 times) you should show SEM but when you have a group of people that do the same test such as a team running a mile, then you show the SD?
Most confusing video in my life
This is the best statistics video I have seen. Such clear explanation and great presentation. Thank you!
Thanks for the presentation on Std.Error of Mean. I calculated std.dev of the sample means and it came out to be 5.25. You noted SEM as 6.33, wondering wherefrom it came. Could you clarify
Hello. I was attempting to double check your work at 4:22 and I happen to find different results. Using Excel, the sample standard deviation for the set of averages is 5.483 and the population sample deviation is 5.250. Is this indeed a mistake or am I missing something?
Thank you very much for enhancing my understanding of SD and SEM.
How do you get 6.33? I got 5.25
Great video for SEM concept. Liked and subed. However, a question. SEM=SD/sqrt(n), here SD is directly from one sample set data? shouldn't it be the real statistic SD?
Excellent explanation! my compliments. However when you mentioned Jacks 12 trials,
The Mean of the 12 means = 100.20 and SD = 5.48.
The SEM = SD/ sqrt(12) = 1.58, whereas you mention 6.33
That's ture, my final result is 5.48 as well. I'm quite confused about 6.33...
"The Mean of the 12 means = 100.20" - where did you get this number? The mean of 12 means is 100.42
the SEM is just the calculation of the STD of the mean of that 12 numbers, which means you do not have to /sqrt12.
so the SEM is 5.48 as Mattew mentioned below.
@@matthewx2105you are correct
@@zhongdasun8772 Thanks
I appreciate the simplicity of the explanation, but how is it that when I calculate stdev of the averages in red (from 3:50) using Excel's function STDEV.S, I get 5.48 and not 6.33?
Ferenczi69Aron try STDEV.P
is there a transcript for this video?
yes! if you have a transcript pls send!! :)
Sorry I don’t :(
Ok, so in the last part of the video you show that there is no difference between the two populations. Then you collect som more samples, and show that there is a difference between the two populations. So the question becomes how many samples to collect. How to know that?
6:00 it's never been shown. it is one of those _"if drawn to infinity, and presuming no duel counting - or closer to infinity to compensate, all states are accounted for"_ . it is just an assumption, that cheats by adding infinity. Like adding time travel to a film.
it's been shown by calculating the integral! The area under the graph within 1 SD makes up about 68% of the total area
You are my angel~ Mr. Kevin thank you!
Im confused. When I calculate the SD at 4:25 I get ~5.47. I‘m adding them all up and divide by 12 to get the mean. Then subtract that mean from each value, square it and add it up, dividing the result by 11 and take the square root of that. What am I doing wrong? Excel gives me the same result, far from 6.33.
In calculations, everything seems to be rounded.
For example 5:15 actual stdev is 15.81 but rounded up to 16.
16 / sqrt(5) = 7.155417527999327 and rounded up to 7.16 etc.
In the example of Jack, can we precisely say the exact value of mean or only say the true mean will be between 68 and 100? Thanks a lot Sir.
Sad that I watched it too late. But as it is said, better late than never. Amazing video. Now, I will never forget the difference. Thank you.
I'm glad I found this too. Better late then never honestly
Thank You Sir. It is a best explanation ever i have seen on this topic. Amazing... God bless you.
Thanks for the video with such a nice explanation :)
I didn't understand how you calculated Jack's average time samples around the 8 min mark.
holy shit thank you for this explanation. it all makes sense now!
GRACIAS!!!! Well done! Clearly and cleverly stated.:)
finally the video that answered all my questions! good job and thank you 💪🏼
±2 SE is 95% only for infinite measurements, if not is ± t student (2 tails) for n-1 observations/items
How did you take 6.33 value. Isnt it 1.58...
Thank you very much for the video. I am a medical student. I needed a understanding of the difference between SD & SEM for community medicine. Your video helped me a lot...
very helpful video. thanks again
Am I missing something? I think he's wrong.
90+80+100+120+110 = 500
500 / 5 = 100
Thus 100 is the mean.
(90-100)2 = (-10)2 = 100
(80-100)2 = (-20)2 = 400
(100-100)2 = (0)2 = 0
(120-100)2 = (20)2 = 400
(110-100)2 = (10)2 = 100
100 + 400 + 0 + 400 + 100 = 1000
1000 / 5 = 200
√200 = 14.14
How did the professor come to the conclusion that the Standard Deviation is 16?
From what I recently learned, that when we calculate SD we divide the SUM of Squares by the degrees of freedom (DF) rather than the sample size. DF should be equal to (n - 1) where n is the sample size. In that case we divide 1000/4 and take the square root which should be 15.8113883008419. By rounding off this number the result should be 16.
Great video! So whatvshould we plot SD or SEM.
Read the definition of «standard error» in radiotherapydictionary.blogspot.pt/2016/11/standard-error-se.html
I would have topped in maths if I had found you sooner. 😅
9:37 will these also be the same as the hypotheses testing using softwares like SPSS
Here is how you can see that 68% percent of values lie within +/- 1 standard deviation of the mean, etc.
1. note that the normal distribution is a probability density function
2. note that the indefinite integral of a probability density function is 1
3. now consider what the value would be if you took the definite integral from -1SD to +1SD - it would be (about) 68%.
SANTA MARIA!!! You saved my life!
Tq very much..this is awesome video..helped so much
Would u teach us tests of significance..please
(Standard error of "mean , proportion , mean difference , proportion difference")
Thanks in advance
Thanks for the video. It was extremely helpful! My only question concerns the interpretation of the two graphs at the end of the video where they either completely overlap or don't overlap at all. You stated this means there is either no difference between the two populations or there is a difference. Could you clarify this point? Does it suggest that (in the first graph), it doesn't make a difference whether you watch the videos or not whereas with the last graph, there is an impact, on say, a student's grade?
That is certainly a good summary of the graphs as far as I understand it. There are probably more nuanced interpretations, but that is a good working model. Cheers.
Kevin Piers Thank you for your response. It helps clear up some things.
THANK YOU. I have consulted 10+ sources trying to figure out the difference and was beginning to feel like a complete idiot until you.
Best explanation! much thanks.
This solves my question which bothers me for so long. Thank you so much!
This man really knows his stuff!
that's is really a simple access to the two complex concepts, thanks, that's really helpful for the psychology student without a math background to understand. you really knows what we are confusing in each steps.
Thank you very much for taking the time and effort to prepare this is a really very interesting and useful video. God bless you
Thank you so much for the video.
Excellently explained, please make more videos of these sort in statistics and any other topic. :) Great job again!!
This was such a helpful video thank u so much
very good and quick refresher of SD and SEM evaluations❤ Thank you!!!
Sir i have a question the SEM we calculated is the standard deviation of the sample means right if we were to take multiple samples: So why do apply it lastly to the sample mean , in the video at 7:55 u applied that SEM to the sample mean while it should have been applied to the mean of the sample means. Can Someone please explain?
“Inception like qualities”🤣
An Incredible way to put it!
so, sem is useful because it represents less influence from outliers?
thank you professor God bless you
watching this and I have a submission in two days, really helped thank you Doc!
subscribed just after seeing this video..cant explain how aewsome this explanation is
Thanks sir it's very help full
I need to watch it twice to confirm my understanding. Thank you for a clear explanation
I know Dr. Piers says he doesn't care about how we know that 95% of the probability lies within 2 SDs of the mean in a normal distribution, but it's actually really simple. Take an integral from -2 SD to 2 SD of a normal (or Gaussian) distribution. Anyone who's worked with probability distribution functions should be familiar with doing things like this, since you integrate the probability distribution function to calculate probabilities (indeed that's what we're doing here) or cumulative distribution functions.
Excellent explanations of sd and adm
Clear explanation of standard error mean
Thank you!
Well explained
Thank you for making this video, I have a chem quiz on this and for some reason I couldn't wrap my head around the subject, your video helps alot bro.
Thank you! Appreciate you taking time to explain. I cannot say how much this helped me.
So helpful video! Thank you Kevin Piers!! ((Wow =your failure is Statistically significant!!!)
Thank you so much, this is a wonderful tutorial. I am a TA for a data analysis course and was having the hardest time figuring out how to emphasize the difference between SD and SEM. You did it very clearly and simply!! You just saved my students from a very confusing lab explanation :)
Thanks for the positive comment Genevieve! I'm glad you found it helpful.
Thanks for the video! I have the mean difference of two data sets, how do I calculate the standard deviation of the mean differences, please?
You are a legend sir 🙏
Fantastic explanation. Thanks a lot for your time and help.
Great video!
If your students do not understand the video well, it's only going to be because you didn't use your golden catchphrase:
H E R E ' S
T H E
K I C K E R
Also, would you say that the way students are attentively drawn to these videos can be described as positive phototaxis?
This video is most important for the understand of S.D.& S.E
.Thnku so mush sir!!
Thank you Dr Piers!
Thanks very much. This info is statistically a life challenger.
For 4:45 I spent nearly a week trying to get the explanation as to why the formula is different
Thank you for this video, it's a must-watch for statistic student
Wonderful video!
wow really fantastic!
very simple but effective way to teach these two terms and their use. Thanks a lot.
So, SE is standard error. SE = SD/sqrt(n). SEM is standard error of the Mean. SEM = to what? According to Dr. Piers @ 4:06, SEM is the SD of the Mean. This makes absolutely no sense, both terminology and usage. When do you use SE? Everyone explains it differently and uses it differently.
SEM is to be used when you have garbage data, SD for good data. SEM should not be used, unless you tried it on the same subject, with several trials.
I'm doing a stats class now and I understand more from this 10 minutes than the whole lecture on it. Much thanks
Watched this three times and finally I understood this. The best explanation I have found on this topic, no doubt!
There is a slight error in your interpretation of the 95% confidence interval. You say that the error bar contains an interval that, if you were to repeat your experiment 100 times, would contain the average 95 of 100 times. Instead the confidence interval (indicated by the error bars) changes each time you repeat your experiment. The correct statement is that in 95 out of 100 times, the calculated error bars will contain the “true mean.” This is true for both SEM and for SD. Otherwise, a truly excellent explanation with great graphics!
Thanks for the clarification Rebecca!
Awesome
very good thank you!
Really good refresher to these statistics ideas, I had completely forgotten what standard deviation and SEM was! Thanks a lot!