Your videos on regression are amazing! Interesting, clear, very informative, you the stats & intuition behind regressions into something fun to lose oneself in :D
Thanks. My professor threw this into the review of SLR and MLR (which we didn't originally learn), but he didn't explain it very well. This video was a big help Edit: and by "this," I mean interaction terms
20:08 you should also take into account that being in cat4 gets you -0.390 so apart from adding 0.123 of the pink slip, you should subtract 0.390 fro being in cat4
think of it in terms of marginal effect: consider that you have a simpler form of regression with only pink slip (X1) and agecat4( X2) + interaction term (X1*X2) with a coef of b3, if you re going to asses the marginal effect of the var1 ( pink slip ) , you ll proceed with the derivative of Y over deriv of X1 and the result il the following ( Dy/Dx1 = b1+ b3* X2 ) , thus considering the " boolean" nature of X2 if X2=1 , then Y will increase by b1+b3 else it will only increase by b1, same thing if you want to see the marginal effect of X2 ! => Conclusion : always think of it as the derivative over the variable and you ll get the answer
There is no omission: Ln(P) = constant (b0) - 0.181Ac2 - 0.800Ac3 - 0.390Ac4 -0.209 LnD + 0.123PS +1.371(Ps x Ac4) The term [- 0.390Ac4] is what you counted as omitted, but it is present. Model7 Coef Var (ind) LN Anti-Ln c 9.125 1 9.1250 ac2 -0.181 0 0.0000 ac3 -0.800 0 0.0000 ac4 -0.390 1 -0.3900 lnd -0.209 5.669880923 -1.1850 ps 0.123 1 0.1230 psac 1.371 1 1.3710 9.0440 8,467.54 (Apologies, the table does not show very well here)
I love your videos, especially this video has given me the insight on explaining a regression model with interactions which I have struggled with a lot. Thank you so much
Hi Justin, hope you are doing great! I love your videos and have been following them on your website as well. I have one doubt in this video. At 3:48, where you added pink slip variable, you say that having a pink slip increases the price by 15.6% as coeff for pink slip is 0.156. I am confused here as y is logged and when y is logged and x is not, general interpretation is that 1 unit increase in x means exp(coeff)-1 percent change in y. Please help me understand where I am going wrong. Thanks :)
iam totally sold by ths video. i mean your teaching pattern is totally upto par. please make some videos on logistic regression, random forest, neural network and clustering as well.
Thank you for the interesting videos. I think there is a little bit confusion in the interpretation of the resulting coefficients. The change in one of the independent variables holding all the others constant does not mean increasing the dependent variable by the same percentage. Please check an example of this at 11:28
In doing an actual analysis, would you still add the pink-slip coefficient to the pink-slip x cat4 interaction even when the pink slip variable alone is not statistically significant?
@@malikakbar It does, even if the variable on itself is not statistically significant. Think about this scenario: There's a new AgeCat5 variable that is added for all cars that are over 70 years old; and also we are gonna add another variable called redCar, that takes 1 as a value if the car is red, and 0 otherwise. Now, if I were to ask you: "Does the color of the car being red have an impact in the price of the car?" Probably not, cars are customizable and the color on itself doesn't seem to be too relevant to determine the price of the car. For the sake of the example, let's stirr things up and claim that in the 40's(80years ago) almost all the cars were either grey or black, and only BMW was producing fancy red cars that were way more expensive than the grey/black cars; but since it's been so long ago it's really hard nowadays to find those special edition BMW cars. So, I believe it's pretty easy to tell that if you find a car that is over 80years old whose original paint is red, it's almost certain that it is one of those special edition BMW red cars that are way more expensive than the others. So, for all the other cars that were fabricated less than 70 years ago, the color is not a factor that affects the price of the car, so the variable "redCar" is probably not gonna be statistically significant on itself, but in the case that the car is 70years older, the color of the car is gonna play a huge role to predict the value of the car, because if it is red,. the price is gonna be way more expensive, therefore, it's gonna be significant in that scenario. In this case, intuiton tells me we should add another predictive variable to the equation that describes that relationship, which you would do like this "+ Beta6 * redCar*ageCat5
This is probably a very basic question - you added variables to get the final price of your car at the end of IVb that were not statistically significant - why - thanks gary
Hey Z, You mentioned interaction terms should only be included when the added IV(2) affects the relationship between IV(1) and Y. How do you determine the affect of adding IV(2), is it a change in the regression coefficient of IV(1)? I’m conducting orthopaedic related research involving canine models, and I’m using multiple regression models to control for the size of the specimen. This would really help me out as I’m working to nail down models for our various parameters.
Actually I think it is when the p value of the interaction term is statistically significant. Then you see there's an effect modifying e.g. age modifying selling a car with a pink slip.
Firstly Thannk You so much Sir for your videos filled with detailed explamation! Any query pops up in my mind..mostly gets solved up within few minutes in the videos...Got a clear idea about Interaction terms and correlation.. So just a small doubt....is correlation only difference bw Multicolineaity and Interaction terms...as we donot prefer Multicolinearity...?
Why all videos on interaction terms discuss only the cases where both variables are expected to have positive impact on Y and the results of the interaction are also positive? What about having 2 independent variables (x1 and x2) with one of them having positive impact on Y and another - negative. How to interpret the interaction term if it is positive? If it is negative?
Don't you have to exponentiate the coefficient of the dummy variable and then interpret it as a multiplier? The sales are still logged, so I guess you would have to exponentiate both sides of the equation to see the change in sales from non pink slips to pink slips.
Hey, what if the categorical variable would have had a lot of levels (nationality, religion...) so that it would effectively mean so many variables that the adjusted R squared would be very low? And 2) - when some variables in your model are insignificant, should't you remove them from the model? but then some of the then-significant variables become insignificant... :(
So when do we actually interact our variables? Is there a way to see if it is necessary or do we just do it and then see if the coefficient on the interaction term is statistically significant?
Why do we say there's a 47.4% increase in price in Cat4 as compared to Cat1 and why don't we read it as a 47.4% jump from Cat3, i understand that the base category is 1 but i fail to see the reason behind the interpretation
Yes I was thinking this as well. I have always learned that amount of dummies = category levels (age in this case 4) - 1 = amount of dummies that meet be created
Restoring the Ac1 (age 1 to 5 years) into the Model is okay. However, the value of Ac1 = 0, annulling the entire term and contributing nothing to the Model, just like the Ac2 and Ac3. Other interactions are feasible. (e.g., odometer vs. age).
Changing the format of age into categorical variable increased the significance of pink slip. Is this by chance or it happens for every variables? Is it okay if we had just dropped the pink slip variable for the next model without changing the format of the age variable?
Why did you not include [-0.390(AgeCat4)] when the car is older than 35 years and has a pink slip? Isn't your answer incorrect? The answer should be, "... for models older than 35 years, attaining a pink slip increases the price by an average of 110.4%, holding all else constant."
I don't understand why we turned age, a continuous variable, into a categorical variable. Was that just for explaining how a multi-level categorical variable works or it was actually important to our model?
In the model 7, the pvalue for pink slip is 0.5 and therefore, quite highly insignificant. Shouldn't it be removed? If yes, then after removing the base variable, can the interaction variable pink slip*agecat4 be retained in the model?
why do we add the pinkslip and pinkslip*agecat4 coefficients for the interpretation but not the agecat4? is it because it's not significant? or because we were only interpreting the coefficient of pinkslip with the condition that agecat4 is true?
You can also interpret the effect of agecat4, which works the same way. Models that are in agecat4, but have NO pink-slip, have on average a 39% lower price than the baseline, holding all else constant. If they do have a pink-slip, the price is on average 98,1% (1,371-0,39) higher than the baseline, hold all else constant.
Here you intuitively decided that the coefficient of a certain feature is not as you would expect. So usually we have thousands of dimensions, how can we know if the coefficients make sense? any sort of analysis other than manual checking of coefficients we can do here?
Hi, can we use the model for Datsun if pinkslip's pvalue is not significant and we are highly restrictive? I'm asking because pvalue is higher than 0.05 and with common methodology it seems insignificant.
1. In the regression models with an intercept, the coefficients can not be interpreted as % change as the coefficient doesn't effect the constant intercept. 2. At 20mins you forget to include the ageCat4 solo term. I.e. there are 3 terms to sum not just 2. Otherwise well done for a perspicuous explanation!
Hi! Thanks for watching! Regarding your observations: 1. They certainly can be interpreted as a % change on y. Remember that if you were to exponentiate the log, youd find that y=exp(B0+B1X). Or in other words y = exp(B0)*exp(B1X), so the constant term just becomes a multiplying constant (ie. won't affect the percentage change in y for a given absolute change in X). 2. The ageCat4 term is def there! Just on the second line of the equation. Hope that helps.
@@zedstatistics 1. Erm exactly my point. When B1 increases by 10%, B1X increases by 10%. But Exp(B1X) does not. 2. It was in the forumula but not summed
@@thenineteennn Be careful here, B1 does not increase at all. B1 is the coefficient. It is X that increases. Also, if you're talking about a log-linear relationship (ie. ln(y)=B0 + B1X...) then a 1 unit increase in X has a constant % effect on Y. As per my first response. If you have a log-log relationship (ie. ln(y)=B0+B1(ln(X))), only THEN can you interpret it as "a 1% increase in X relates to a 1% increase in Y". But this is different to your example above. Hope that helps :)
Any combination can interact! But to what effect? If the interaction is INSIGNIFICANT, we save time by ignoring the crossing, except we are constrained, say in an exam setting.
@@EpicLaith By now, you have settled this matter. To use Excel, change 'log' to 'LN' and add = ahead of the minus sign. = - 0.209 * LN(290) and enter (voila!)
Pre-ZedStats I respected my profs and trusted they knew best even though it was impossible to understand the jargon they mumbled. Nowadays I'm furious that those small-minded fools are getting thousands of dollars for smugly explaining nothing while I'm spending hours each week learning from ZedStats' free videos. American universities are the worst.
@@zedstatistics but without considering Cat4, that term is 0. You should combine the effect of having the pinkSlip on cat4, not over Cat1 (BASE category). Thus, the price increases 110.4%.
Your videos on regression are amazing! Interesting, clear, very informative, you the stats & intuition behind regressions into something fun to lose oneself in :D
One of the best explanations of the Dummy Variable Trap and how to circumvent it!
This video was exactly what I needed. Clear and thorough. Keep it up!
These videos are a godsend. These are going to save my life for my massive regression analysis research paper. Thanks for the great content!!
Thanks. My professor threw this into the review of SLR and MLR (which we didn't originally learn), but he didn't explain it very well. This video was a big help
Edit: and by "this," I mean interaction terms
How'd the rest of your class go?
20:08 you should also take into account that being in cat4 gets you -0.390 so apart from adding 0.123 of the pink slip, you should subtract 0.390 fro being in cat4
You are absolutely right in pointing this omission
think of it in terms of marginal effect: consider that you have a simpler form of regression with only pink slip (X1) and agecat4( X2) + interaction term (X1*X2) with a coef of b3, if you re going to asses the marginal effect of the var1 ( pink slip ) , you ll proceed with the derivative of Y over deriv of X1 and the result il the following ( Dy/Dx1 = b1+ b3* X2 ) , thus considering the " boolean" nature of X2 if X2=1 , then Y will increase by b1+b3 else it will only increase by b1, same thing if you want to see the marginal effect of X2 ! => Conclusion : always think of it as the derivative over the variable and you ll get the answer
There is no omission: Ln(P) = constant (b0) - 0.181Ac2 - 0.800Ac3 - 0.390Ac4 -0.209 LnD + 0.123PS +1.371(Ps x Ac4)
The term [- 0.390Ac4] is what you counted as omitted, but it is present.
Model7 Coef Var (ind) LN Anti-Ln
c 9.125 1 9.1250
ac2 -0.181 0 0.0000
ac3 -0.800 0 0.0000
ac4 -0.390 1 -0.3900
lnd -0.209 5.669880923 -1.1850
ps 0.123 1 0.1230
psac 1.371 1 1.3710
9.0440 8,467.54
(Apologies, the table does not show very well here)
I was wondering about this as well. Glad you caught it as well!
Edit: He did do it while calculating price in model 7 at the very end.
That time slot is about the effect of attaining a 'pink slip' and the -0.390 item is only for the age of the car, hence not included, I think.
Love the excitement of the pink slip variable becoming significant lol 9:55
I love your videos, especially this video has given me the insight on explaining a regression model with interactions which I have struggled with a lot. Thank you so much
You explain 100 times better than my statistics teacher, you've just save my exam, thanks a lot !!
Hey I know it's been awhile, but I just came across your comment and was curious. How'd the rest of the class go?
Thank you SO much for this series! Best I've seen on UA-cam so far!
Categorical X? More like "Certainly the best." These videos rock!
Hi Justin, hope you are doing great! I love your videos and have been following them on your website as well. I have one doubt in this video. At 3:48, where you added pink slip variable, you say that having a pink slip increases the price by 15.6% as coeff for pink slip is 0.156. I am confused here as y is logged and when y is logged and x is not, general interpretation is that 1 unit increase in x means exp(coeff)-1 percent change in y. Please help me understand where I am going wrong. Thanks :)
I think it is because he is not doing a logistic regression but a linear regression.
iam totally sold by ths video. i mean your teaching pattern is totally upto par. please make some videos on logistic regression, random forest, neural network and clustering as well.
Thank you for the interesting videos.
I think there is a little bit confusion in the interpretation of the resulting coefficients. The change in one of the independent variables holding all the others constant does not mean increasing the dependent variable by the same percentage. Please check an example of this at 11:28
On the time stamp you provided, I think it should be 60.64% higher rather than 47.4%.
Yes! I found the same issue in the last video as well, it should be: the median of the car price will be e^0.474=1.6064 times. Thus 60.64% increase
@@imglenngarcia I think the pink slip interaction term case also has the same problem..
brooooo, how did I not see your videos in 5 years *sight* i would have gone through uni so much easier!!
"15-35, well, they're the shit boxes aren't they?" lol
Think about the kids learning advanced regression! /s lol
Sir, you are amazing. Thank you for making these videos.
Change ur channel name to "zstatistics for machine learning" you goona soon have million subs
In doing an actual analysis, would you still add the pink-slip coefficient to the pink-slip x cat4 interaction even when the pink slip variable alone is not statistically significant?
I have the same question, would appreciate if anyone willing to share the answer for this one
@@malikakbar It does, even if the variable on itself is not statistically significant. Think about this scenario:
There's a new AgeCat5 variable that is added for all cars that are over 70 years old; and also we are gonna add another variable called redCar, that takes 1 as a value if the car is red, and 0 otherwise.
Now, if I were to ask you: "Does the color of the car being red have an impact in the price of the car?" Probably not, cars are customizable and the color on itself doesn't seem to be too relevant to determine the price of the car.
For the sake of the example, let's stirr things up and claim that in the 40's(80years ago) almost all the cars were either grey or black, and only BMW was producing fancy red cars that were way more expensive than the grey/black cars; but since it's been so long ago it's really hard nowadays to find those special edition BMW cars.
So, I believe it's pretty easy to tell that if you find a car that is over 80years old whose original paint is red, it's almost certain that it is one of those special edition BMW red cars that are way more expensive than the others.
So, for all the other cars that were fabricated less than 70 years ago, the color is not a factor that affects the price of the car, so the variable "redCar" is probably not gonna be statistically significant on itself, but in the case that the car is 70years older, the color of the car is gonna play a huge role to predict the value of the car, because if it is red,. the price is gonna be way more expensive, therefore, it's gonna be significant in that scenario.
In this case, intuiton tells me we should add another predictive variable to the equation that describes that relationship, which you would do like this "+ Beta6 * redCar*ageCat5
This is probably a very basic question - you added variables to get the final price of your car at the end of IVb that were not statistically significant - why - thanks gary
omg you have saved my thesis now. Thank you 3000
Very well explained, Thank you so much !!. Stay safe Professor !.
Awesome videos man! You are a very good teacher :-)
Quality content and well explained. Thanks a ton!
Hey Z,
You mentioned interaction terms should only be included when the added IV(2) affects the relationship between IV(1) and Y. How do you determine the affect of adding IV(2), is it a change in the regression coefficient of IV(1)?
I’m conducting orthopaedic related research involving canine models, and I’m using multiple regression models to control for the size of the specimen. This would really help me out as I’m working to nail down models for our various parameters.
Actually I think it is when the p value of the interaction term is statistically significant. Then you see there's an effect modifying e.g. age modifying selling a car with a pink slip.
Awesome videos, any book you particularly recommend to understand how to do linear regressions?
You forgot minus 1 at the end 😉
Thanks for the awesome content
You saved my thesis
Firstly Thannk You so much Sir for your videos filled with detailed explamation! Any query pops up in my mind..mostly gets solved up within few minutes in the videos...Got a clear idea about Interaction terms and correlation..
So just a small doubt....is correlation only difference bw Multicolineaity and Interaction terms...as we donot prefer Multicolinearity...?
thank you Professor, why do we have to spare the variable age in model 5 ?
amazing work with the car example
Why all videos on interaction terms discuss only the cases where both variables are expected to have positive impact on Y and the results of the interaction are also positive? What about having 2 independent variables (x1 and x2) with one of them having positive impact on Y and another - negative. How to interpret the interaction term if it is positive? If it is negative?
Don't you have to exponentiate the coefficient of the dummy variable and then interpret it as a multiplier? The sales are still logged, so I guess you would have to exponentiate both sides of the equation to see the change in sales from non pink slips to pink slips.
Same here, I thought it was (exp(coef) - 1) * 100 for non-transformed independent variables. Someone please correct me.
Hey, what if the categorical variable would have had a lot of levels (nationality, religion...) so that it would effectively mean so many variables that the adjusted R squared would be very low?
And 2) - when some variables in your model are insignificant, should't you remove them from the model? but then some of the then-significant variables become insignificant... :(
So when do we actually interact our variables? Is there a way to see if it is necessary or do we just do it and then see if the coefficient on the interaction term is statistically significant?
Why is this video categorized as comedy?
statistics is funny
...to them that are statistically lost, it is indeed a comedy! 1CO1!18
Why do we say there's a 47.4% increase in price in Cat4 as compared to Cat1 and why don't we read it as a 47.4% jump from Cat3, i understand that the base category is 1 but i fail to see the reason behind the interpretation
Hi, thanks for your videos. They are being very helpful to me.
Could we simply code the new AgeCat variable such that:
AgeCat:
= 0 if age
Yes I was thinking this as well. I have always learned that amount of dummies = category levels (age in this case 4) - 1 = amount of dummies that meet be created
Restoring the Ac1 (age 1 to 5 years) into the Model is okay. However, the value of Ac1 = 0, annulling the entire term and contributing nothing to the Model, just like the Ac2 and Ac3. Other interactions are feasible. (e.g., odometer vs. age).
Changing the format of age into categorical variable increased the significance of pink slip.
Is this by chance or it happens for every variables?
Is it okay if we had just dropped the pink slip variable for the next model without changing the format of the age variable?
Why did you not include [-0.390(AgeCat4)] when the car is older than 35 years and has a pink slip? Isn't your answer incorrect? The answer should be, "... for models older than 35 years, attaining a pink slip increases the price by an average of 110.4%, holding all else constant."
It should be e^1.104 times higher
@@helloworld1537 We generally think in decimals.
You're a legend!
If you add age and ln(age) in the model, it will create multicollinearlity. Which coefficient is useful, age or ln(age)?
I don't understand why we turned age, a continuous variable, into a categorical variable. Was that just for explaining how a multi-level categorical variable works or it was actually important to our model?
The interpretation of the pink slip coefficient in Model 7, 19:44 - is that still 149,4% relative to cat 1?
In the model 7, the pvalue for pink slip is 0.5 and therefore, quite highly insignificant. Shouldn't it be removed? If yes, then after removing the base variable, can the interaction variable pink slip*agecat4 be retained in the model?
I have the same exact question. Have you found the answer?
If an interaction variable is SIGNIFICANT, t]then its components must be admitted into the model.
These videos are fantastic thank you!
Thankyou so much for such an amazing explanation you're my saviour thankyou :)
15:35 really clarifies it
"but mate, this thing goes"
bloody oath
why do we add the pinkslip and pinkslip*agecat4 coefficients for the interpretation but not the agecat4?
is it because it's not significant?
or because we were only interpreting the coefficient of pinkslip with the condition that agecat4 is true?
You can also interpret the effect of agecat4, which works the same way. Models that are in agecat4, but have NO pink-slip, have on average a 39% lower price than the baseline, holding all else constant. If they do have a pink-slip, the price is on average 98,1% (1,371-0,39) higher than the baseline, hold all else constant.
Here you intuitively decided that the coefficient of a certain feature is not as you would expect. So usually we have thousands of dimensions, how can we know if the coefficients make sense? any sort of analysis other than manual checking of coefficients we can do here?
Amazing content!!!
Any idea how does one calculate SE for multi parameter non-linear regression?
Hi, can we use the model for Datsun if pinkslip's pvalue is not significant and we are highly restrictive? I'm asking because pvalue is higher than 0.05 and with common methodology it seems insignificant.
I noticed you're interpreting insignificant variables multiple times. Could you please leave these out when performing calculations?
How removing one agecategory variable is making sense and including all four wont?
Great video! Thank you!
How will I understand the price conversion e9044 to real dollar terms $8468. Can you please help me understand!
extremely clear!
1. In the regression models with an intercept, the coefficients can not be interpreted as % change as the coefficient doesn't effect the constant intercept.
2. At 20mins you forget to include the ageCat4 solo term. I.e. there are 3 terms to sum not just 2.
Otherwise well done for a perspicuous explanation!
Hi! Thanks for watching! Regarding your observations:
1. They certainly can be interpreted as a % change on y. Remember that if you were to exponentiate the log, youd find that y=exp(B0+B1X). Or in other words y = exp(B0)*exp(B1X), so the constant term just becomes a multiplying constant (ie. won't affect the percentage change in y for a given absolute change in X).
2. The ageCat4 term is def there! Just on the second line of the equation.
Hope that helps.
@@zedstatistics
1. Erm exactly my point. When B1 increases by 10%, B1X increases by 10%. But Exp(B1X) does not.
2. It was in the forumula but not summed
Sorry ignore 2.
@@thenineteennn Be careful here, B1 does not increase at all. B1 is the coefficient. It is X that increases. Also, if you're talking about a log-linear relationship (ie. ln(y)=B0 + B1X...) then a 1 unit increase in X has a constant % effect on Y. As per my first response.
If you have a log-log relationship (ie. ln(y)=B0+B1(ln(X))), only THEN can you interpret it as "a 1% increase in X relates to a 1% increase in Y". But this is different to your example above. Hope that helps :)
@@zedstatistics how come at 20:00 you added the main effect of pink slip but didn't subtract the main effect of agecat4?
Thank you!
I love u, made my life so much easier
You are the best!
This was so helpful!
Keep posting!
thank you very much for this video
brooo, you didnt show how you coded the interaction term with dummy vars
excellent!
well-done bro !
brilliant!!!
Valeu!
Tx sir
How do you identify which two variable can interact. is this based on business or domain knowledge
I think according to what he said is a bit subjective (domain knowledge)
Any combination can interact! But to what effect? If the interaction is INSIGNIFICANT, we save time by ignoring the crossing, except we are constrained, say in an exam setting.
thanks
👌
can i remove one control variable in the model ? and dont remove the others ( interaction and one control variable )
Whatever you remove demands that you recalculate your Model Regression and obtain new coefficients and significances.
Why you take (Age)^2
haha i can get everything but cannot solve -0.209Ln(290). can someone please quickly show me the calculations for this?
Why not? Use a calculator and type in -0.209log(290)
@@EpicLaith By now, you have settled this matter. To use Excel, change 'log' to 'LN' and add = ahead of the minus sign. = - 0.209 * LN(290) and enter (voila!)
Pre-ZedStats I respected my profs and trusted they knew best even though it was impossible to understand the jargon they mumbled. Nowadays I'm furious that those small-minded fools are getting thousands of dollars for smugly explaining nothing while I'm spending hours each week learning from ZedStats' free videos. American universities are the worst.
Big mistake in min 19:15 , you should have considered the coefficient of Cat4 honey
We're talking about the effect of pinkslip. So not a mistake, sweetie.
@@zedstatistics but without considering Cat4, that term is 0. You should combine the effect of having the pinkSlip on cat4, not over Cat1 (BASE category). Thus, the price increases 110.4%.
Could you elaborate? Please
@@OskarBienko he shoud have sum the coefficients when cat4=1, pinkslip=1 and pinkslipXcat4=1
The extra effect of pinkslip is zero without consdering cat4
For log-level regression interpretation
sites.google.com/site/curtiskephart/ta/econ113/interpreting-beta
18:00 nah bro but thanks.
that s ar really cheap and cool car
speak LOUDER man!!!!!
Rock that volume button bruh!