Why multicollinearity is a problem | Why is multicollinearity bad | What is multicollinearity

Why multicollinearity is a problem | Why is multicollinearity bad | What is multicollinearity
#MulticollinearityInRegression #UnfoldDataScience
Hello ,
My name is Aman and I am a Data Scientist.
About this video:
In this video, I explain in detail, Why multicollinearity is a problem , Why is multicollinearity bad and What is multicollinearity. I explain with example what is multicollinearity and what is problem if we have multicollinearity. in data. I also explain what is the way to handle multicollinearity in Data. Below topics are explained in this video:
1. Why multicollinearity is a problem
2. Why is multicollinearity bad
3. What is multicollinearity
4. Problems with multicollinearity
5. Multicollinearity in regression model
About Unfold Data science: This channel is to help people understand basics of data science through simple examples in easy way. Anybody without having prior knowledge of computer programming or statistics or machine learning and artificial intelligence can get an understanding of data science at high level through this channel. The videos uploaded will not be very technical in nature and hence it can be easily grasped by viewers from different background as well.
If you need Data Science training from scratch . Please fill this form (Please Note: Training is chargeable)
docs.google.com/forms/d/1Acua...
Book recommendation for Data Science:
Category 1 - Must Read For Every Data Scientist:
The Elements of Statistical Learning by Trevor Hastie - amzn.to/37wMo9H
Python Data Science Handbook - amzn.to/31UCScm
Business Statistics By Ken Black - amzn.to/2LObAA5
Hands-On Machine Learning with Scikit Learn, Keras, and TensorFlow by Aurelien Geron - amzn.to/3gV8sO9
Ctaegory 2 - Overall Data Science:
The Art of Data Science By Roger D. Peng - amzn.to/2KD75aD
Predictive Analytics By By Eric Siegel - amzn.to/3nsQftV
Data Science for Business By Foster Provost - amzn.to/3ajN8QZ
Category 3 - Statistics and Mathematics:
Naked Statistics By Charles Wheelan - amzn.to/3gXLdmp
Practical Statistics for Data Scientist By Peter Bruce - amzn.to/37wL9Y5
Category 4 - Machine Learning:
Introduction to machine learning by Andreas C Muller - amzn.to/3oZ3X7T
The Hundred Page Machine Learning Book by Andriy Burkov - amzn.to/3pdqCxJ
Category 5 - Programming:
The Pragmatic Programmer by David Thomas - amzn.to/2WqWXVj
Clean Code by Robert C. Martin - amzn.to/3oYOdlt
My Studio Setup:
My Camera : amzn.to/3mwXI9I
My Mic : amzn.to/34phfD0
My Tripod : amzn.to/3r4HeJA
My Ring Light : amzn.to/3gZz00F
Join Facebook group :
groups/41022...
Follow on medium : / amanrai77
Follow on quora: www.quora.com/profile/Aman-Ku...
Follow on twitter : @unfoldds
Get connected on LinkedIn : / aman-kumar-b4881440
Follow on Instagram : unfolddatascience
Watch Introduction to Data Science full playlist here : • Data Science In 15 Min...
Watch python for data science playlist here:
• Python Basics For Data...
Watch statistics and mathematics playlist here :
• Measures of Central Te...
Watch End to End Implementation of a simple machine learning model in Python here:
• How Does Machine Learn...
Learn Ensemble Model, Bagging and Boosting here:
• Introduction to Ensemb...
Build Career in Data Science Playlist:
• Channel updates - Unfo...
Artificial Neural Network and Deep Learning Playlist:
• Intuition behind neura...
Natural langugae Processing playlist:
• Natural Language Proce...
Understanding and building recommendation system:
• Recommendation System ...
Access all my codes here:
drive.google.com/drive/folder...
Have a different question for me? Ask me here : docs.google.com/forms/d/1ccgl...
My Music: www.bensound.com/royalty-free...

[@dariakrupnova6245]

Wow, I think I owe you my mark on the Econometrics final, you blew my mind, I had no idea it was so simple. Thank you!

[@sangeethasaga]

Never seen someone with such a clear understandable explanation...thank you so much!

[@jhonatangilromero2311]

It is evident that a lot of work goes into developing these very informative videos. Thank you!

[@swatikute219]

Amazing pace, crisp word selection and good examples, thank you Aman for great videos !!

[@UnfoldDataScience]

Thanks Swati.

[@shahbazkhalilli8593]

I don't know which one should I take. By the way video is great

[@swatikute219]

If x1 and x2 are strongly correlated then we should check their individual correlation with target and will select the variable which is highly correlated with target and can also check p value for the variables.

[@UnfoldDataScience]

Correct. Thank you.

[@rizalrifai6010]

whats p value ?

[@datapointpune6216]

Very Informative aman

[@UnfoldDataScience]

Thanks a lot.

[@KastijitBabar]

The best explaination on whole UA-cam! Thank You.

[@datafuturelab_ssb4433]

Great explaination sir . Thanks for sharing and making my fundamentals strong

[@UnfoldDataScience]

Most Welcome.

[@koustavdutta1176]

Firstly great explanation !! Now coming to your question, we have to check the bi-variate strength between dependent variables with independent variables. The independent variable with weakest strength should choose to remove from model

[@UnfoldDataScience]

Awesome. Thank you. :)

[@jamiainaga5853]

what is bi- viariate?

[@sowsow5199]

@@jamiainaga5853 the two variables that have been found to be highly correlated with each other

[@kavankomer3048]

How to find this bi-variate strength?

[@sanjeevkmr5749]

Thanks a lot for the detailed discussion on this topic. For the question asked in the video(Which feature to be removed incase of high correlation), I guess among the two, we have to remove the one which least contributes(less correlated) with the target variable. In that way, we will be able to preserve the feature which has high contribution.

[@UnfoldDataScience]

Thanks Sanjeev. True.

[@babareddy44]

How do we know which contributes least, help?

[@arslanshahid3454]

@@babareddy44 from R2, F- value or p- value?

[@beautyisinmind2163]

@@babareddy44 you can use random forest model to see the significance of feature that contribute the most

[@manavgora]

great, easily understandable

[@ChenLiangrui]

awesome video! very clear and beginner friendly, no broken train of thought, very problem-focused

[@UnfoldDataScience]

Glad you liked it!

[@DataScience111]

best explanation....keep the good work up.

[@UnfoldDataScience]

Thanks a lot.

[@roshinidhinesh5490]

Such a great explanation sir.. Thanks a lot!

[@UnfoldDataScience]

Thanks Roshini.

[@smegala3815]

Thank you sir... Best explanation

[@UnfoldDataScience]

Most welcome

[@umeshrawat8827]

To omit either X1 or X2, we can use PCA and remove the variable with low variance.

[@akashchatterjee7678]

Why with low variance

[@zakiaa7464]

You are a genius. Thanks

[@RamanKumar-ss2ro]

Great content.

[@UnfoldDataScience]

Thanks a lot.

[@allaboutstat1103]

thanks for clear explanation and God bless!

[@UnfoldDataScience]

Welcome.

[@Bididudy_]

Thank you for detailed explanation. I tried this concept from other channels but was bit difficult to get it. Your way of explaining terms is very simple and which helps to understand subject. Really glad that i visited your channel.👍

[@UnfoldDataScience]

Thanks Bala. Keep learning

[@ugwukelechi9476]

You are a great teacher! I learnt something new today.

[@UnfoldDataScience]

Pls share within data science groups

[@csprusty]

We can create and compare two models based on choosing each of the correlated explanatory variables one at a time and select the model having better R-squared value.

[@samruddhideshmukh5928]

Simple, Clear and Amazing explanation!!!
I think we can remove one of the columns seeing the p value. If p>0.05 then we fail to reject the Null hypothesis for that variable and thus that coefficient value will be equal to 0.Hence that variable will not contribute significantly.
Sir pls do make a video on how to use Ridge-Lasso regression to handle multicollinearity.

[@UnfoldDataScience]

Thanks Samruddhi,
Videos u asked:
ua-cam.com/video/7XvBwQeT9OI/v-deo.html
ua-cam.com/video/21TgKhy1GY4/v-deo.html

[@shivamthakur4079]

really loved sir what u said i can say that u have great idea of explaining concepts. i can blindly follow u sir

[@UnfoldDataScience]

Thanks Shivam.

[@prateeksachdeva1611]

excellent explanation

[@abdulhaseebshah9109]

Amazing Explanation Aman, I have a question that VIF and auxiliary regression both use to detect multicollinearity?

[@YourRandomVariable]

Hi Aman, What should we do when the constant term p-value is high? Mostly I see that people keep it without worrying about it. Could you please give an explanation for this?

[@harshadbobade2200]

Simple and to the point explaination 🤘

[@UnfoldDataScience]

Thanks Harshad.

[@suryadhakal3608]

Great.

[@UnfoldDataScience]

Thanks Surya.

[@muhammadaliabid5793]

Thankyou for excellent explanation. I have fews questions please:
1. I used Polynomial features method in sklearn and it significantly improved accuracy of my linear regression prediction model, but i found that the newly created features are correlated with the existing features since i created square and cubes! I understand as per your explanation that it will lead to multicollinearity problem! So i understand that the coefficients are not the true picture, However can i use this type of model for predictions?
2. What would you suggest the threshold correlation value for multicollinearity?
Thanks

[@shadow82000]

If X1, X2 have high correlation, can I choose to drop the X with lower correlation to Y? Based on the correlation matrix

[@UnfoldDataScience]

Yes Right.

[@shadow82000]

@@UnfoldDataScience Thank you kind sir. High quality content as always!

[@carlmemes9763]

👍❤️

[@bezagetnigatu1173]

Thank you!

[@UnfoldDataScience]

Welcome.

[@shanmukhchandrayama8508]

Aman, Your videos are great. But there are many videos which have some connection with other, so can you please make a video in which you can say which order to follow the playlists to learn the machine learning from basics. It would be really helpful😅

[@UnfoldDataScience]

Noted. Thank you for suggesting.

[@nurlanimanov9503]

Hello sir, After reading the comments I saw the answer to your question. They said we have to remove the one which has less correlation coefficient with the target variable due to the correlation matrix. It confused me at one point, Can we say that the coefficients in front of each feature that we get after running the regression model indicate us impact of each feature on the target? So, I mean can I take these coefficients when I decide which feature I have to remove bw two correlated features instead of taking correlation matrix value with the target variable? Can we say that the coefficients in front of each feature actually say the same thing as the value in the correlation matrix with the target variable in this context?

[@hakimandishmand1068]

Good and perfect

[@UnfoldDataScience]

Thank you

[@arshiyasaba2259]

If value is less then thresholds value 0.5/0.7 as per the reference suggests. Then we can remove those values

[@faozanindresputra3096]

is multicollinearity will be problem too in correlations? just focus on getting which variables that correlate, not focus on regression. like in PCA

[@nivednambiar6845]

Hi Aman, hope you are doing well !
I want to ask one thing, what you are mentioning regression models is related to linear models right not the tree based regression models am i correct ?
does multicolinearity effects the tree based models ?

[@ethiodiversity-1184]

great explanation

[@UnfoldDataScience]

Thank you 🙂

[@bhavanichatrathi7435]

Hi Aman it's very good explanation...please do video on penalised regression like lasso ridge and elastic..too much of mathematics into those please explain in simple way Thank you

[@UnfoldDataScience]

Thanks Bhavani, Sure will do,

[@shafeeqaabdussalam6195]

Thank you

[@UnfoldDataScience]

Thanks again.

[@sudheeshe1384]

You always rocks :)

[@UnfoldDataScience]

Thanks for watching Sudheesh.

[@mariapramiladcosta1972]

Sir if the there are 3 predictors and one dependent variable. all the three independent variables are highly correlated then which type of regression model can be used. multiple regression can not be used rt?can we use the linear regression? can the tolerance of .1 and the VIF less than 10 not a good enough to indicate that there is no multicollinearity?
for your question i think the one with weak correlated one to be removed

[@user-hy8me5cz2l]

Sr ap ko js ne jo answr dia he sb ka answr correct he ap sb ko yes bol rhen hn

[@UnfoldDataScience]

Moslty answers are correct only.

[@ShubhamSharma-zb9uh]

09:11 The Data which More Coefficient Value that we have to consider for analysis.

[@sandipansarkar9211]

finished watching

[@ameerrace2284]

Great video. Please create video on python implementation of Lasso and ridge regression

[@UnfoldDataScience]

Thanks Ameer. Sure!!

[@ashulohar8948]

Please please make a vedio how to select drivers in linear regression which drive the sales

[@bijaynayak6473]

which one will eliminate ? VIF of each features set the threshold >5

[@sudhirnanaware1944]

Hi Aman,
As per my knowledge we can use VIF (Variation Inflation Factor) function, heatmap,Corr() function to remove the multicoliniarity. Please confirm another techniques

[@UnfoldDataScience]

Yes Sushir, apart from some other regression techniques can be used.

[@sudhirnanaware1944]

Thanks Aman, may I know the regression techniques to remove multicoliniarity. so I will definitely learn this and it will helpful for me.

[@anmolpardeshi3138]

regarding the question- which variable to remove out of a set of highly correlated variables? Can this be answered by PCA (principal component analysis)? or will the PCA weight them the same because they are highly correlated?

[@UnfoldDataScience]

Hi Anmol , not in terms of pca, generally I asked.

[@nurlanimanov9503]

Hello sir! Firstly thank you for the video!
I have 2 questions if you answer I will be glad:
1) Can we say that we don't need to be concerned about correlated features in for example decision tree-based models? I mean do we need this concept only in linear-based models?
2) Don't we need to touch correlated features when we use Lasso or Ridge regression is that true? Will the model do that by itself in that case? Don't we need to touch?

[@UnfoldDataScience]

1. This is a problem with regression based models where coefficients come into picture.
2.still you need to take care.

[@hemanthkumar42]

@@UnfoldDataScience from you first answer, then why multicollinearity is not a problem in neural network? Pls make a video regarding this sir...

[@saurabhagrawal9874]

@@hemanthkumar42 Note that multicollinearity does not affect prediction accuracy of the linear regression ,it only make the interpretation harder in the linear regression and mostly for interpretation we go to linear regression and when we go to neural network we already know its type of blackbox and we dont want to interpret ,but want good prediction results ,thats why we dont bother about multicollinearity in neural network

[@sidrahms7458]

Awesome explanation, I have a question: if I have nominal,ordinal and continuous variables how can I find multicollinearity among them?

[@UnfoldDataScience]

Hi Sidrah, answered.

[@sidrahms7458]

I can't find your answer, I understand that we should use vif for continuous variables but what if I need to see correlation among all ordinal, numeric and nominal?

[@AMVSAGOs]

Great Explanation...
At 7.50 you said "that's why we should not have multicollinearity in regression" . So, Is it okay if we have multicollinearity in classification?? Could you please make it clear..

[@UnfoldDataScience]

When I say, it means regression family of Algorithms. Logistic regression also.

[@AMVSAGOs]

@@UnfoldDataScience Thank you Aman Sir

[@prateeksachdeva1611]

we will drop that feature from the model whose correlation with the dependent variable is lesser as compared to the other one

[@beautyisinmind2163]

can we remove highly negatively correlated features also or not? someone reply, please

[@trushnamayeenanda5431]

The independent variable with higher correlation among the similar factors should be removed

[@KumarHemjeet]

Remove that feature which is in less correlation with target.

[@akhileshgandhe5934]

Hi Aman, I have 9 categorical and 6 numerical columns and it's a regression problem.
So I can find the correlation between numerical using correlation heatmap but how to find the relation between categorical..??
Can I use chi square test..??
If I use I am getting all 9 categorical are dependent on each other. So what should be my next step..??
Please guide me.
Thanks

[@UnfoldDataScience]

Yes, chi square can be used, I have a dedicated video for the same topic.

[@RAJANKUMAR-mi1ib]

Hi...Thanks for the nice explaination. Have a question that is multicollinearity a problem for linear regression only? if not then how its a problem for non-linear regression?

[@UnfoldDataScience]

For regression based models like linear/logistic etc

[@kar2194]

Sorry so it means when there is multicollinearity for example x2 and x3, so if I increase x2, x3 will automatically increased? Great video by the way!

[@UnfoldDataScience]

internally at some level yes. tq

[@atomicbreath4360]

Sir can given some ideas on how to know which type of ml models is affected by multicollinearity?

[@UnfoldDataScience]

Regression based model

[@jaheerkalanthar816]

I think which variable highly CO relate with target variable

[@sriadityab4794]

Should we need to remove multicollinearity while building time series model?

[@UnfoldDataScience]

Not necessarily.

[@sharadpkumar]

Hi Aman, nice work, keep it up.....i have a doubt that why normal distribution is so important? why we need our independent variable should show normal distribution for a good model? i am not finding a satisfying answer. can you please help?

[@UnfoldDataScience]

Hi Sharad, in simple language, its easy for the model to learn pattern if you give examples from a large set of range.(That is your normal distribution).
Take a example below:
Predict salary of an individual(Y - target) based on his/her expense(X variable)
Scenario 1 - in your training set you have Y as - 10LPA, 15LPA,20LPA, like that, here model wont be able to learn the pattern for 3LPA guys, may be there is difference is income/expense pattern for junior guys.
Scenario 2 - You give many values of Y from all over like 2LPA, 4LPS,5LPA,100LPA, all values like they are normally distributed.
Here its easy for model to learn pattern as it sees a range of values and the resulting model will be more reliable.
Hope its clear now.

[@sharadpkumar]

@@UnfoldDataScience thanks for clarification . Does a huge dataset always show normal distribution?

[@UnfoldDataScience]

No, not always...it depends on data

[@karthikganesh4679]

Sir plz do the video for post pruning decision tree

[@UnfoldDataScience]

ok Karthik

[@kunalchakraborty3037]

My question..
1. Is multicollinearity a concern for predictive modeling. I mean the prediction is altered by neglecting this phenomenon or not.
2. In case of GAM do we have to worry about multicollinearity.
3. How collinearity inflates the variation.

[@UnfoldDataScience]

Thanks Kunal for asking it. Answer to first question is prediction will not be impacted more however eoefficoents will be impacted.
2nd and 3rd, I will. Cover in other video

[@kunalchakraborty3037]

@@UnfoldDataScience thanks 👍. Really appreciate your videos.

[@user-hy8me5cz2l]

Thnks sr g I think uncecessary variable remove

[@UnfoldDataScience]

Yes True.

[@khoaanh7375]

this shit is pure gold

[@squadgang1678]

I will find the correlation between x1 and y and x2 and y individually and see which one is lesser the one with lesser correlation i will delete it

[@omkarlokhande3692]

Sir what to do if the multi collinearity is affecting the binary classification problem

[@UnfoldDataScience]

many ways to take care of it. I have discussed in classification videos.

[@datafuturelab_ssb4433]

Remove the variable which have low impact on target variable...
Sir I hv 2 question
1. If there is multicollinearity in Classification problem. How to handle that
2. What is VIF & how standardization done
3. Can we use standard scaler in regression problem

[@UnfoldDataScience]

There are three questions, I will cover them in separate video. Thanks for asking.

[@rohitnalage6366]

Sir please explain Lasso and ridge if you made it,link pl.

[@UnfoldDataScience]

ua-cam.com/video/7XvBwQeT9OI/v-deo.html
ua-cam.com/video/21TgKhy1GY4/v-deo.html

[@hemanthkumar42]

Is multicollinearity is the problem for neural network?

[@UnfoldDataScience]

Not always.

[@ahmad3823]

at least two variables!

[@salajmondal3437]

Should I check multicolinearty for classification problem?

[@UnfoldDataScience]

For logistic regression - yes.

[@salajmondal3437]

@@UnfoldDataScience Is it necessary to check multicollinearity between categorical features or numerical and categorical features??

[@rafibasha4145]

Multicolinearity is problem in classification as well right .@3:57

[@UnfoldDataScience]

Yes, if it's a linear model like logistics regression.

[@sreejadas4417]

I want to be a data analyst but I want sequential courses from you please guide

[@UnfoldDataScience]

www.unfolddatascience.com

[@anirudhchandnani9917]

Hi Aman,
Could you please make a detailed video explaining the difference between Gradient Boost, AdaBoost and ExtremeGradientBoosting?
Why is AdaBoost called adaptive? Is it only because it edits the weights of the misclassified instances? XGBoost and GradientBoost also are adaptive in that way, arent they?
Also, why are XGBoost and Gboost more robust to outliers than AdaBoost despite all of them having a term of log in their loss functions?
Would really appreciate your reply.
Thanks

[@sujithreddy1599]

It depends on feature importance. the feature with less importance will be dropped.
correct me if am wrong :0

[@UnfoldDataScience]

Correct Sujith

[@naziakhatoon3058]

Jo less Cor related ho usko remove karna hai

[@UnfoldDataScience]

True. "less correlated with target" :)

[@squadgang1678]

Is Machine learning better than deep learning or deep learning better than machine learning

[@UnfoldDataScience]

Depends on problem statement, data availability, Infra availability etc, can't say one is better then other

[@squadgang1678]

@@UnfoldDataScience oh ok got it ✌️

[@tesfayesime9434]

Neither x1 or x2