Handle Categorical features using Python

Here is a video which provides you the detailed explanation of how we can handle the categorical features using Python. We will basically be applying the get_dummies() function from the pandas library,
#HandlingCategoricalfeatures
Github url: github.com/krishnaik06/Catego...
You can buy my book in Finance from amazon
amazon url: www.amazon.in/Hands-Python-Fi...

[@HrishikeshShinde495]

This is the simplest way of encoding the categorical features. Thanks man!!

[@timothythampy7202]

Thank you so much, sir! You are the best teacher

[@amylock8275]

Exactly what I was looking for! Thank you

[@sreedharsree361]

Thanks krish for this video ..
I have a doubt, at last part of the video .. while converting from categorical feature to numerical feature 2001 pincode represented at one instance as 1 and at other instance it is represented as 0 .. on what basis we represented like this ?

[@UnstoppableBird]

thank you so much, this is actually clearer than the stupid class I enrolled earlier

[@simanchalpatnaik2566]

Thanks for the video Krish.
When I ran the "df" command after concating, why all the values of Florida & New York comes as "NaN" ?

[@ishmeetsinghrayat]

you explain like pro bro.....

[@sahanjayarathne3472]

Thank you so much for the video

[@MrKB_SSJ2]

Thank you so much... It was so easy...

[@ijeffking]

Thank you Krish.

[@slowhanduchiha]

One Question Sir. I was working on a classification dataset. My out put variable is also categorical in nature . I applied OHE and later when i saw the heatmap it made no sense because the columns were bit blank. Correct me where i am wrong here

[@louerleseigneur4532]

Thanks Krish

[@nikhilshingadiya7798]

what happen if we have all independent variable as categorical i.e. movies data set country origin,movie_type,director now i want to predict the imdb missing data how can i handle those categorical variable

[@somnathbanerjee2057]

Hi Krish, could you please guide me how can I handle text column for a regression problem. It's not about encoding categorical features. But what I am looking for is---extracting some meaningful information from the existing column containing text data using string manipulation method from regex...Please recommend me an effective way of doing this.

[@arjyabasu1311]

Sir, please upload a video on how to perform mean encoding !!

[@51kaushal]

Kris, but what if we have a regression problem then we would not have output as 0/1, then how do we encode the categorical features like pincode, do we use frequency/count encoding in there??

[@priyak1008]

How to apply onehot encoding if we have categorical data in Y (dependent column).

[@inderpreetkaur1441]

I want to create box plot for categorical variable (like subscribed: yes/no)
Firstly I wrote the code: train= pd.get_dummies(train['subscribed'], drop_first=1)
And then for creating box plot: train['subscribed'].plot.box()
But this will show error as- keyword: 'subscribed'
Please let me know my mistake.

[@programsolve3053]

Well explained🎉🎉

[@SuperJg007]

what to do if there is mixed data, continuous and categorical?

[@vijaynale7893]

Thanks bro

[@aakashsinghrawat3313]

can someone please give me link of solved example using target encoding, mean encoding like above

[@satishakuthota6290]

please make video on visuvalistion using matplotlib

[@chaithanyaravi8400]

Hi Krish,
suppose in the case if we have 8 categorical names at that time it will generate 8 new columns?

[@aakashsinghrawat3313]

yes

[@aanniirr100]

how can we save the count of a particular category obtained to be used later in any calculation

[@ranoyavanniysingam3390]

if we have more than 5 categorical-feautures column, what to do for that? for example -- country, age group like this?

[@nashrakhan5361]

getting key error for the column which I used for categorical data, please help

[@dilipgawade9686]

Hi Krish,
can you show how to convert categorical variable to numeric variable through coding ?

[@swaruppanda2842]

At 17:08 u made it clear for 2001 as 1 as output will be 2/3=0.6 what about 2001 as 0 as output?

[@aakashsinghrawat3313]

actually the idea is getting mean of 2001, suppose 2001 has value 1 and 0 obvious mean will be 0.5 for both

[@surbhisarawagi1821]

If we have large number of categorical variables say 21, then if we use get dummies we'll have large number of columns so how to deal such a case?

[@gurjotsingh752]

are u able to find, how to cater your problem ie 21 categorical variables?

[@aakashsinghrawat3313]

i think, we can use feature_selection like SelectKBest to get top k(any N no. upto total columns) which means these new features have strong relationship with target

[@gurjotsingh752]

@@aakashsinghrawat3313 yes, your approach is good. I also find one more approach. Here we will replace categorical values with their number of count. Eg) We have 29 states. and suppose we have 10K records and Delhi has come 500 times, Karnataka come 900 times. So, i will replace delhi by 500 and Karntaka by 900.

[@aakashsinghrawat3313]

@@gurjotsingh752 isn't it inefficient to labelling 500,900 to categorical feature? This method might be good to ordinal features.

[@parakhsrivastava7743]

I have a doubt...
When dealing with categorical values having many classes, you took all 2001's and find out the probability where O/P is 1.
Suppose, that is coming 0.6(as in video). Now you are replacing all 2001's with 0.6, no matter O/P is 0 or 1... WHY?
Should we not replace 2001's by 0.6 only if O/P is 1, else replace it with 0.4?
Thanks for the video btw!

[@swinal9710]

The O/p column comes from where can you explain me that?

[@theeagleseye4989]

Same doubt here

[@sahilkamboj747]

@Premjith Augustine what if the output is not a classification variable. Target variable can be Continuous like Price, Fees,Profit

[@aakashsinghrawat3313]

that 0.6 is mean for 0 as well as to 1

[@ankita684]

Hi Krish.. I wrote exactly the same codes simultaneously to practice, but my score came out to be -5.667 (I got a negative value) whereas you got 0.9304. I am not able to understand why am I not getting the same value. Please explain.

[@krishnaik06]

Just set the seed once

[@vipinamar8323]

What is the last encoding type called?

[@alimisumanthkumar2769]

mean encoding