wow I was so confused about column transformer and why everyone is using that. I was so confused. People usually include that in the encoding videos without any explanation. You are the first person to explain it separately in your series. I am amazed. Thank you Nitish, I will remember you throughout my journey.
one quick question sir we can do this isntead of these sklearn function this way we can arrange and give orders to our data and its fast too easy to understand instant output df.education[df['education'] == 'School'] = 0 df.education[df['education'] == 'UG'] = 1 df.education[df['education'] == 'PG'] = 2 df.review[df['review'] == 'Poor'] = 0 df.review[df['review'] == 'Average'] = 1 df.review[df['review'] == 'Good'] = 2 df.purchased[df['purchased'] == 'Yes'] = 1 df.purchased[df['purchased'] == 'No'] = 0
Thank you for the great video, sir. I wanted to ask wouldn't it be better to encode the data before splitting it? This way we don't have to transform the train and test sets individually.
oe = OrdinalEncoder(categories=[['Poor','Average','Good'],['School','UG','PG']]) when we have this already defined, so why we need to do oe.fit(X_train) I mean, how will it acutally help when all the calculation was done on oe in first line?
"Whenever we are doing a project, instead of train_test_split, we should always do k-fold cross validation." Sir, is my thinking correct ?? If wrong, please rectify me.
Doubt 1. Can't we use ordinal encoding and label encoding before X train and Xtest split???? It would have been an easy task to do the encoding before its split. 2. Cant we use replace function of pandas like replace yes and no to 1 and 0, and replace poor , avg and good to some value like 0, 1 2
Can somebody explain me what order should be followed while doing any ML Project. Like whether feature scaling should be applied first or encoding categorical data should be done etc. Would be extremely grateful if someone can please clarify. Thanx.
Why do we even do Label Encoding on target column . I mean that is essentially just 0s and 1s right ? So why we just can’t create dummies ? What’s the logic behind using Label Encoding here ?
Does it matter if the output column is ordinal or nominal before applying label encoding? How to do encoding of categorical feature column with high cardinality? Please help me
ValueError: Shape mismatch: if categories is an array, it has to be of shape (n_features,). Im getting this error after I run oe.fit(X_train) can any one help me to solve this problem
zy sub samjh aey gia but need a video after Encoding us py Analysis kesi kry ge aur fine result ko kesi again male female or yes and no mi change kry gy, after 2 or 3 video bad uni video py practical project video bi bny, problem zy ha transform data ho gye ab usi py analysis kesi kry? final output kesi pta chly zy male ha?
Hello Sir, Thank you for the session. Can we extend concept of ordinal encoding on numeric column such as Age? Like in your dataset at 11.45, the values of column Age are: - 98, 16, 53, 69, 77. With more number of records we will have more number of distinct values under the column and at maximum we can have 100 values. Thus, if we classify the numeric values into categories will that not help to make our data analysis and ML model better? For example: We can have a category: Teenager for all Age values from 13 to 19., College students: 20 to 23, Young professionals: 24 to 30, Mid age: 31 to 65 and Senior citizen: 66 to 99. And then finally apply Ordinal encoding on these categories since now we will have order among the classified values. It would be very helpful sir to seek your views on the above. Thank you
Sir what will happen if the output is categorical data but nominal, should I apply Label Encoding there also...?? Actually I want to say that If the output data is categorical, may be that Nominal / Ordinal, in both of case should I apply Label Encoding....?? Thank you for this video🙂
I have one doubt during train test split we are fitting on training data while transforming both training and testing suppose this was standardization case then if we fit of train data we get mean and variance of that how can we transform test data using this train data mean and var . I just mean test data should be independent of train data there shouldnt be any type of relationship between them to prevent data leakage . So we must calculate seperate mean and variance for both train and test and fit tranform individually? Pls clarify
when ordinalencoding or onehotencoding done using coloumn transformer the output array columns index are changed i mean encoding done on 5th coloumn after tranformation it is appering to be 1st in array after transformation any solution
The encoders have additional parameters that you can set to decide what to do incase unseen categories are encountered in the test data. You can check the documentation for more details
Q. If we have a big dataset. let's say Housing_price.csv(from Kaggle), then how would I know which column has ordinal data? is there any API to check? @CampusX #CampusX
I purchased a data science course with around 50k fees but even that they are not teaching this level education. You are such fabulous person.. 👍
50K pani main gaye chapak
bhai 50k ka koi course khardta hai kya koi
wow I was so confused about column transformer and why everyone is using that. I was so confused. People usually include that in the encoding videos without any explanation.
You are the first person to explain it separately in your series. I am amazed. Thank you Nitish, I will remember you throughout my journey.
best teacher on youtube respect to data science
Never knew about Label encoder and Ordinal encoder, I used to apply label encoder on input features, thanks for this hidden insight Nitish Sir.
Sir kya bolo...just itna hi
U r doing great job...with quality conceptual clearity...
Best explanation ever 🙏🏻
one quick question sir we can do this isntead of these sklearn function this way we can arrange and give orders to our data and its fast too easy to understand instant output
df.education[df['education'] == 'School'] = 0
df.education[df['education'] == 'UG'] = 1
df.education[df['education'] == 'PG'] = 2
df.review[df['review'] == 'Poor'] = 0
df.review[df['review'] == 'Average'] = 1
df.review[df['review'] == 'Good'] = 2
df.purchased[df['purchased'] == 'Yes'] = 1
df.purchased[df['purchased'] == 'No'] = 0
Instead of doing this you can use pandas Map function it'll do same thing
But when so many categories then instead of this sklearn do fast and easily
This is a great channel. I'm glad I found it. Thank you so much, Sir, for making this so simple.
Well explained. This really helped me to improve my understanding of ML. Thank you sir.
Thanku so much for clearing concepts of encoding technique with example. Very helpful n informative video.
Course Started : ML
Lecture-01: 14/08/2024
Lecture-02: 14/08/2024
Lecture-03: 14/08/2024
Lecture-04: 14/08/2024
Lecture-05: 14/08/2024
Lecture-06: 15/08/2024
Lecture-07: 15/08/2024
Lecture-08: 15/08/2024
Lecture-09: 15/08/2024
Lecture-10: 15/08/2024
Lecture-11: 16/08/2024
Lecture-12: 16/08/2024
Lecture-13: 17/08/2024
Lecture-14: 17/08/2024
Lecture-15: 18/08/2024
Lecture-16: 19/08/2024
Lecture-17: 20/08/2024
Lecture-18: 20/08/2024
Lecture-19: 21/08/2024
Lecture-20: 21/08/2024
Lecture-21: 22/08/2024
Lecture-22: 22/08/2024
Lecture-23: 23/08/2024
Lecture-24: 23/08/2024
Lecture-25: 24/08/2024
Lecture-26: 24/08/2024
The way you teach is amazing sir , i was trying for months to learn ML , finally i am glad that i found you .
Thank you for the great video, sir.
I wanted to ask wouldn't it be better to encode the data before splitting it? This way we don't have to transform the train and test sets individually.
explicit and to-the-point 👍
You And Krish Naik Sir are the Brahma And Bishnu Of Data Science.
Right sir
M first time comment kar rha ,, Kosi channel p because info is really precious ,,, quality bole to Nitish sir
sir hamane classes karake bhi hamko yeh concept nahi sikhaee...thank you sir
oe = OrdinalEncoder(categories=[['Poor','Average','Good'],['School','UG','PG']])
when we have this already defined, so why we need to do oe.fit(X_train) I mean, how will it acutally help when all the calculation was done on oe in first line?
I dont know why this channel is not famous
you are great bro
bestest video content
Thanku so much for clearing encoding concepts. Very good explaination with example.
"Whenever we are doing a project, instead of train_test_split, we should always do k-fold cross validation." Sir, is my thinking correct ?? If wrong, please rectify me.
Yes, or some other form of cross valuation
Kabhi-2 main sochta hu agar aap nahi hote to hamara kya hota. Great explanation
The feeling is mutual sir
Sir, you are doing a great job
Best explanation ever
At 16:29 why didn't we convert our transformed numpy array to dataframe?
You are great efforts 👌 a appreciate you god bless ❤
Excellent explanation 👍
Doubt
1. Can't we use ordinal encoding and label encoding before X train and Xtest split????
It would have been an easy task to do the encoding before its split.
2. Cant we use replace function of pandas like replace yes and no to 1 and 0, and replace poor , avg and good to some value like 0, 1 2
sir plz make a video on high cardinality categorical value
Can somebody explain me what order should be followed while doing any ML Project. Like whether feature scaling should be applied first or encoding categorical data should be done etc. Would be extremely grateful if someone can please clarify. Thanx.
can we perfom label encoding and oridinal encoding before train test split
Thank you sir
You Are Really Great Sir
Why do we even do Label Encoding on target column . I mean that is essentially just 0s and 1s right ? So why we just can’t create dummies ? What’s the logic behind using Label Encoding here ?
finished watching and coding
Too good content sir it is helping me alot
It would be great if there is dataset link in aal the videos.
recommend these tutorials to aspiring data scientist
Thanks Sir for this Amazing Session
11:18 *Encoding to Categorical Features*
Super explanation sir❤
Best
why label encoding can't used to transform input columns ?
Hello sir, which lecture has the introduction to sk learn library?
In label encoding how would the object know whether to map 0 to 'NO ' and 1 to 'YES'. As we didnt specify.
Does it matter if the output column is ordinal or nominal before applying label encoding? How to do encoding of categorical feature column with high cardinality? Please help me
What if there are null values in education column and then how should we start the process?
ValueError: Shape mismatch: if categories is an array, it has to be of shape (n_features,). Im getting this error after I run oe.fit(X_train) can any one help me to solve this problem
Great Content
Thank You😀
maine socha ajj video nhi ayegi but thank you
zy sub samjh aey gia but need a video after Encoding us py Analysis kesi kry ge aur fine result ko kesi again male female or yes and no mi change kry gy, after 2 or 3 video bad uni video py practical project video bi bny, problem zy ha transform data ho gye ab usi py analysis kesi kry? final output kesi pta chly zy male ha?
Hello Sir,
Thank you for the session. Can we extend concept of ordinal encoding on numeric column such as Age?
Like in your dataset at 11.45, the values of column Age are: - 98, 16, 53, 69, 77.
With more number of records we will have more number of distinct values under the column and at maximum we can have 100 values.
Thus, if we classify the numeric values into categories will that not help to make our data analysis and ML model better?
For example: We can have a category: Teenager for all Age values from 13 to 19., College students: 20 to 23, Young professionals: 24 to 30, Mid age: 31 to 65 and Senior citizen: 66 to 99. And then finally apply Ordinal encoding on these categories since now we will have order among the classified values.
It would be very helpful sir to seek your views on the above.
Thank you
Sir what will happen if the output is categorical data but nominal, should I apply Label Encoding there also...?? Actually I want to say that If the output data is categorical, may be that Nominal / Ordinal, in both of case should I apply Label Encoding....??
Thank you for this video🙂
don't we have to convert the array output in dataframe after transformation is done
Not required
I think sir did in previous videos just be make us visualize the numpy array into the better form.
Nice explanation
I have one doubt during train test split we are fitting on training data while transforming both training and testing suppose this was standardization case then if we fit of train data we get mean and variance of that how can we transform test data using this train data mean and var . I just mean test data should be independent of train data there shouldnt be any type of relationship between them to prevent data leakage . So we must calculate seperate mean and variance for both train and test and fit tranform individually? Pls clarify
Sir if there are yes/no data in data then which encoding should be used?
Please clear my this doubt
Please make one vedio on neural architecture search
thank you so much
when ordinalencoding or onehotencoding done using coloumn transformer the output array columns index are changed i mean encoding done on 5th coloumn after tranformation it is appering to be 1st in array after transformation any solution
what to do if the number of features are high (ordinal ones)? I have 40 such features
loved it
Sir then which encoder we should apply on feature which are not ordinal
❤🔥🔥
day2-date:10/1/24
in ordinal encoding an error is raised: Shape mismatch: if categories is an array, it has to be of shape (n_features,)
can any one resolve please
I got he concept but all information are in array, do we need to convert them into DF and merge to proceed further ?
Instead you can use a transformer
sir csv file are not download
please help me urgent
11 20 to 12 10 - why column transformer
how to download dataset from your Github ,it is showing "raw file download" and not downloading
please help anyone
Copy the url and load directly in pandas
done
How to deal with the situation where there are unseen categories in the test data?
its always good to apply encoding without train test split .
The encoders have additional parameters that you can set to decide what to do incase unseen categories are encountered in the test data. You can check the documentation for more details
@@anjushac9307 will check the doc. Thanks!!
👾👾👾
bol sakte hai sir 3:02
kaggle pay nhi milra
Please help anyone
Jitni tareef ki Jae Kam h . ..
sir data set upload kar o
Engg branch should be the example of ordinal data 🤣
Q. If we have a big dataset. let's say Housing_price.csv(from Kaggle), then how would I know which column has ordinal data? is there any API to check? @CampusX #CampusX
Best explanation ever 🙏🏻
thank you sir