Python Pandas Tutorial 5: Handle Missing Data: fillna, dropna, interpolate
Вставка
- Опубліковано 21 лип 2024
- In this tutorial we'll learn how to handle missing data in pandas using fillna, interpolate and dropna methods. You can fill missing values using a value or list of values or use one of the interpolation methods.
Topics that are covered in this Python Pandas Video:
0:00 Introduction
2:30 Convert string column into the date type
3:15 Use date as an index of dataframe usine set_index() method
4:10 Use fillna() method in dataframe
7:35 Use fillna(method="ffill") method in dataframe
8:57 Use fillna(method="bfill") method in dataframe
9:56 "axis" parameter in fillna() method in dataframe
11:18 "limit" parameter in fillna() method in dataframe
13:46 interpolate() to do interpolation in dataframe
15:34 interpolate() method "time"
16:50 dropna() method Drop all the rows which has "na" in dataframe
17:50 "how" parameter in dropna() method
18:33 "thresh" parameter in dropna() method
Code link: github.com/codebasics/py/tree...
Do you want to learn technology from me? Check codebasics.io/?... for my affordable video courses.
Popular Playlist:
Complete python course: • Python 3 Programming T...
Data science course: • Data Science Full Cour...
Machine learning tutorials: • Machine Learning Tutor...
Pandas tutorials: • Pandas Tutorial (Data ...
Git github tutorials: • Git/Github Tutorial
Matplotlib course: • Matplotlib tutorial
Data structures course: • Data Structures And Al...
Data Science Project - Real Estate Price Prediction: • Machine Learning & Dat...
To download csv and code for all tutorials: go to github.com/codebasics/py, click on a green button to clone or download the entire repository and then go to relevant folder to get access to that specific file.
🌎 My Website For Video Courses: codebasics.io/?...
Need help building software or data analytics and AI solutions? My company www.atliq.com/ can help. Click on the Contact button on that website.
#️⃣ Social Media #️⃣
🔗 Discord: / discord
📸 Dhaval's Personal Instagram: / dhavalsays
📸 Instagram: / codebasicshub
🔊 Facebook: / codebasicshub
📝 Linkedin (Personal): / dhavalsays
📝 Linkedin (Codebasics): / codebasics
📱 Twitter: / codebasicshub
🔗 Patreon: www.patreon.com/codebasics?fa...
Check out our premium machine learning course with 2 Industry projects: codebasics.io/courses/machine-learning-for-data-science-beginners-to-advanced
This Man its Amazing ❤❤👍👍👌👌
@codebasics
CAN we replace missing value using Na_values like we did in pervious videos cleaning messy data ?
It's been 5 years since you posted this video but still, we can't find a better video than this to understand the concepts. Not only this but your complete playlist is GEM. Thanks a ton.
I just don't know how exactly can I say THANK YOU enough to convey my thank you to you THANK YOU, THANK YOU, THANK YOU for all the tutorials in ALL your playlists, so awesome and helpful.
ALL ARE SUPER EXCELLENT.
Thanks very much for this! I'm doing an edX course and I was doing fine until this very topic came up. Your video perfectly cleared everything up which is a great relief!
Step by step roadmap to learn data science in 6 months: ua-cam.com/video/H4YcqULY1-Q/v-deo.html
Machine learning tutorials with exercises:
ua-cam.com/video/gmvvaobm7eQ/v-deo.html
Step by step roadmap to learn data science in 6 months: ua-cam.com/video/H4YcqULY1-Q/v-deo.html
5 FREE data science projects for your resume with code: ua-cam.com/video/957fQCm5aDo/v-deo.html
Best course that I can find after 2 months research, you are the only guy who know why we need pandas to do excel operations
I think your playlist has more rich content than paid courses
100% true....
I too agree.
I sure has
100%
I love the way you used the graph/visual to explain the different interpolation techniques!
What an amazing presentation!! You did a fantastic job there! Very professional approach! You just made me a subscriber with that! :) Big thanks!
great tutorial. A very comprehensive explanation of the topic. I have a good command of pandas but I still learnt some new tricks I didn't know.
This video has convinced me to buy some of your courses. Serious and in depth approach.
Totally recommended!
Excellent presentation of these tools. I like the practical way you have demonstrated HOW to use the tools. Thanks.
Finally! Thank you so much I was trying figure out how to drop only the columns that have NaN. I had done it by hand it took me so long Pandas is amazing.
LIFE SAVER BRO... THANKS A LOT
THIS IS WHAT I EXACTLY WANTED
Good presentation in simple words, easy to understand. Great job Sir!
the threshold parameter is so useful! never heard about this before! thank you!
Thanks for your great pandas tutorials. Just what I was looking for. You explain it very clear. Thanks again :)
Very well organized and intuitive tutorial. Great job. Thanks a lot
OMG this is exactly what I was looking 4!
Thanks so much man u just earnt yourself a new sub
With this kind of tutorials I can say to myself: I can do it!
This tutorial should be awarded as the best tutorial, understood all the concepts ............thankyou so much.
Thankyou so much. I love how you have given all ipynb files with such good markdowns and details. Love you for it. Youre a life saver...got my Python exam exam
Awesome video. Loved all the concepts and mostly your teaching style. :) Thank you, sir.
This guy is a great teacher. Very simple way to teach. Very basics. great dude. Thanks for sharing you skill for free.
I am glad you liked it pramod 😊
Simple, lucid explanations with examples. Wonderful!!!
Vaibhav, I am glad you liked it
completeness of the material is commendable. keep it up thanks a lot :D
Interpolate will definitely boost my kaggle score! Thanks so Much!
Very well explained sir. I appreciate that you suggested those little tricks rather than just sticking to the concept.
Thanks Himani, glad you liked it. 😀
You need more subscribers buddy !
Simply awesome
clean and simple, yet effective, understandable and efficient. Thanks !
Glad it was helpful!
Thanks! Great tutorial! Just finished python and now continue right away with pandas...still am enjoying and learning...
Glad it helped!
The only video that makes sense to me :')
Thanks a lot for this tutorial!
As always very well and understandably explained! Many thanks for that!
Glad it was helpful!
the perfect example had taken to explaining all the scenarios and nicely presented
How to learn coding for beginners | Learn coding for free: ua-cam.com/video/CptrlyD0LJ8/v-deo.html
Simple and clear demonstrations coupled with simple and clear explanations. Very well presented!!
kudos to you sir!!
Glad it was helpful!
This one of the best explain step by step tutorial,
Well explanation than most paid courses. Thanks a lot.
You have done an amazing job. Thanks a lot!
Awesome method of teaching code. it is so clear to the beginners, you even tell minute things like what shortcuts you are using and how to find them. Really boss you deserve a respectful SALUTE. I am so satisfied with your teaching method that i am writing this pausing the video in between.
🙏👍👍👍
Sir, you are top! Keep up the good work!
When you googled, I saw the pangolin valentine Google doodle. I remember that vividly, as I played that game in my old office in 2017. I can't believe it's been 7 years and wish I had known these videos 7 years back.
Nice one, I watched it second time and finds very helpful for EDA starters.
I have been learning a lot from watching your videos. Than you so much.
It is a real gem🤩 for those who wants to learn pandas
man I hope you are lecturing at the best university where you are. you have such a pedagogical approach.
Thank you for the great explanation!!!!
thank you sir for clear explanations. one small doubt: after setting up day as index value, there is a gap in the first row, why is it so and how to avoid that.
Superb Explanation !
Thank you :)
Great video. Thanks!
Great job. Congrats!!
you are a magician sir...thanks a lot
Thank you, you got yourself a new subscriber.
🔥🔥🔥🔥🙏Thank you sir for these all videos ! Really helpful .
Really a life saver bro!! Thanks a ton!!
Beautifully explained! :)
Your tutorials are great. Thanks so much. While executing the df in the jupyter notebook, why I can't see the table outline as I can see the video?
Thanks a lot it was helpful !!
I was using fill('np.nan') that changed the dtype to 'object' from 'float64' that did not allow me to interpolate. I was able to pick-up on that because of your video! Now, I have to try and use linear interpolation.
really nice videos and much better than paid courses
Thank you sir. This video is fantastic!
This is a very useful video! Thank you🙂
Just wow.... Thank u so much!!!!
Thank you for step by step explanation. Good job!
😊👍
Thank you so much! So helpful!
Glad it was helpful!
perfect job. keep going
This is much better than Coursera Intro to Data Science course
Best clearing understand video
Great tutorial. Thank you.
Hi,
Excellent examples and explanation.
I am facing an issue , after using dictionary with fillna method for replacing 0 values in 'event' column , the df still has only 0s.
Krish
Good One bro keep it up.
TBH, this is better than the paid course contents like datacamp
T
hank you very much, it was really helpful.
Sir thank u so much...
With one dataset i am facing problems but by seeing u video i got to know sir ...
Thanksss sir !
I am glad you liked it
Can you please make a series of videos for the datetime and os libraries?
Thanks for your awesome vids!
what a great video
thanks a lot!!
Excellent work!!
Thank you! Cheers!
Excellent video!! Thank you!! 😊👍👍
Irene, I am glad you liked it
Very very helpful.
Very good explanation! Thank you very much.
Glad it was helpful!
wait for it....................Awsome..:)
thank you sir, really nice explanation.
You're the best!
This is the best rich in content best for free you are a man of god
glad you liked it Vinay
brother what kind of method do you use to record this video so clear and professional please let me know i am trying to videos for data analysis. Thanks.
Superb!!!
Have a few questions:
(1) your data is already sorted by the date column, hence using just the interpolate method makes sense. But if my data is not sorted, should I first create a data frame by sorting on the dates and then use that data frame as an input to the interpolate method?
(2) when you use interpolate with "time", how does the program know that it has to use the date column for the time? What if I had date1 like you do and had another column date2 with some other dates, how would the program know that it has do a time-based interpolation on date1 and not date2?
(2) can interpolation be done for specific columns only? What if I wanted to do interpolate for temperature and forward fill for windspeed? what would be the syntax like?
Thank you! Great video!
Thank you very much it is very efficient to me , Really helpful !!
👍🤓
really well done!
Very nicely explained
thanks for all your videos...
Glad it was helpful!
Assuming from your data that you have all the events, how can you fill in the temperature based on the event, eg if the event was "sunny" fill in 32. etc ?
Wow ,learned a lot to handle datasets. Thank you Sir
You're most welcome
Very helpful. Thank you
Glad it was helpful!
You are a smooth talking pro
Good job..thank u . My request is make some videos for "seaborn" it will be more useful..
the perfect tutorial thanks a lot
Great! Thank you!
At 20:20 , you passed dt in DatetimeIndex() to make it DateIndex type. But when we will create a date range from pd.date_range it itself is DatetimeIndex type and we can skip the pd.DatetimeIndex function part.
Lovely explanation
BTW - i have also subscribed. Thank you once again.
Wow. Thank you for uploading series on pandas. Currently going through each and every video and it seems to be a better video.
Could you please help me to understand below scenario -
16:45 - Lets assume, we have two dates...Eg. Invoice Pay date, Invoice rec date..is it possible to specify particular date for guessing using interpolate ?