23:46 To the viewers who already have prior knowledge of TSA, can directly visit 23:46 to skip the fundamentals and get to the main content of imputations strategies
Wow! just awesome, I rarely comment on any UA-cam video but this series is a masterpiece, and I am looking forward to completing it. For those who are searching for a good time series playlist, this playlist is sufficient to learn from A to Z.
Thank you Satyam.. I will be going to algorithm only next month. This month is more for foundational concepts of timeseries which is key before getting into algorithm
Wonderful content sir! Loved the hands-on approach taught me how to handle a problem apart from theory! Thankyou, will be following the entire playlist and waiting for more!
Thank you for making such elaborate videos on such a topic where not enough content is present. This really helps. Just the last imputation of previous year has a mistake, the correct code would be below: aq_df_imp['TIME_PYEAR'] = aq_df_imp.apply(lambda x: aq_df.loc[x['year_month_day_hour'] + pd.offsets.DateOffset(years=-1)]['TEMP'] if pd.isna(x['TEMP']) else x['TEMP'],axis=1)
Lots of details and concepts discussed. Thanks a lot for sharing. Sir if you could share your GitHub repo for this codes in your description box, it would be a lot easy for us to replicate the codes, and to spend more time on consuming concepts and less time in re-writing coding.
Jaspreet - My git repo for this series is here - github.com/srivatsan88/End-to-End-Time-Series Maybe I need to go back and edit my video with individual links when I get time :)
@@AIEngineeringLife Thanks for sharing the link. This repo will help a lot. To ease out time taken to edit descriptions, we viewers will also drop this repo link in comment section of other videos in this playlist :) Keep sharing your knowledge and helping us.
great video, thank you very much. I believe the 'TEMP_ROLLING' correction should be: aq_df_imp['TEMP_ROLLING'] = aq_df_imp['TEMP'].fillna(aq_df_imp['TEMP'].rolling(3, min_periods=1).mean()) ...where the window size is 3 because it includes the NA value itself.
Actually instead of taking the previous year value of temperature we are taking the next year value as we apply "- pd.offsets.DateOffset(year=-1)", should be either "-+pd.offsets.DateOffset(year=-1)" or "- pd.offsets.DateOffset(year=1)"
Arun.. Yes that was a mistake I did. Basically must have tested the code properly :) . But for that small change other concepts are same. I have commented the right code in another comment as well
Hello Sir, Did you consciously impute 2015 Temperature missing values with 2016 values? You gave an offset of "-1" years and subracted it from 2015 index, resulting in 2016 values.
For the monthly sales data. If there is no sales in a particular month. How to handle this discontinuity?? Can we impute 0 there because there is no sales happened that time?
Hey, for the last code the substituting null with previous year values, m getting a timestamp error..I checked in a lot of places but couldn't find. Can u help me with it?
Dear, why the 1st NAN (2015-02-21 120000) + 2nd NAN (2015-02-21 140000) + 3rd NAN (2015-02-21200000) of TEMP_PREVY = [1.9, 4.9, 3] 31:20; while they are [3, 3.5, 1.2] respectively in 2014 30:02 ?
Randy.. Sorry, That was my mistake. A bug in code. DataOffset should be added and not subtracted in the function. Try below aq_df_imp['TEMP_PREVY']=aq_df_imp.apply(lambda x: aq_df.loc[x['year_month_day_hour'] + pd.offsets.DateOffset(years=-1)]['TEMP'] if pd.isna(x['TEMP']) else x['TEMP'], axis=1)
when I make this aq_df_imp['TEMP_PREVY']=aq_df_imp.apply(lambda x: aq_df.loc[x['year_month_day_hour'] + pd.offsets.DateOffset(years=-1)]['TEMP'] if pd.isna(x['TEMP']) else x['TEMP'], axis=1) i have error KeyError: Timestamp('2017-03-29 00:00:00')
Search for github srivatsan88 and in that this is available in end to end time series playlist. Let me know in case if you have trouble locating this file
--------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last) in ----> 1 import plotly.express as px 2 3 fig = px.line(delhi,x='date',y = 'pm25',title='pm25 with Slider') 4 5 fig.update_xaxes( ModuleNotFoundError: No module named 'plotly' I'm getting this error even though I installed 'pip install plotly'
Sorry for it. Sometimes recording plays against me while I try my best to fix it. Can you please watch without headset this video so noise is less. I am trying to fix it permanently
23:46
To the viewers who already have prior knowledge of TSA, can directly visit 23:46 to skip the fundamentals and get to the main content of imputations strategies
Wow! just awesome, I rarely comment on any UA-cam video but this series is a masterpiece, and I am looking forward to completing it. For those who are searching for a good time series playlist, this playlist is sufficient to learn from A to Z.
Excellent and clear video for manipulating missing values in time series data. Thanks a lot for such knowledge sharing !
Actively following your time series' series!!!!!!! Please upload more videos and in depth algorithms of time series forecasting. Thank a lot sir.
Thank you Satyam.. I will be going to algorithm only next month. This month is more for foundational concepts of timeseries which is key before getting into algorithm
Wonderful content sir! Loved the hands-on approach taught me how to handle a problem apart from theory! Thankyou, will be following the entire playlist and waiting for more!
Thank you for making such elaborate videos on such a topic where not enough content is present. This really helps.
Just the last imputation of previous year has a mistake, the correct code would be below:
aq_df_imp['TIME_PYEAR'] = aq_df_imp.apply(lambda x: aq_df.loc[x['year_month_day_hour'] + pd.offsets.DateOffset(years=-1)]['TEMP'] if pd.isna(x['TEMP']) else x['TEMP'],axis=1)
Thank you and thanks for pointing it out. :)
Very intuitive and clear video with understanding point of view.
Lots of details and concepts discussed. Thanks a lot for sharing.
Sir if you could share your GitHub repo for this codes in your description box, it would be a lot easy for us to replicate the codes, and to spend more time on consuming concepts and less time in re-writing coding.
Jaspreet - My git repo for this series is here - github.com/srivatsan88/End-to-End-Time-Series
Maybe I need to go back and edit my video with individual links when I get time :)
@@AIEngineeringLife Thanks for sharing the link. This repo will help a lot.
To ease out time taken to edit descriptions, we viewers will also drop this repo link in comment section of other videos in this playlist :)
Keep sharing your knowledge and helping us.
great video, thank you very much. I believe the 'TEMP_ROLLING' correction should be:
aq_df_imp['TEMP_ROLLING'] = aq_df_imp['TEMP'].fillna(aq_df_imp['TEMP'].rolling(3, min_periods=1).mean())
...where the window size is 3 because it includes the NA value itself.
very good explanation ! and a very nice youtuber !
That was helpful !
i am Mathe Tutor on UA-cam
and learning Python
much Love from Germany !
Thank you 🙏
Actually instead of taking the previous year value of temperature we are taking the next year value as we apply "- pd.offsets.DateOffset(year=-1)", should be either "-+pd.offsets.DateOffset(year=-1)" or "- pd.offsets.DateOffset(year=1)"
Arun.. Yes that was a mistake I did. Basically must have tested the code properly :) . But for that small change other concepts are same. I have commented the right code in another comment as well
Thanks for showing the slider graph from Plotly, was very helpful....
Thnak you very much for making concepts Lucid and clear..Could you please post code repo also..?
Here is my repo - github.com/srivatsan88/End-to-End-Time-Series
Very good approach to time series data, i have questions, out of the all approach which one you think fits best
Anshul.. it depends on the data and business understanding.. but these are different common ways and there can be more depending on need
Hello Sir,
Did you consciously impute 2015 Temperature missing values with 2016 values?
You gave an offset of "-1" years and subracted it from 2015 index, resulting in 2016 values.
Dammalapati.. Oh really.. I think I typed by mistake then.. Offset should be 1. Thanks for pointing it out. Will check it in sometime
Thank you so much. This is very helpful.
Very informative 👍
For the monthly sales data. If there is no sales in a particular month. How to handle this discontinuity?? Can we impute 0 there because there is no sales happened that time?
Hi, How would I impute the previous years data but matching on date as well as latitude and longitude point?
Hey, for the last code the substituting null with previous year values, m getting a timestamp error..I checked in a lot of places but couldn't find. Can u help me with it?
Thanks ! That was helpful
Dear, why the 1st NAN (2015-02-21 120000) + 2nd NAN (2015-02-21 140000) + 3rd NAN (2015-02-21200000) of TEMP_PREVY = [1.9, 4.9, 3] 31:20; while they are [3, 3.5, 1.2] respectively in 2014 30:02 ?
Randy.. Sorry, That was my mistake. A bug in code. DataOffset should be added and not subtracted in the function. Try below
aq_df_imp['TEMP_PREVY']=aq_df_imp.apply(lambda x: aq_df.loc[x['year_month_day_hour'] + pd.offsets.DateOffset(years=-1)]['TEMP'] if pd.isna(x['TEMP']) else x['TEMP'], axis=1)
Thank you so much 🙌
when I make this aq_df_imp['TEMP_PREVY']=aq_df_imp.apply(lambda x: aq_df.loc[x['year_month_day_hour'] + pd.offsets.DateOffset(years=-1)]['TEMP'] if pd.isna(x['TEMP']) else x['TEMP'], axis=1)
i have error KeyError: Timestamp('2017-03-29 00:00:00')
where can i find the code?
Could you please send me this notebook ? It will be really helpful.
Thanks
Can you please share code file
Search for github srivatsan88 and in that this is available in end to end time series playlist. Let me know in case if you have trouble locating this file
can you share the code
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
in
----> 1 import plotly.express as px
2
3 fig = px.line(delhi,x='date',y = 'pm25',title='pm25 with Slider')
4
5 fig.update_xaxes(
ModuleNotFoundError: No module named 'plotly'
I'm getting this error even though I installed 'pip install plotly'
Are you using colab or local notebook. If local can you check if it is pointing to right python environment
Sound quality is not good
Sorry for it. Sometimes recording plays against me while I try my best to fix it. Can you please watch without headset this video so noise is less. I am trying to fix it permanently
@@AIEngineeringLife no worries other videos audio quality is good. Thanks for making efforts for us