NOTE: You can support StatQuest by purchasing the Jupyter Notebook and Python code seen in this video here: statquest.gumroad.com/l/uroxo Support StatQuest by buying my books The StatQuest Illustrated Guide to Machine Learning, The StatQuest Illustrated Guide to Neural Networks and AI, or a Study Guide or Merch!!! statquest.org/statquest-store/
Dear Josh ... I have a request for new videos or livechats ... could you explain us these tests maybe ? ... Tukey, Bonferroni and Scheffé , it's hard for me to understand , you explain everything so well ... could be very helpful for a lot of people out there ... have a nice day , greetings from Europe
Please keep doing these long-form Python tutorials on the various ideas we've covered in earlier 'Quests. They're great for those of us working in Python, and they give me another way to support the channel. It has been a more-than-pleasant surprise that as I've grown from learning the basics of stats to machine learning and eventually deep learning, StatQuest has grown along with me into those very same fields. Thanks Josh.
You are amazing. Can't imagine how much work you put into those step-by-step tutorials. Just bought the Jupyter Notebook code and it's beyond worth it! Thank you :)
Hey Josh, I just purchased all of your 3 Jupyter Notebook! I transferred from Econ major to Data Science, it was a nightmare before I find your channel. Your channel shed the light upon my academic career! Look forward to more of the 'Python from Start to Finish' series, and I will definitely support it!
I am purchasing the Jupiter notebook to contribute to your work! Thanks a lot for this video! You are awesome! Will be very very happy to have more ML tutorials and thank you Josh!
Wow. Finally I see a face for the name. Your previous videos have had immensely helpul. I assumed you are a very senior person. I am not measuring your age. I mean, your way of explaining seemed like a professor with half a century of experience. But in reality, you are quite young. Thank you for all your simple-yet-detailed videos. No words to quantify how much I appreciate them. 🙏
Outstanding! Complete with the thinking path - how to analyze variables in a logical way - and with common errors. Just purchased the Notebook. Thank you for all of your work on this channel.
Josh, this video is epic and really helped me understand the actual process of tuning hyperparameters, something that had been a bit of a black box until I saw this video. Your channel is awesome too - great jingles as well :D
@@statquest one small request..can you provide some valuable information through a video like which model to chose for different datasets..how do we decide what model we should chose...thanks in advance
Absolutely loved this video Josh. It breaks down everything into understandable chunks. Thank you and God bless. BAM! The only thing I missed (and its very minor) was taking in a new data row and making an actual prediction by using the model.
Hey Mate, amazing tutorial. Very complex problem explained in really simple and effective way. I am using XGBOOST for one of the classification model and after watching your video it made me realise I can further improve my model. So thank you again and keep making those videos. Kudos to you and long live data science 🙏🙏
You're a very kind human being Josh!! Thank you so much for making these videos. Your content is gold!!! I am new to data science and this is exactly what I needed!! :) Much love from India!
@@statquest Hey Josh! I am learning about Bayesian Optimizer and I don't seem to get it even after watching tons of tutorials, can you suggest where I should learn it from please? I couldn't find a video on your channel on this.
41:21 In my version of xgboost were parameters early_stopping_rounds=10 and eval_metric='aucpr' moved to XGBClassifier, so if it is not working I suggest trying this. Although I really appreciate value of StatQuest videos, I have to admit I really hate all that singing and Bams. Makes me feel like attending school for slower children
Quick note: At 43:30 instead of using the plot_confusion_matrix() which is now depreciated, you need to use ConfusionMatrixDisplay.from_estimator(). This can be done as follows: Include: from sklearn.metrics import ConfusionMatrixDisplay at the start with the other imports. Then when printing the confusion matrix you need to use the line: ConfusionMatrixDisplay.from_estimator(clf_xgb, X_test, y_test, display_labels=["Did not leave", "Left"])
I am getting this error following what you say: XGBoostError: [17:07:52] c:\buildkite-agent\builds\buildkite-windows-cpu-autoscaling-group-i-08de971ced8a8cdc6-1\xgboost\xgboost-ci-windows\src\c_api\c_api_utils.h:167: Invalid missing value: null Do you know why it might be? Thanks.
I solved the above error by setting missing=0 on the above code as follows: clf_xgb = xgb.XGBClassifier(objective='binary:logistic', missing=0, seed=42, ## the next three arguments set up early stopping. eval_metric='aucpr', early_stopping_rounds=10)
This is a great piece of work, thanks for sharing it! Maybe the only additional piece I'd add which I've found useful on the documentation of XGBoost is that one can take advantage of parallel computing (more cores or using a graphic card your machine or you could have on the cloud) by simply passing the parameter (n_jobs = -1) while doing both, the RandomizedSearchCV stage and the setting the XGB regressor type (XGBRegressor for example).
It was extremely helpful. Please continue making these videos. I suggest making a video to explain the clustering with unlabeled data, and predicting the future trend in time-series data.
Appreciate the Python related videos...helps to manoeuvre the code when I try to replicate the method later on...easy to follow the whole thing, also for beginners... 🙂
First you’ve saved me this is super clear! I love all your videos so much 😊 I do have two questions… 1. How would you handle a classification problem with time series data? 2. Is there any other evaluation test you should or could do to evaluate the effectiveness of your model?
1. I've never used XGBoost with time series (or done much of any time series stuff before), so I can't answer this question. 2. There are lots of ways to evaluate a model. I only present a few, but there are many more, and they really depend on what you want your model to do. Just google it.
Greatly appreciated this videoLike you said, telcos should gives more effort to tie the current customers. In real practice you want to know what the probability is that a current customer will no longer renew the subscription. You should then try to bind the customer with a high risk with incentives.
Wonderful video josh.....pleasee pleasee pleasee make more videos on start to finish on python for different models.....i havr actually submitted my assignments using your techniques and got better results than what i have learned in my class Waiting for more to come especially on python :)
Yikes, if I ever understand something enough to explain it as succinctly as you do then I'd be very happy. I've been smashing through a lot of your videos the last few days after spending countless months on python, sklearn and all the usual plug and play solutions and it's not been until I've started watching these that I've started to feel things click into place
Amazing. Wonderful videos. I started only 3 months ago and with your videos I am very confident to do nalysis with Python. Manny thanks. Is it possible to create a video for Nomogram for competic#ing risks for Time-Event (survival analysis) based on CPH outputs?
Thanks Josh for another GREAT video! Just some sharing and minor questions. 1. try pandas_profiling when doing EDA. I personally love it. :) 2. some features are highly correlated (eg: city name and zip code). Do we need to handle that before running XGB? 3. Why choose 10 for early_stopping_rounds 4. What’s the difference between - df.loc[df['Total_Charges']==' '] - df[df['Total_Charges']==' '] 5. What’s the difference between - y=df['Churn_Value'].copy - y=df['Churn_Value'] Many thanks in advance! H
1) Thanks for the tip on pandas_profiling. 2) No. 3) It's a commonly used number 4) I don't know. 5) I believe the former is copy by value and the latter is copy by reference.
@@statquest Sir your double and triple bam has really taught lots of things for me to be honest. Me as a student doing Post-Graduate in AI, lots of love form Nepal.
38:05 stratify=y is not for yes is for dependent variable y. if you have a different variable, you gotta pass your response variable's name to stratify
Thank you for this great tutorial Josh! Your videos have immensely helped in understanding some of the complex topics. One thing I noticed while watching this tutorial is the handling of categorical features. I think the explanation you gave for "Why not to use LabelEncoding?" is applicable for models like Linear Regression, SVM, NN but not for Trees because they only focus on the order of the feature values. For example, in a set of [1,2,3,4], threshold < 1.5 would be equivalent to threshold == 1. Please let me know if my thought process in wrong.
To be honest, I'm not really sure I understand your question or your example. If we have a categorical feature with 4 colors, red, blue, green, and black, and we give them numbers, like 1, 2, 3 and 4, then a threshold < 2.5, would not make much sense and, based on how trees are implemented, there would be no options for threshold == 2 or threshold == 3. So we wouldn't be able to separate colors very well.
@@nikhilshaganti5585 Sure, you can continue to separate things in later branches - but the greedy nature of the algorithm doesn't ensure that you'll get to those later branches. So you start by making a guess that it makes sense to group red and blue together.
Again another quality video, I was following along with your every word, which did bring up come questions: 1. When XGBoost deals with missing data, does it ever consider splitting the missing data in half? --Using your example, would it ever do 1 blue and 1 green? What would happen if XGBoost encountered a data set with alot of missing values? 2. When you ran your Cross-Validation, was there a reason you only used 3 values for each hyperparameter? Could you have done more if you wanted to? 3. When I ran my Cross-Validation, my scale_pos_weight didn't change even though I used the same parameters you did. What do you think the problem could be?
1. Not that I know of. If there was a lot of missing values, it would still proceed just as described. 2. I wanted the cross validation to run in a short period of time, so I picked 3 values for each hyperparameter. If I had more time on my hand, or a cluster of computers, I might have considered trying more. 3. I'm not sure.
@@statquest Gotcha! Thank you for actually answering my questions haha I really appreciate the help as someone getting into more Machine Learning. Not sure what your upcoming videos will be but I think some great videos would be: -How you did the Cross Validation for the hyper parameters in this video? ( I have watched your Cross-Validation video, but I want to learn more about actually doing it) [I also still can get my first run to output the same values you had lol] -XGBoost for Regression in Python -How to select the best algorithm based on the scenario for Regression or Classification Just some thoughts :)
NOTE: You can support StatQuest by purchasing the Jupyter Notebook and Python code seen in this video here: statquest.gumroad.com/l/uroxo
Support StatQuest by buying my books The StatQuest Illustrated Guide to Machine Learning, The StatQuest Illustrated Guide to Neural Networks and AI, or a Study Guide or Merch!!! statquest.org/statquest-store/
Dear Josh ... I have a request for new videos or livechats ... could you explain us these tests maybe ? ... Tukey, Bonferroni and Scheffé , it's hard for me to understand , you explain everything so well ... could be very helpful for a lot of people out there ... have a nice day , greetings from Europe
Please keep doing these long-form Python tutorials on the various ideas we've covered in earlier 'Quests. They're great for those of us working in Python, and they give me another way to support the channel. It has been a more-than-pleasant surprise that as I've grown from learning the basics of stats to machine learning and eventually deep learning, StatQuest has grown along with me into those very same fields.
Thanks Josh.
That's the plan!
You are amazing. Can't imagine how much work you put into those step-by-step tutorials.
Just bought the Jupyter Notebook code and it's beyond worth it! Thank you :)
Thank you very much for your support! :)
Hey Josh, I just purchased all of your 3 Jupyter Notebook! I transferred from Econ major to Data Science, it was a nightmare before I find your channel. Your channel shed the light upon my academic career! Look forward to more of the 'Python from Start to Finish' series, and I will definitely support it!
Awesome! Thank you!
Thank you for the great work!
Wow! Thank you so much for supporting StatQuest!!! BAM! :)
This is hands down the best Python tutorial on UA-cam.. not just for XGBoost, but overall Python logic and syntax. Nice work, subscribed!!
Wow! Thank you!
I am purchasing the Jupiter notebook to contribute to your work! Thanks a lot for this video! You are awesome! Will be very very happy to have more ML tutorials and thank you Josh!
Thank you very much! :)
Your pronunciation is the most authentic and clearest that I have ever heard
Wow! Thank you!
Wow. Finally I see a face for the name. Your previous videos have had immensely helpul. I assumed you are a very senior person. I am not measuring your age. I mean, your way of explaining seemed like a professor with half a century of experience. But in reality, you are quite young. Thank you for all your simple-yet-detailed videos. No words to quantify how much I appreciate them. 🙏
Wow, thanks!
Thanks!
WOW! Thank you so much for supporting StatQuest!!! BAM! :)
Outstanding! Complete with the thinking path - how to analyze variables in a logical way - and with common errors. Just purchased the Notebook. Thank you for all of your work on this channel.
Triple bam!!! Thank you very much for supporting StatQuest! :)
Josh, you’re well and truly phenomenal ! Love from Madras !
Chennai
BAM! Thank you very much!!!
Hi Rahul I taught atIIT-madras 19192-1993 lived on campus across from post office josh visited us there
Frank Starmer Hello Frank, wow! That’s great to know ! :) I’m sure you must have had a good time here. Cheers :)
@@starmerf wow the world is a small place ☺
Extremeley helpful - would love to see more from the "start to finish" series
I'm working on it.
Josh, this video is epic and really helped me understand the actual process of tuning hyperparameters, something that had been a bit of a black box until I saw this video. Your channel is awesome too - great jingles as well :D
Thank you!
Can't thank you enough for the clearest and best explanation on UA-cam
Thank you!
I understand you better than if it would be explained in my native language) Thank you for your work!
Thank you!
I love your teaching style. Extremely helpful for a beginner like me. Really helped me a lot in my exams. No words. You are the best!!!!
Thank you!
I watched your all video for XGBoost. It helps me a lot. very appreciated!
Glad it helped!
What did we do to deserve a great guy like Josh ? Thank you Josh!
Thanks! :)
A true, real Master Class - You got my support!
Thank you! :)
Liked, favorited, recommended, shared, and sacrificed my first-born to this video.
TRIPLE BAM! :)
😂😂😂
Awesome video! The cleanest xgboost explanation a have ever seen.
Wow, thanks!
That's the Best video I've ever seen. Period.
TRIPLE BAM! :)
Wow, thanks!
"25:36" that's what i was waiting for from the beginning...Truly amazing.. You are providing precious information..CHEERS
Glad it was helpful!
@@statquest one small request..can you provide some valuable information through a video like which model to chose for different datasets..how do we decide what model we should chose...thanks in advance
@@RahulVarshney_ I'll keep that in mind. In the mean time, check out: scikit-learn.org/stable/tutorial/machine_learning_map/index.html
@@statquest that is amazing ...i will complete it today itself thanks again for your prompt reply
Can i get your email
Hi Josh, great job really helpful material as I'm discovering XGBoost just now.
Thank you and keep you great work!
Thank you very much! :)
Such an amazing job Josh.. Couldn't find any better explanation than this!
Mesmerizing!
Wow, thanks!
Thanks for the great tutorial! You covered a lot of details (mostly data cleaning) that are often overlooked or skipped as 'trivial' steps.
Thank you! Yes, "data cleaning" is 95% of the job.
I really appreciate your content Josh. Thanks for your time
Thank you!
Absolutely loved this video Josh. It breaks down everything into understandable chunks. Thank you and God bless. BAM! The only thing I missed (and its very minor) was taking in a new data row and making an actual prediction by using the model.
Thanks! For new data, you just call clf_xgb.predict() with the row of new data.
I'm very grateful to have you as my teacher.
Thanks!
Hey Mate, amazing tutorial. Very complex problem explained in really simple and effective way. I am using XGBOOST for one of the classification model and after watching your video it made me realise I can further improve my model. So thank you again and keep making those videos. Kudos to you and long live data science 🙏🙏
Glad it helped!
This kind of content is SUPER HARD to produce. I really understand and appreciate your effort here. Thanks and congratulations.
Thank you very much!
You're a very kind human being Josh!! Thank you so much for making these videos. Your content is gold!!! I am new to data science and this is exactly what I needed!! :)
Much love from India!
Glad you like my videos!! BAM! :)
@@statquest Hey Josh! I am learning about Bayesian Optimizer and I don't seem to get it even after watching tons of tutorials, can you suggest where I should learn it from please? I couldn't find a video on your channel on this.
@@muskanroxx22 Unfortunately I don't know of a good source for that.
Yes pls more videos with python❤thank u for the webinar
Thanks! :)
I haven't watched it yet but I know this will be great!!!!!!!! Thank you Josh.
BAM! :)
41:21 In my version of xgboost were parameters early_stopping_rounds=10 and eval_metric='aucpr' moved to XGBClassifier, so if it is not working I suggest trying this.
Although I really appreciate value of StatQuest videos, I have to admit I really hate all that singing and Bams. Makes me feel like attending school for slower children
Sorry to hear that.
@@statquest i like it. its cool.
This guy is amazing.
DOUBLE BAM 💥 💥
Thank you! :)
BAMMMMM !!!
This is awesome 👍 Josh !! Thank you for your contribution, really helpful for new learners.😊😊😊
Glad you liked it!
@@salilgupta9427 Thanks!
Thank you for your job, the explanation of the topic is very clear and transparent.
Thank you very much! :)
I was wondering how to find stuff regarding dealing with actual churn data and sampling issues. The tutorial addressed a lot of them. Thanks!
Thanks!
Thank you so much for the work that you used in step by step tutorial. it was amazing.
You're very welcome!
Amazingly organized and well explained!
Thank you!
Man, this video is awesome! Congratulations!
Thank you! :)
It is really lovely to be able to put a face to the "Hooray!", "BAM !!!" and "Note:"s 😄❤
bam!
I love the channel! Eu aprendo + aqui do que a Graduação! You great josh!
Muito obrigado! :)
Lovely and priceless video Josh...BAM BAM BAM as usual !! :) God bless. .
Thank you very much! :)
hurray, i picture you totally different!
Thanks a lot for all the videos!
Glad you like them!
Thank you Josh. Needed this tutorial to better solve a ML Problem as part of my internship :)
Glad it helped!
Great video! You did a wonderful job of explaining the process. Thanks!
Thanks!
Jesus, i just learned more over 10 minutes of this than i did throughtout an entire semester of a similar subject on CS. ++ tutorial
Thank you!
you are simply an amazing human being, also the notebooks are great! :D
Thanks!
I'm so glad you are a bad-ass stats guru and a teacher waaaaaaaaay before a singer and a guitarist ...Thank you! ;)
joshuastarmer.bandcamp.com/
StatQuest with Josh Starmer ...not bad. A poor man’s Jack Johnson 🤔
Just pulling your leg. Thanks for all the content on stats
Quick note: At 43:30 instead of using the plot_confusion_matrix() which is now depreciated, you need to use ConfusionMatrixDisplay.from_estimator(). This can be done as follows:
Include: from sklearn.metrics import ConfusionMatrixDisplay at the start with the other imports. Then when printing the confusion matrix you need to use the line: ConfusionMatrixDisplay.from_estimator(clf_xgb, X_test, y_test, display_labels=["Did not leave", "Left"])
Thank you very much! I really appreciate it.
@@statquest no problem!
I am getting this error following what you say:
XGBoostError: [17:07:52] c:\buildkite-agent\builds\buildkite-windows-cpu-autoscaling-group-i-08de971ced8a8cdc6-1\xgboost\xgboost-ci-windows\src\c_api\c_api_utils.h:167: Invalid missing value: null
Do you know why it might be? Thanks.
I solved the above error by setting missing=0 on the above code as follows:
clf_xgb = xgb.XGBClassifier(objective='binary:logistic',
missing=0,
seed=42,
## the next three arguments set up early stopping.
eval_metric='aucpr',
early_stopping_rounds=10)
@@emilioluissaenzguillen5719 thanks it works
Freaking amazing! You explain everything so well. Thank you!
Thank you!
Great explanation and walk-through, big thanks!
Glad you enjoyed it!
just here to say thank you! will come back in a month when I have time to watch it. :)
BAM! :)
Amazing! Thanks so much for the detailed video
Thanks!
Great video! Love it!
request that you do a comparison of XGBoost, CatBoost, and LightGBM, and a quest on ensemble learning.
I'll keep those topics in mind.
many thanks to your great and so understandable video. It literaly helps me a lot in Python and XGBoost package
Glad it helped!
Love u Josh.... you are a TRIPLE BAM!!! Greetings from Bogotá, Colombia.
Muchas gracias!!! :)
Josh, you're the didactic in person form.
Thanks!
I appreciate that!
Good to also see you sing rather than just hear :).. i had to comment this even before starting the training
😊 thanks
Hard work here, I'ts funny how the responsabile scientist and the funny guy coexist, very useful lesson, thanks!
Thanks! 😃!
This is a great piece of work, thanks for sharing it!
Maybe the only additional piece I'd add which I've found useful on the documentation of XGBoost is that one can take advantage of parallel computing (more cores or using a graphic card your machine or you could have on the cloud) by simply passing the parameter (n_jobs = -1) while doing both, the RandomizedSearchCV stage and the setting the XGB regressor type (XGBRegressor for example).
Great tip! BAM!
It was extremely helpful. Please continue making these videos. I suggest making a video to explain the clustering with unlabeled data, and predicting the future trend in time-series data.
I'll keep that in mind. :)
all week searched for this thank u very much
Enjoy!
Appreciate the Python related videos...helps to manoeuvre the code when I try to replicate the method later on...easy to follow the whole thing, also for beginners... 🙂
Thanks! There will be a lot more python stuff soon.
could not help u with money right now , but i watched all the adds in video , hope that helps u financially . love u videos . keep up!!
I appreciate that
Thank you for the awesomeness!!
bam!
First you’ve saved me this is super clear! I love all your videos so much 😊
I do have two questions…
1. How would you handle a classification problem with time series data?
2. Is there any other evaluation test you should or could do to evaluate the effectiveness of your model?
1. I've never used XGBoost with time series (or done much of any time series stuff before), so I can't answer this question.
2. There are lots of ways to evaluate a model. I only present a few, but there are many more, and they really depend on what you want your model to do. Just google it.
Greatly appreciated this videoLike you said, telcos should gives more effort to tie the current customers. In real practice you want to know what the probability is that a current customer will no longer renew the subscription. You should then try to bind the customer with a high risk with incentives.
True!
Love these Python tutorials after watching theory videos:D
Glad you like them!
Thank you so much Josh Starmer! BAM!
bam!
You are amazing! Thank you so much !!
You're so welcome!
Hello josh, you are doing amazing work keep doing
Thanks!
This man deserves Nobel Prize for peace of mind ❤❤
bam! :)
Wonderful video josh.....pleasee pleasee pleasee make more videos on start to finish on python for different models.....i havr actually submitted my assignments using your techniques and got better results than what i have learned in my class
Waiting for more to come especially on python :)
Thanks! There should be more python coming out soon.
Another great tutorial. Thx
Glad you liked it!
Very enjoyable webinar Josh. Thanks for posting. I'm not 100% sure how to interpret the leaves. The largest leaf value is 0.188 where Dependents_No
The leaves are how much to increase or decrease the log(odds) for one category. For more details, see: ua-cam.com/video/8b1JEDvenQU/v-deo.html
If "Stay Cool" had a face, it'd be you :)
Bam!
Thank you very much for nice video! Very helpful for me.
Glad it was helpful!
Yikes, if I ever understand something enough to explain it as succinctly as you do then I'd be very happy. I've been smashing through a lot of your videos the last few days after spending countless months on python, sklearn and all the usual plug and play solutions and it's not been until I've started watching these that I've started to feel things click into place
Awesome! I'm glad my videos are helpful! :)
Thank you Josh!!
As a suggestion, you could do a StatQuest explaining the measures in market basket analysis?
I'll keep that in mind.
Amazing. Wonderful videos. I started only 3 months ago and with your videos I am very confident to do nalysis with Python. Manny thanks.
Is it possible to create a video for Nomogram for competic#ing risks for Time-Event (survival analysis) based on CPH outputs?
I'll keep that topic in mind.
Thanks a lot Josh!
Any time! And thanks for your support! :)
Excellent Video @StatQuest ! Can we please have more Start to Finish python videos? Like Lightgbm maybe?
I'll keep that in mind! :)
Thanks Josh for another GREAT video! Just some sharing and minor questions.
1. try pandas_profiling when doing EDA. I personally love it. :)
2. some features are highly correlated (eg: city name and zip code). Do we need to handle that before running XGB?
3. Why choose 10 for early_stopping_rounds
4. What’s the difference between
- df.loc[df['Total_Charges']==' ']
- df[df['Total_Charges']==' ']
5. What’s the difference between
- y=df['Churn_Value'].copy
- y=df['Churn_Value']
Many thanks in advance!
H
1) Thanks for the tip on pandas_profiling.
2) No.
3) It's a commonly used number
4) I don't know.
5) I believe the former is copy by value and the latter is copy by reference.
This guy is just amazing
Thanks!
Thanks for sharing! Informative.
Thanks!
I am not a math geek. I am here strictly for the intro song!
same here
DOUBLE BAM! :)
@@statquest Sir your double and triple bam has really taught lots of things for me to be honest. Me as a student doing Post-Graduate in AI,
lots of love form Nepal.
awesome!! I hope you can do a video about XGBoost with regression too.
I'll keep that in mind.
38:05 stratify=y is not for yes is for dependent variable y. if you have a different variable, you gotta pass your response variable's name to stratify
Oops! thanks for catching that.
Great Content, subscribed
Also, single best python package run through Ive seen.
Thank you very much! :)
Thank you for this great tutorial Josh! Your videos have immensely helped in understanding some of the complex topics. One thing I noticed while watching this tutorial is the handling of categorical features. I think the explanation you gave for "Why not to use LabelEncoding?" is applicable for models like Linear Regression, SVM, NN but not for Trees because they only focus on the order of the feature values. For example, in a set of [1,2,3,4], threshold < 1.5 would be equivalent to threshold == 1. Please let me know if my thought process in wrong.
To be honest, I'm not really sure I understand your question or your example. If we have a categorical feature with 4 colors, red, blue, green, and black, and we give them numbers, like 1, 2, 3 and 4, then a threshold < 2.5, would not make much sense and, based on how trees are implemented, there would be no options for threshold == 2 or threshold == 3. So we wouldn't be able to separate colors very well.
right, in your example, the threshold of threshold
@@nikhilshaganti5585 Sure, you can continue to separate things in later branches - but the greedy nature of the algorithm doesn't ensure that you'll get to those later branches. So you start by making a guess that it makes sense to group red and blue together.
# very helpful and informative, thank you!
Thank you! :)
it would be a great idea to have an entire series of "start to finish"
Thanks!
amazing tutorial Josh! Shared with my friends =D
Could you do one of these about pygam? It would be amazing :)
I'll keep that in mind.
very good explaination
Thank you!
Triple Bam! thanks for your great tutorial
Any time!
Again another quality video, I was following along with your every word, which did bring up come questions:
1. When XGBoost deals with missing data, does it ever consider splitting the missing data in half?
--Using your example, would it ever do 1 blue and 1 green? What would happen if XGBoost encountered a data set with alot of missing values?
2. When you ran your Cross-Validation, was there a reason you only used 3 values for each hyperparameter? Could you have done more if you wanted to?
3. When I ran my Cross-Validation, my scale_pos_weight didn't change even though I used the same parameters you did. What do you think the problem could be?
1. Not that I know of. If there was a lot of missing values, it would still proceed just as described.
2. I wanted the cross validation to run in a short period of time, so I picked 3 values for each hyperparameter. If I had more time on my hand, or a cluster of computers, I might have considered trying more.
3. I'm not sure.
@@statquest Gotcha! Thank you for actually answering my questions haha I really appreciate the help as someone getting into more Machine Learning. Not sure what your upcoming videos will be but I think some great videos would be:
-How you did the Cross Validation for the hyper parameters in this video? ( I have watched your Cross-Validation video, but I want to learn more about actually doing it) [I also still can get my first run to output the same values you had lol]
-XGBoost for Regression in Python
-How to select the best algorithm based on the scenario for Regression or Classification
Just some thoughts :)
@@andrewnguyen5881 I'll keep those topics in mind.