How to find the best model parameters in scikit-learn
Вставка
- Опубліковано 8 лют 2025
- In this video, you'll learn how to efficiently search for the optimal tuning parameters (or "hyperparameters") for your machine learning model in order to maximize its performance. I'll start by demonstrating an exhaustive "grid search" process using scikit-learn's GridSearchCV class, and then I'll compare it with RandomizedSearchCV, which can often achieve similar results in far less time.
Download the notebook: github.com/jus...
Grid search user guide: scikit-learn.or...
GridSearchCV documentation: scikit-learn.or...
RandomizedSearchCV documentation: scikit-learn.or...
Comparing randomized search and grid search: scikit-learn.or...
Randomized search video: • Scikit Learn Workshop ...
Randomized search notebook: github.com/amu...
Random Search for Hyper-Parameter Optimization: www.jmlr.org/pa...
WANT TO GET BETTER AT MACHINE LEARNING? HERE ARE YOUR NEXT STEPS:
1) WATCH my scikit-learn video series:
• Machine learning in Py...
2) SUBSCRIBE for more videos:
www.youtube.co...
3) JOIN "Data School Insiders" to access bonus content:
/ dataschool
4) ENROLL in my Machine Learning course:
www.dataschool...
5) LET'S CONNECT!
Newsletter: www.dataschool...
Twitter: / justmarkham
Facebook: / datascienceschool
LinkedIn: / justmarkham
Having problems with the code? I just finished updating the notebooks to use *scikit-learn 0.23* and *Python 3.9* 🎉! You can download the updated notebooks here: github.com/justmarkham/scikit-learn-videos
Ans thanks for updating it.
just brilliant. Not an inch of fluff in what you teach and you do it so beautifully.
Wow, thank you so much for your very kind words!
@@dataschool Thank you for such a nice explanation. Can I know this please
why I can't get the first element of the tuple? Thank you
print (grid.cv_results_[0])
KeyError Traceback (most recent call last)
in
----> 1 print (grid.cv_results_[0])
KeyError: 0
There's a humbleness in the way you teach and deliver the explanations. It helps me understand very very well whatever the topic is.
Wow, thank you! I really appreciate your comment.
That is exactly what I was thinking. If you can't explain it simply, you don't understand it well enough (Einstein). And you sir know what you are taking about!
At some point my eyes became teary while watching this series. I have never come across such an amazing and passionate teacher. you explain every single thing, even the questions that pop up in my mind it feels like you foresee and address them, what's baffling is that you even tag them as questions before answering them. The additional resources are also pure gold. My God will bless you sir, may you live long and always be happy
Pls don’t stop making these videos you’re saving my dissertation 😭😭 thank you so much
You're very welcome!
I like the way you explain everything so slowly and briefly. Thank you for such a quality content on ML!
Thanks for your kind words!
This is straight tutorial I've ever seen. I've not skipped a second of this video.
Glad it's helpful to you! 🙌
Kevin, you are an awesome teacher. Not only your knowledge; your teaching style is also so good that it can make extremely happy both the new comers into machine learning as well as those are already in this field but need to understand the concepts further. I have gone through quite a few of your videos in youtube and everywhere it's just outstanding !!!!
Wow, thank you very much for your kind comments! I'm so glad the videos have been helpful to you!
Your videos are excellent. You have had the ability of explaining ML using Python in a very accessible way.
I usually use R but now thanks to your great tutorials I will start using Python.
Thanks and please don't stop producing these fantastic tutorials.
+Pedro Carmona Ibáñez Thanks for your kind comments! I just published a new tutorial (55 minutes): ua-cam.com/video/85dtiMz9tSo/v-deo.html
As well, I'm teaching a new course on Machine Learning with Text: www.dataschool.io/learn/
I watched the whole playlist in 1 day !! This is one of if not the best sklearn tutorials out..
Awesome, thank you!
Just want to say, all the video in Scikit series are excellent. Especially you have made so easy to understand the complex topic of cross validation and optimal parameter tuning. Keep up the good work. Can't wait for the next one...
+Pankaj Mathur Awesome! Thanks so much for your kind words.
Your explanation makes each and every topic so simple and easy to understand. Really, I would like to thank you for your immense efforts for all your videos which are so informative and added resources help us to dig deeper in the related topics.
Thanks very much for your kind words! 😊
Data School thank you so much for these videos. You're showing all the topics that no one has put the time to explain to us plebs. Keep the good work up!
Evan Miller You're very welcome!
These videos are all really great, and I particularly like how the iPython notebooks serve so well to help me review what was in any particular video.
I'd like to put in a vote for one of the future videos to cover clustering.
Bob DuCharme Great to hear... I spend a lot of time crafting these notebooks! And, thanks for the feedback, I will certainly consider clustering.
Succinct, self contained, and very clear. The best video on youtube ive found so far explaining parameter search. Thanks so much!
Excellent, that's great to hear!
Whaoo! Brillant. Very well explained, you speak very clearly too. You certainly have a gift for teaching. Continue the good job
Thanks very much for your kind words! :)
You bring it in every video and make it easy to understand!
Thanks for your kind comment!
A M A Z I N G ! In one day I've learned what I need to get into machine learning in python and scikit-learn. I've been using matlab for a while, and it's overwhelming the possibilities of scikit and the IPython notebook. Thank you again! great job ;) I hope to keep hearing from you!
+Diogo Gonçalves Wow! What a kind and thoughtful comment! I greatly appreciate it.
Kevin and Data School, you have been angels to me. I learned a hell lot in Machine Learning, thank you so much.
You're very welcome! I'm glad the videos have been helpful to you!
If you are running Python 3.4 and getting a "Parameter values should be a list" error, then make the following adjustment to the code to properly read in the range:
k_range = list(range(1,31))
Also, as always, print statements in 3.4 require parentheses.
+Stacy H Thanks for passing along that tip!
Thanks Stacy! :)
We can use numpy.arange(1,31) instead of range.
Thanks! I recently updated the code to use Python 3.6 and scikit-learn 0.19.1. The updated code can be found here: github.com/justmarkham/scikit-learn-videos
Your the beast mate. I wanted to go through Machine Learning basics Revision, and I'm quite amazed this is the best Video I have ever watched on UA-cam, in fact best from all the reading platforms as well.
Awesome! Thank you so much for your kind words! If you want to support Data School, my Patreon campaign might interest you: www.patreon.com/dataschool/overview
Thank you for making this video. What a crystal clear explanation!
You're very welcome! 🙏
Your lectures are very clear and esay to understand. Excellent!!!
Thanks for your kind comment!
your classes are simply AMAZING, thank you so much for all your efforts putting them together!
Wow, thank you!
I like your talking speed. It gives me enough time to absorb the concepts. May be its just me.
Anyways, Thank you very much !
Great to hear - you're very welcome! :)
Thank you so much, one of the best and clearest explanation I've ever came across!!
Great to hear!
*Note:* This video was recorded using Python 2.7 and scikit-learn 0.16. Recently, I updated the code to use Python 3.6 and scikit-learn 0.19.1. You can download the updated code here: github.com/justmarkham/scikit-learn-videos
This helped. Thank you :) :)
This is really helpful. Grateful for your efforts taken in creating this resource.
You're welcome!
I would love to see more such videos, especially on handling huge datasets with categorical values
+Dinesh Kumar Murali Thanks for the suggestion! I'll take it into consideration.
Best gridsearch tutorial on youtube. Thanks!
You're welcome! Glad it was helpful to you!
U are an amazing teacher...thank you so much.
You're welcome! Thanks for your kind words :)
Your teaching and explanations are amazing Kevin! Many thanks for all the effort you have put on preparing these tutorials.
Do you have any tutorials about "DecisionTreeClassifier", to which you briefly refer in this video?
Thanks for your kind words! I don't have a video tutorial about decision trees, but I do have this lesson notebook: github.com/justmarkham/DAT8/blob/master/notebooks/17_decision_trees.ipynb
Hope that helps!
at 19:26, the knn.fit(X, y) is to display the classifier parameters only, since we already instantiate the knn model with the best parameters (1 step before the fitting).
in another word, the 'fit' method will always give you the same output, no matter you fit(X_train, y_train), or fit(X_test, y_test), or fit(X, y). It just shows the best parameters in the model. just same as print(knn)
knn.fit(X, y) is a necessary step in order to train the model with the data. Instantiating the model in the previous step does not train the model, rather it just prepares the model to be trained using certain parameters.
In a typical workflow, you would split your original dataset into a training set and a separate test set. Then, during cross-validation, you further divide the training set into multiple subsets (folds) and iteratively train and evaluate the model on these folds. The test set remains untouched and is used for final evaluation after the model selection or parameter tuning process.
Whether or not you need to do that depends on your goals: Whether you are solely trying to select the best model, or whether you also need an accurate estimate of how that best model will perform on out-of-sample data. The workflow you are proposing is a good one if you care about both goals and you have enough data such that the initial split does not compromise the first goal.
15:08 just to add something, there is another advise like this : it is better to select an odd value for k, since it may avoid the ties caused during classification . In this case [in the example on video] we have best scores when k=13, k=18, k=20, so probably it picked 13 because it is an odd number. I may be wrong but I heard it somewhere :)
hello Kevin - really great videos, love your clear and precise way of explaining concepts yet keeping them accessible. a request - there are not too many seaborn tutorial videos out there. could you consider making some? thanks and keep rocking.
I have a question but I should express my compliments before asking it. The way you explain things is marvelous, and one can understand where things come from clearly. Really, thank you so much.
The question is: What about splitting the data at the beginning of the whole process. And fit the grid search on x_train, y_train. And with the best parameter found, test the model on X_test. (I did these just to pretend to have real out-of-sample data). When I did it, the result was not that different. Now I wonder why that is so; Is it because of the smallness of the data or because what I did does not change the results so much anyway?
Thank you for your kind words! 🙏
As for your question: whether or not you need to split beforehand depends on your goals. It's complicated to explain briefly, I'm sorry! But I'll be covering it in my next ML course. Subscribe here for updates: www.dataschool.io/subscribe/
@@dataschool thank you so much. ☺️
Thank you abundantly for everything that you are doing for the world.
You are very welcome! :)
Thank you so much for this clear and helpful tutorial and all the effort you put in to your work 🌸^^
You're very welcome!
You're amazing. Thanks for the free knowledge.
You're welcome!
Fantastic. We should all join his support group. 5 a month is cheap
You´re the master! Thanks for such a well explained video.
Thank you!
simply superb, you have amazing teaching skills. Great explanations and material suggested.
I wanted to switch my career to MLearning and have been watching all your playlists. Once I complete all your videos, what is the best way to master?any suggestions.
Thanks so much for your kind words! My thoughts are here: www.dataschool.io/launch-your-data-science-career-with-python/
Thank-you so much for these amazing tutorials, I've learned so much from them and now I use scikit-learn frequently. Was this the final video or did you have any idea how many you were planning to make in this series?
***** Excellent, glad to hear! I do have more videos planned, though I haven't yet decided how many more I will make. The next one will come out in a few weeks... stay tuned!
mesmerized by your explanations.
Thank you so much!
You're very welcome!
Ahhh so awesome tutorial ,
struggling with this part for very long , now it got clear
Thanks
Regards
Great to hear!
Pure gold! Thank you, Kevin!
You're very welcome! Glad this video was helpful to you!
Another amazing video! Thank you!
Thanks again!
Brilliant! You have excellent teaching skills. Please help me understand this better. Like with feature selection, whether finding optimal model (hyper)parameters should also be performed ideally within each cross-validation iteration? If i am not mistaken, you recommend using whole dataset (unsplit) for grid search in this video. Doesn't it constitute as (some form of) training from entire data? I would be happy if you can answer. Anyways, great stuff!!
+Sunil Kalmady Thanks for your kind words! You are correct that when searching for the optimal model, feature selection and feature engineering should *ideally* occur within each cross-validation fold. However, the question is always whether the added complexity is worth the increase in reliability of your performance estimates. This is a nice, short Q&A on that topic: stats.stackexchange.com/questions/92502/cross-validation-feature-information-outside-the-fold
+Data School Thank you for your answer!!
I found the video series very useful! Thank you! :) Do you plan to have more videos in the future?
***** You're welcome! I will be creating more videos in the series, though it will be a few weeks from now before I have time to make the next one.
Hi Kevin, thank you so much for sharing this with us. enjoying any single minute of your videos. One question: receiving a warning message when running the .predict method on the dataset: "Passing 1d arrays as data is deprecated in i0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample."
Not sure I am fully understanding what is about.
Thanks again!
Glad you are enjoying the videos! Regarding the warning message, it's complicated to explain, but I discuss it in detail in this blog comment: www.dataschool.io/linear-regression-in-python/#comment-2521926219
Hope that helps!
Thanks. Great explanation. Your videos are very helpful and easy to understand.
Antony Mapfumo You're very welcome!
I like your way of explanation. Thank you so much for your effort. Could you please upload the videos on unsupervised learning.
Thanks for your kind words! I'll consider your suggestion for the future.
Very good video, very good explanation. Thank you!
Hi Kevin, thank you for making these videos as they have served as a very informative introduction to machine learning for me. I've watched each video of the series and studied the code thoroughly. Any recommendations as to what to do next as far as teaching myself machine learning? I'm planning to go back through the videos and study all of the supplemental resources you've provided and then complete the "An Introduction to Statistical Learning with Applications in R" course. Is this a good plan of action or should I focus more so on Python and scikit-learn? My main goal is to use my Physics background along with what I've been teaching myself to become a Data Scientist, but I would greatly appreciate your opinion on the matter. Thank you.
+smaxwell89 That sounds like a great plan! Here's some advice that I give my data science students: github.com/justmarkham/DAT7/blob/master/other/advice.md
Thank you so much for this. This is literally gold
You're welcome!
Very nice video series :) I'd be curious if you used any of my material apart from the video you linked to [everything I do is CC-0 unless published via a publisher so don't fear the lawyers]. I found the continuous talking / walking through for videos really hard, you do a much better job than me, I think.
+Andreas Mueller Thanks very much for the compliments - I'm flattered!
No, I'm pretty sure I didn't use any outside materials, with the exception of some images. I basically created every notebook from scratch.
Regarding the talking during videos: It takes some getting used to, for sure! I scripted this series (every word and every action), which made the recording easier, but took FOREVER (probably 150 hours of work for a 4-hour series!!)
I changed tactics for my latest video series (pandas), which is mostly unscripted, and thus I can make videos a lot faster.
Looking forward to your book! Tentative release date? :)
+Data School Summer It was announced for June but I don't think we'll make that. Stay tuned for an early release. I scripted my O'Reilly series (tried without script first, totally failed) and it also to for ever.
I'll check out your pandas videos.
Keep up the good work! I'm glad this material is openly available -- mine is not :-/
+Andreas Mueller Cool, I'll be on the lookout for the book this summer!
one of the best explanation using pathon notebook ..can you please upload video for feature selection
+akash kabra I'll consider making one for the future, thanks!
You might be interested in my recent video about feature selection: ua-cam.com/video/YaKMeAlHgqQ/v-deo.html
Thank you very much for such a very beautiful explanation!!!
Thanks for appreciating!
u r a great teacher, optimazation, gridsearch, exhaustive search were a mist to me, nut now bcz of u, they kind of on my finger tips
Great!
really awesome.... great work man... and thank youuuuu
Thanks!
NIce video,
1. please correct me if i am wrong , with the concept of gridSearch is to select the best parameter and cross - validation is to for generalizing model so that overfitting does not occur.
2. If u train the SVM model using scikit learn library with the dataset other than iris, training time is really high as compared to other model, and performance is also poor, i am confused what can be the solution.
3. There the way in IPython that we can use multiple cllient at the same time(Parallel computing), but i am lacking idea how it can be implemented here , i mead in instantiating the model, fitting the model or predicting stage. i would be happy for getting answer. I am waiting for your next video, about unsuvervised learning (clustering)
+unique raj 1. Grid search is for parameter tuning, and cross-validation is for estimating how well your model will generalize. 2. Every model takes a different amount of time to train (depending on the model itself, the data, and the tuning parameters), and no particular model is guaranteed to work for any particular problem.
may be its an update, it is now from sklearn.model_selection import GridSearchCV
very good explanation! I still got an issue in understanding your statement at 19:35 - I thought training the model not enirely on all the data available was key, because otherwise you would have no test-set to evaluate or overfit the model??
Brilliant as usual. Can you please make video on nested and non-nested Cross validation
Thanks for your suggestion!
Love simplicity in your approach! Excuse me for my naive question - So How to determine if cluster size 13 is optimal or 17 is optimal? If one was to use said KNN logic in a TRULY real life situation, should he choose 13 or 17?
Not sure if I can take on Kevin's role here but I remember from his teachings that you should choose the parameter that makes your model SIMPLER. Since 17 gives you simpler decision boundaries, you should choose 17. In general, in KNN you should use the highest parameter out of the best performing ones.
Thaanks! Congrats Its a great explanation! I was also checking some other videos of yours and a doubt appeared, I wanted to combine the pipe with the gridsearch. For instance I tried to put down the gridsearch within my pipeline and extract the results as pipe.cv_results_, however could not use the .cv_results with pipe. Could you give any hint about the combination pipe for preprocessing and then search grid? Maybe another topic for your videos Thanks!
This is really a wonderful material and really helpful to be used for teaching. Is there any resource I might use for analytically calculating predicted probabilities ? thanks!
Thanks for your kind comments! I'm not sure what you mean by "analytically calculating predicted probabilities". Could you explain in more detail, or give an example? Thanks!
I mean , for a given classifier, and for a given feature vector, to calculate the predicted probability for each class
Thanks for clarifying! Yes, most scikit-learn classifiers include a 'predict_proba' method that outputs the predicted probabilities of class membership. I cover that in video 9 in this series: ua-cam.com/video/85dtiMz9tSo/v-deo.html
Thanks for the video. I have a question. When you put the parameters in the GridSearchCV, you put the knn as the estimator. 'knn' was instantiated with 'n_neighbors = k', looking at the above code. Does GridSearchCV ignore the things inside the parenthesis of the estimator? Or do we have to put 'n_neighbors = k' inside the KNeighborsClassifier, instead of a specific number to make the code work?
The parameter grid passed to GridSearchCV will override the parameters passed to knn during instantiation. Hope that helps!
Super helpful videos!
What do you do when you want to do categorisation on a dataset which consists of many categorical variables? As I understand scikitlearn models expect numeric values always? What is the best way to convert datasets full of categorical data represented by strings into integer representations for use in scikitlearn models? Is that even an approach you would advise?
Matt Adshead Great question! Yes, scikit-learn expects numeric values. For unordered categorical features, you generally represent them as dummy variables. For ordered categorical variables, you can represent them using "sensible" numeric values. More details are available in part 3 of this notebook: nbviewer.ipython.org/github/justmarkham/DAT7/blob/master/notebooks/12_advanced_model_evaluation.ipynb
Absolutely brilliant, thank you so much!
You're very welcome!
For those, like me, who have an issue with the round line I suggest : best_scores.append(round(float(rand.best_score_), 3)) # Maybe due to Python 3.5 ## And of course "Merci Kevin"!
+Nebil Jabari That's interesting... rand.best_score_ should already be a float. What error do you get with the original line of code?
Any error message show up. It simply produce the array with all the numbers unrounded. 0.97999999999999998 insteed 0.98 No matter the value of the second argument (3, 4, 2 or 1...)
Hello,
Thank you for your courses :D they are so clear.
I have a question :
Do you think that I can use GridSearchCV with AdaBoostClassifier ? Because I have a dataset with low frequency of y=1.
Sure, GridSearchCV can be used with any model for parameter tuning. Good luck!
Great explanation !!!!! Thank You !!! I have a question. Why that GridSearch returned 13 as the best param for n_neighbors while RandomizedGridSearch returned 17 (or 20 as in 08_grid_search.ipynb) ? Looking at the plot It seems one got the lowest possible value (13) and the other the highest(17 or 20).
I'm not sure I totally understand your question, sorry! But my general response would be that for such a small dataset, and on such an easy problem, the "best" result can vary slightly. Hope that is at least a bit helpful! :)
Great ! Thanks for your ipython notebook.
You're welcome!
Awesome tutorial, I would like to know what happens if you tune parameters of different classification models to give similar predictions? Is it a good idea or a bad one? And how to go about it?
Glad you liked the tutorial! Not sure what you mean by "tuning to give similar predictions"... Instead, if you try tuning another model, your goal is to optimize for your chosen evaluation metric, not to make sure it gives similar predictions to any other model. Does that make sense?
Hi, Kevin. I noticed that the codes are running slower on my PC (which is a standard one). Can I know what requirements are you recommended on the hardware? Or, any solutions to improve the processing speed? Thank you. And thank you for your 'quick-understanding' tutorials.
I'm not sure what hardware recommendations to make, I'm sorry!
Hi Kevin! I've a quick question. In this video you explained the SearchCV methods for tuning parameters. Also, I've found that there are some functions to perform feature selection with scikit learn. But I was wondering if there's a way to perform both feature selection and hyperparameter tuning within the same pipeline in scikit-learn. Any insight is appreciated.
Sure, you can do a GridSearchCV of a Pipeline that contains both a feature selector and a model. This example from the documentation shows something similar: scikit-learn.org/stable/auto_examples/model_selection/grid_search_text_feature_extraction.html
Hope that helps!
Thank you so much for this video. You're just amazing.
Wow, thanks so much! I really appreciate it :)
Hi Mark,
It's very informative. Great video!!
Have one doubt.
I am working on one classification problem. Have used 4 classifiers. Logistic regression, svm, decision tree, knn.
Initially i used normal train test split to calculate the score it showed logistic with high accuracy.
Then I used cross validation score it also gave results with logistic regression being highly accurate compared to others.
But, when I did the parameter tuning using gridsearchcv and then calculated the cross val score it showed me svm with high accuracy then logistic or knn.
Is this expected? . Please help me with this doubt Mark.
Results before tuning can be different from results after tuning. Does that help?
Brilliant work dude
Thanks!
simply Awesome!!!Thanks a lot for all your videos.
I have a question here...Grid search we use to get the best hyper parameter as in this case n_neighbors. But how it will be helpful in linerar/logistic regression . In linear we are not using any tuning parameter (linreg.fit(x,Y)). In deep learning the role of learning rate comes may be that time it will be useful..not sure in scikit how it helps in linear regression.
You're right that linear regression has no tuning parameters. However, logistic regression does have a tuning parameter, so grid search is helpful there.
Hi Kevin,
Thanks a lot for this awesome tutorial series. Can you suggest some datasets from Kaggle competitions on which I can practice this knowledge ?
I don't have any particular suggestions, but there are lots of past competitions in which the datasets are still available. I'd recommend choosing a competition in which the topic interests you!
Do I have to split the data using the train_test_split function in cross-validation technique ??
I have seen people using cross_val_score(SVC() ,x=x_train, y=y_train,cv=10)
But in the video we do like cross_val_score(SVC() ,x, y,cv=10)
I teach the latter, but it depends on your model evaluation process. There is not one "right" evaluation process, but some are better than others. Hope that helps!
@@dataschool I also want to know that we have to scale our data before train/test split or after train/test split.Some people told me that you should scale your data after train/test split
After
Learned a lot from this video. thank u
You're welcome!
For those using Python 3.7, range(1, 31) may give you issues. If you print k_range and see range(1, 31) instead of a list of numbers,
Import numpy as np and replace range(1, 31) with np.arange(1, 31).
Thanks! You can also just wrap range in list: list(range(1, 31))
You can find my code (which has been updated for Python 3) here: github.com/justmarkham/scikit-learn-videos
Hi, quick question, why don’t you split the data before grid search? Also how would you do a confusion matrix using this model?
You are super helpful. Have to give you a thumb up!
Thanks!
Hello,
In the video, we are tuning max_depth and min_samples_leaf which are for decision tree. Why do we use KNN int he video?
Jason
They are just two separate examples.
Thank you for sharing the video. However, I was wondering what's difference of cross_val_score imported from sklearn.cross_validation and from sklearn.model_selection?
The sklearn modules were reorganized after I recorded this video, but they should be the same otherwise!
Your tutorials are so so great. Thank you for everything Kevin. I see the GridsearchCV and RandomizedSearch is kind of similar to Gradient Descent and Stochastic Descent. Am I wrong??? Can yo do a video where you explain Stochastic Gradient and it's Variation??
Thanks for your suggestion! I'll consider it for the future.
Hello,
Does grid.best_score_ take into account both the mean and the std or just the mean?
Thanks for the series. Keep it up.
JM
João Matos The documentation says it picks the "highest score", so I assume that means it only takes the mean into account (and not the standard deviation). However, if you find differently, please let me know!
this video is very useful, thank you
Glad it was helpful!
many many thanks. Just one question. If I do gridsearch, say, with an SVR with the parameter C [0.001, 0.01, 0.1, 1, 10, 100] and finally find an optimal parameter C: Can I reduce the parameter space to this optimal parameter C when I would like to do permutations for statistical testing ? Or do I loose some information during the permutation procedure ? THX!
It's a great question, but I'm not sure of the answer... sorry!
thanks for ur videos and nice voice over it ,, we love u sir thanks
You're welcome!
thankyu very much sir, this video helps me a lot
Great!
amazing teaching..thanks
You're welcome!
Hello,
Can we save the gridsearch "knowledge" to predict at a later time?
Isn't it strange that the RandomizedSearch repeated the combo 17, distance?
It's assumed that it selects random combos, but it shouldn't select duplicates, should it?
Thanks,
JM
João Matos Great questions! Here is scikit-learn's documentation on model persistence, which should allow you to save the best model (grid.best_estimator_): scikit-learn.org/stable/modules/model_persistence.html
Regarding duplicates, here is what the documentation says: "If all parameters are presented as a list, sampling without replacement is performed. If at least one parameter is given as a distribution, sampling with replacement is used." Since none of my parameters in that example were distributions, I would think that we shouldn't see any duplicates. However, either the documentation is incorrect on that point, or I'm misunderstanding what the documentation is trying to say.
You are doing great!
Thank you!
Does this method not take into account biases introduced by using the same dataset to tune hyperparameters and evaluate the model? How does this compare to nested cross-validation?
That's beyond the scope of what I can get into in a UA-cam comment - thanks for the question though!