The estimator
parameter in scikit-learn’s GridSearchCV
allows you to specify the model to use for hyperparameter tuning. This flexibility enables you to search over parameters for various algorithms using the same GridSearchCV
setup.
Grid search is a method for exhaustively searching over a specified set of parameter values to find the best combination. It trains and evaluates the model for each combination of parameters, using a specified scoring metric.
The estimator passed to GridSearchCV
must implement the scikit-learn estimator API, which includes methods like fit()
and predict()
. This allows GridSearchCV
to train and evaluate the model during the search process.
from sklearn.datasets import make_regression
from sklearn.model_selection import GridSearchCV
from sklearn.linear_model import Ridge
from sklearn.linear_model import ElasticNet
X, y = make_regression(n_samples=100, n_features=1, noise=20, random_state=42)
param_grid = {'alpha': [0.1, 1.0, 10.0]}
grid_ridge = GridSearchCV(estimator=Ridge(), param_grid=param_grid, scoring='r2', cv=5)
grid_ridge.fit(X, y)
grid_svr = GridSearchCV(estimator=ElasticNet(), param_grid=param_grid, scoring='r2', cv=5)
grid_svr.fit(X, y)
print("Best parameters found for Ridge:")
print(grid_ridge.best_params_)
print(f"Best R2 score: {grid_ridge.best_score_:.3f}")
print("Best parameters found for ElasticNet:")
print(grid_svr.best_params_)
print(f"Best R2 score: {grid_svr.best_score_:.3f}")
Running the example gives an output like:
Best parameters found for Ridge:
{'alpha': 0.1}
Best R2 score: 0.837
Best parameters found for ElasticNet:
{'alpha': 0.1}
Best R2 score: 0.835
The key steps in this example are:
- Generate a synthetic regression dataset using
make_regression
- Define a parameter grid for the
alpha
parameter, which is common to bothRidge
andElasticNet
- Create two
GridSearchCV
objects, one withRidge
as the estimator and one withElasticNet
- Fit both grid search objects to find the best
alpha
value for each model - Print out the best parameters and corresponding R2 score for each model
This demonstrates how the same GridSearchCV
setup can be used with different estimators by changing the estimator
parameter. This allows for easy comparison of hyperparameter tuning results across multiple models.