SKLearner Home | About | Contact | Examples

Scikit-Learn Configure GridSearchCV "estimator" Parameter

The estimator parameter in scikit-learn’s GridSearchCV allows you to specify the model to use for hyperparameter tuning. This flexibility enables you to search over parameters for various algorithms using the same GridSearchCV setup.

Grid search is a method for exhaustively searching over a specified set of parameter values to find the best combination. It trains and evaluates the model for each combination of parameters, using a specified scoring metric.

The estimator passed to GridSearchCV must implement the scikit-learn estimator API, which includes methods like fit() and predict(). This allows GridSearchCV to train and evaluate the model during the search process.

from sklearn.datasets import make_regression
from sklearn.model_selection import GridSearchCV
from sklearn.linear_model import Ridge
from sklearn.linear_model import ElasticNet

X, y = make_regression(n_samples=100, n_features=1, noise=20, random_state=42)

param_grid = {'alpha': [0.1, 1.0, 10.0]}

grid_ridge = GridSearchCV(estimator=Ridge(), param_grid=param_grid, scoring='r2', cv=5)
grid_ridge.fit(X, y)

grid_svr = GridSearchCV(estimator=ElasticNet(), param_grid=param_grid, scoring='r2', cv=5)
grid_svr.fit(X, y)

print("Best parameters found for Ridge:")
print(grid_ridge.best_params_)
print(f"Best R2 score: {grid_ridge.best_score_:.3f}")

print("Best parameters found for ElasticNet:")
print(grid_svr.best_params_)
print(f"Best R2 score: {grid_svr.best_score_:.3f}")

Running the example gives an output like:

Best parameters found for Ridge:
{'alpha': 0.1}
Best R2 score: 0.837
Best parameters found for ElasticNet:
{'alpha': 0.1}
Best R2 score: 0.835

The key steps in this example are:

  1. Generate a synthetic regression dataset using make_regression
  2. Define a parameter grid for the alpha parameter, which is common to both Ridge and ElasticNet
  3. Create two GridSearchCV objects, one with Ridge as the estimator and one with ElasticNet
  4. Fit both grid search objects to find the best alpha value for each model
  5. Print out the best parameters and corresponding R2 score for each model

This demonstrates how the same GridSearchCV setup can be used with different estimators by changing the estimator parameter. This allows for easy comparison of hyperparameter tuning results across multiple models.



See Also