Scikit-Learn Get GridSearchCV "best_params_" Attribute

The GridSearchCV class in scikit-learn is a powerful tool for hyperparameter tuning and model selection. It allows you to define a grid of hyperparameter values and fits a specified model for each combination of those values using cross-validation.

After the grid search process is complete, the GridSearchCV object stores the best hyperparameters found during the search in the best_params_ attribute. This attribute is a dictionary that contains the optimal hyperparameter values based on the specified scoring metric.

Accessing the best_params_ attribute allows you to retrieve the best hyperparameters found by the grid search. You can use these hyperparameters to configure your model for optimal performance without manually specifying each hyperparameter value.

from sklearn.datasets import make_regression
from sklearn.linear_model import Ridge
from sklearn.model_selection import GridSearchCV

# Generate a synthetic regression dataset
X, y = make_regression(n_samples=1000, n_features=10, noise=0.1, random_state=42)

# Create a Ridge regression estimator
ridge = Ridge(random_state=42)

# Define the parameter grid
param_grid = {
    'alpha': [0.1, 1.0, 10.0],
    'solver': ['auto', 'svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag', 'saga']
}

# Create a GridSearchCV object
grid_search = GridSearchCV(estimator=ridge, param_grid=param_grid, cv=5)

# Fit the GridSearchCV object
grid_search.fit(X, y)

# Access the best_params_ attribute
best_params = grid_search.best_params_

# Print the best hyperparameters
print("Best hyperparameters:", best_params)

Running the example gives an output like:

Best hyperparameters: {'alpha': 0.1, 'solver': 'lsqr'}

The key steps in this example are:

1. Preparing a synthetic regression dataset using make_regression to use for grid search.
2. Defining the Ridge regression estimator and the parameter grid with hyperparameters to tune.
3. Creating a GridSearchCV object with the estimator, parameter grid, and cross-validation strategy.
4. Fitting the GridSearchCV object on the synthetic dataset.
5. Accessing the best_params_ attribute from the fitted GridSearchCV object to retrieve the best hyperparameters found by the grid search.