Scikit-Learn Get GridSearchCV "best_index_" Attribute

The GridSearchCV class in scikit-learn performs an exhaustive search over a specified parameter grid to find the best combination of hyperparameters for a given model. After the grid search is complete, the best_index_ attribute of the GridSearchCV object stores the index of the best performing hyperparameter combination.

Accessing the best_index_ attribute allows you to identify the optimal hyperparameters from the specified parameter grid. You can use this index to retrieve the corresponding hyperparameter values and gain insights into the best configuration for your model.

from sklearn.datasets import make_regression
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.model_selection import GridSearchCV

# Generate a synthetic regression dataset
X, y = make_regression(n_samples=100, n_features=10, noise=0.1, random_state=42)

# Create a GradientBoostingRegressor estimator
gbr = GradientBoostingRegressor(random_state=42)

# Define the parameter grid
param_grid = {
    'n_estimators': [5, 10, 50],
    'learning_rate': [0.01, 0.1, 0.5],
    'max_depth': [3, 5, 7]
}

# Create a GridSearchCV object
grid_search = GridSearchCV(estimator=gbr, param_grid=param_grid, cv=5)

# Fit the GridSearchCV object
grid_search.fit(X, y)

# Access the best_index_ attribute
best_index = grid_search.best_index_

# Print the best index
print("Best index:", best_index)

# Retrieve the best hyperparameters using the best index
best_params = grid_search.cv_results_['params'][best_index]

# Print the best hyperparameters
print("Best hyperparameters:", best_params)

Running the example gives an output like:

Best index: 11
Best hyperparameters: {'learning_rate': 0.1, 'max_depth': 3, 'n_estimators': 50}

The key steps in this example are:

1. Preparing a synthetic regression dataset using make_regression.
2. Defining the GradientBoostingRegressor estimator and the parameter grid.
3. Creating a GridSearchCV object with the estimator, parameter grid, and cross-validation strategy.
4. Fitting the GridSearchCV object on the synthetic dataset.
5. Accessing the best_index_ attribute from the fitted GridSearchCV object.
6. Using best_index_ to retrieve the best hyperparameters from the cv_results_['params'] list.

By accessing the best_index_ attribute, you can easily identify the optimal hyperparameter combination for your model based on the grid search results.