Scikit-Learn Get RandomizedSearchCV "best_params_" Attribute

RandomizedSearchCV is a powerful tool for hyperparameter optimization that allows you to sample from distributions of hyperparameter values.

After running a random search, you can access the best hyperparameters found during the search using the best_params_ attribute.

The best_params_ attribute is a dictionary containing the hyperparameter configuration that achieved the highest mean cross-validated score during the random search.

Accessing best_params_ is crucial for obtaining the optimal hyperparameters to train your final model and achieve the best performance.

from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import randint

# Generate a random classification dataset
X, y = make_classification(n_samples=100, n_classes=2, n_informative=5, n_redundant=5, random_state=42)

# Set up a RandomForestClassifier
rf = RandomForestClassifier(random_state=42)

# Define hyperparameter distributions to sample from
param_dist = {
    'n_estimators': randint(5, 50),
    'max_depth': [3, 5, 10, None],
    'min_samples_split': randint(2, 10),
}

# Run random search with 5-fold cross-validation
random_search = RandomizedSearchCV(rf, param_distributions=param_dist, n_iter=10, cv=5, random_state=42)
random_search.fit(X, y)

# Access the best parameters found
best_params = random_search.best_params_

# Print the best parameters
print("Best parameters found:")
print(best_params)

Running the example gives an output like:

Best parameters found:
{'max_depth': None, 'min_samples_split': 4, 'n_estimators': 26}

The steps are as follows:

1. Import required libraries and prepare a synthetic classification dataset using make_classification.
2. Configure a RandomForestClassifier and define distributions to sample hyperparameters from.
3. Run RandomizedSearchCV with the classifier, hyperparameter distributions, 10 iterations, and 5-fold cross-validation.
4. After fitting, access the best_params_ attribute from the random_search object to retrieve the best hyperparameters found.

By accessing the best_params_ attribute, you can obtain the optimal hyperparameter configuration found during the random search. These hyperparameters can then be used to train your final model, ensuring the best possible performance on your dataset.