The ‘verbose’ parameter in scikit-learn’s GridSearchCV controls the amount of logging information displayed during the execution of the grid search. Adjusting this parameter helps in monitoring the progress and debugging issues during hyperparameter tuning.
Grid search is a method for exhaustively searching over a specified set of parameter values to find the best combination. It trains and evaluates the model for each combination of parameters.
The ‘verbose’ parameter can be set to different integer levels (e.g., 0, 1, 2, 3) to control the amount of detail in the logging output.
The default value is 0, which means no logging. Higher values provide more detailed logs, useful for debugging or monitoring long-running searches.
from sklearn.datasets import make_classification
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
# create a synthetic dataset
X, y = make_classification(n_samples=1000, n_classes=2, random_state=42)
# define the parameter grid
param_grid = {'C': [0.1, 1, 10], 'gamma': [1, 0.1, 0.01]}
# define and perform a grid search with verbose level 1
grid_verbose_1 = GridSearchCV(estimator=SVC(), param_grid=param_grid, verbose=1, cv=5)
grid_verbose_1.fit(X, y)
# define and perform a grid search with verbose level 3
grid_verbose_3 = GridSearchCV(estimator=SVC(), param_grid=param_grid, verbose=3, cv=5)
grid_verbose_3.fit(X, y)
Running the example gives an output like:
Fitting 5 folds for each of 9 candidates, totalling 45 fits
Fitting 5 folds for each of 9 candidates, totalling 45 fits
[CV 1/5] END ....................C=0.1, gamma=1;, score=0.500 total time= 0.0s
[CV 2/5] END ....................C=0.1, gamma=1;, score=0.510 total time= 0.0s
[CV 3/5] END ....................C=0.1, gamma=1;, score=0.510 total time= 0.0s
[CV 4/5] END ....................C=0.1, gamma=1;, score=0.520 total time= 0.0s
[CV 5/5] END ....................C=0.1, gamma=1;, score=0.555 total time= 0.0s
[CV 1/5] END ..................C=0.1, gamma=0.1;, score=0.870 total time= 0.0s
[CV 2/5] END ..................C=0.1, gamma=0.1;, score=0.880 total time= 0.0s
[CV 3/5] END ..................C=0.1, gamma=0.1;, score=0.845 total time= 0.0s
[CV 4/5] END ..................C=0.1, gamma=0.1;, score=0.845 total time= 0.0s
[CV 5/5] END ..................C=0.1, gamma=0.1;, score=0.825 total time= 0.0s
[CV 1/5] END .................C=0.1, gamma=0.01;, score=0.885 total time= 0.0s
[CV 2/5] END .................C=0.1, gamma=0.01;, score=0.895 total time= 0.0s
[CV 3/5] END .................C=0.1, gamma=0.01;, score=0.835 total time= 0.0s
[CV 4/5] END .................C=0.1, gamma=0.01;, score=0.835 total time= 0.0s
[CV 5/5] END .................C=0.1, gamma=0.01;, score=0.835 total time= 0.0s
[CV 1/5] END ......................C=1, gamma=1;, score=0.605 total time= 0.0s
[CV 2/5] END ......................C=1, gamma=1;, score=0.550 total time= 0.0s
[CV 3/5] END ......................C=1, gamma=1;, score=0.600 total time= 0.0s
[CV 4/5] END ......................C=1, gamma=1;, score=0.595 total time= 0.0s
[CV 5/5] END ......................C=1, gamma=1;, score=0.610 total time= 0.0s
[CV 1/5] END ....................C=1, gamma=0.1;, score=0.885 total time= 0.0s
[CV 2/5] END ....................C=1, gamma=0.1;, score=0.880 total time= 0.0s
[CV 3/5] END ....................C=1, gamma=0.1;, score=0.865 total time= 0.0s
[CV 4/5] END ....................C=1, gamma=0.1;, score=0.850 total time= 0.0s
[CV 5/5] END ....................C=1, gamma=0.1;, score=0.830 total time= 0.0s
[CV 1/5] END ...................C=1, gamma=0.01;, score=0.900 total time= 0.0s
[CV 2/5] END ...................C=1, gamma=0.01;, score=0.895 total time= 0.0s
[CV 3/5] END ...................C=1, gamma=0.01;, score=0.870 total time= 0.0s
[CV 4/5] END ...................C=1, gamma=0.01;, score=0.825 total time= 0.0s
[CV 5/5] END ...................C=1, gamma=0.01;, score=0.845 total time= 0.0s
[CV 1/5] END .....................C=10, gamma=1;, score=0.615 total time= 0.0s
[CV 2/5] END .....................C=10, gamma=1;, score=0.565 total time= 0.0s
[CV 3/5] END .....................C=10, gamma=1;, score=0.620 total time= 0.0s
[CV 4/5] END .....................C=10, gamma=1;, score=0.595 total time= 0.0s
[CV 5/5] END .....................C=10, gamma=1;, score=0.610 total time= 0.0s
[CV 1/5] END ...................C=10, gamma=0.1;, score=0.865 total time= 0.0s
[CV 2/5] END ...................C=10, gamma=0.1;, score=0.875 total time= 0.0s
[CV 3/5] END ...................C=10, gamma=0.1;, score=0.830 total time= 0.0s
[CV 4/5] END ...................C=10, gamma=0.1;, score=0.810 total time= 0.0s
[CV 5/5] END ...................C=10, gamma=0.1;, score=0.805 total time= 0.0s
[CV 1/5] END ..................C=10, gamma=0.01;, score=0.895 total time= 0.0s
[CV 2/5] END ..................C=10, gamma=0.01;, score=0.885 total time= 0.0s
[CV 3/5] END ..................C=10, gamma=0.01;, score=0.865 total time= 0.0s
[CV 4/5] END ..................C=10, gamma=0.01;, score=0.825 total time= 0.0s
[CV 5/5] END ..................C=10, gamma=0.01;, score=0.845 total time= 0.0s
The key steps in this example are:
- Generate a synthetic binary classification dataset using
make_classification. - Define a parameter grid for
SVCwithCandgammavalues to search over. - Create two
GridSearchCVobjects with different ‘verbose’ settings (1 and 3). - Fit both grid search objects to observe the different logging outputs.
- Highlight how increasing verbosity can aid in debugging or monitoring the grid search process.
This demonstrates the usefulness of the ‘verbose’ parameter for providing insight into the progress and execution details of the grid search, which can be particularly helpful for long-running searches or troubleshooting issues.