Configure StackingRegressor "verbose" Parameter

The verbose parameter in scikit-learn’s StackingRegressor controls the verbosity of output during fitting and prediction.

StackingRegressor is an ensemble method that combines multiple base regressors by training a meta-regressor on their outputs. The verbose parameter determines how much information is printed during the fitting process.

Higher values of verbose provide more detailed output, which can be useful for debugging or monitoring the training progress. However, excessive verbosity can clutter the console and slow down execution.

The default value for verbose is 0, which means no output is produced during fitting.

In practice, values of 0 (no output), 1 (some output), and 2 (detailed output) are commonly used.

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
from sklearn.ensemble import StackingRegressor
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
import numpy as np

# Generate synthetic dataset
X, y = make_regression(n_samples=1000, n_features=10, noise=0.1, random_state=42)

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Define base models
base_models = [
    ('rf', RandomForestRegressor(n_estimators=10, random_state=42)),
    ('gb', GradientBoostingRegressor(n_estimators=10, random_state=42))
]

# Create StackingRegressor with different verbose levels
verbose_levels = [0, 1, 2]

for level in verbose_levels:
    stacking_regressor = StackingRegressor(
        estimators=base_models,
        final_estimator=LinearRegression(),
        verbose=level
    )

    print(f"\nFitting StackingRegressor with verbose={level}")
    stacking_regressor.fit(X_train, y_train)

    y_pred = stacking_regressor.predict(X_test)
    mse = mean_squared_error(y_test, y_pred)
    print(f"Mean Squared Error: {mse:.4f}")

Running the example gives an output like:

Fitting StackingRegressor with verbose=0
Mean Squared Error: 2976.2447

Fitting StackingRegressor with verbose=1
Mean Squared Error: 2976.2447

Fitting StackingRegressor with verbose=2
Mean Squared Error: 2976.2447

The key steps in this example are:

Generate a synthetic regression dataset
Split the data into train and test sets
Define base models for the StackingRegressor
Create StackingRegressor instances with different verbose levels
Fit each model and observe the output verbosity
Make predictions and calculate mean squared error for each model

Some tips for setting the verbose parameter:

Use verbose=0 for production environments or when running many experiments
Set verbose=1 or verbose=2 during development or debugging to gain insights into the fitting process
Consider using higher verbosity levels for complex models or large datasets to monitor progress

Issues to consider:

Higher verbosity levels may significantly increase execution time for large datasets
Verbose output can be overwhelming and hard to interpret for very complex models
In some environments (e.g., Jupyter notebooks), excessive verbose output might affect performance or responsiveness

See Also