Configure ExtraTreesRegressor "verbose" Parameter

The verbose parameter in scikit-learn’s ExtraTreesRegressor controls the level of output during model training.

Extra Trees Regressor is an ensemble learning method that builds multiple decision trees and combines their predictions to improve performance and reduce overfitting. It differs from Random Forest in how it constructs the trees.

The verbose parameter determines how much information is displayed during the fitting process. Higher values provide more detailed output, which can be useful for monitoring progress and debugging.

The default value for verbose is 0, which means no output is produced during fitting. Common values are 0 (no output), 1 (some output), and greater than 1 (more detailed output).

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.ensemble import ExtraTreesRegressor
from sklearn.metrics import mean_squared_error
import time

# Generate synthetic dataset
X, y = make_regression(n_samples=10000, n_features=20, noise=0.1, random_state=42)

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train with different verbose values
verbose_values = [0, 1, 2]
mse_scores = []
training_times = []

for verbose in verbose_values:
    start_time = time.time()
    etr = ExtraTreesRegressor(n_estimators=100, random_state=42, verbose=verbose)
    etr.fit(X_train, y_train)
    training_time = time.time() - start_time

    y_pred = etr.predict(X_test)
    mse = mean_squared_error(y_test, y_pred)

    mse_scores.append(mse)
    training_times.append(training_time)

    print(f"verbose={verbose}, MSE: {mse:.4f}, Training Time: {training_time:.2f} seconds")

Running the example gives an output like:

verbose=0, MSE: 3873.3552, Training Time: 3.21 seconds
[Parallel(n_jobs=1)]: Done  49 tasks      | elapsed:    1.5s
[Parallel(n_jobs=1)]: Done  49 tasks      | elapsed:    0.0s
verbose=1, MSE: 3873.3552, Training Time: 3.20 seconds
building tree 1 of 100
building tree 2 of 100
building tree 3 of 100
building tree 4 of 100
building tree 5 of 100
building tree 6 of 100
building tree 7 of 100
building tree 8 of 100
building tree 9 of 100
building tree 10 of 100
building tree 11 of 100
building tree 12 of 100
building tree 13 of 100
building tree 14 of 100
building tree 15 of 100
building tree 16 of 100
building tree 17 of 100
building tree 18 of 100
building tree 19 of 100
building tree 20 of 100
building tree 21 of 100
building tree 22 of 100
building tree 23 of 100
building tree 24 of 100
building tree 25 of 100
building tree 26 of 100
building tree 27 of 100
building tree 28 of 100
building tree 29 of 100
building tree 30 of 100
building tree 31 of 100
building tree 32 of 100
building tree 33 of 100
building tree 34 of 100
building tree 35 of 100
building tree 36 of 100
building tree 37 of 100
building tree 38 of 100
building tree 39 of 100
building tree 40 of 100
[Parallel(n_jobs=1)]: Done  40 tasks      | elapsed:    1.3s
building tree 41 of 100
building tree 42 of 100
building tree 43 of 100
building tree 44 of 100
building tree 45 of 100
building tree 46 of 100
building tree 47 of 100
building tree 48 of 100
building tree 49 of 100
building tree 50 of 100
building tree 51 of 100
building tree 52 of 100
building tree 53 of 100
building tree 54 of 100
building tree 55 of 100
building tree 56 of 100
building tree 57 of 100
building tree 58 of 100
building tree 59 of 100
building tree 60 of 100
building tree 61 of 100
building tree 62 of 100
building tree 63 of 100
building tree 64 of 100
building tree 65 of 100
building tree 66 of 100
building tree 67 of 100
building tree 68 of 100
building tree 69 of 100
building tree 70 of 100
building tree 71 of 100
building tree 72 of 100
building tree 73 of 100
building tree 74 of 100
building tree 75 of 100
building tree 76 of 100
building tree 77 of 100
building tree 78 of 100
building tree 79 of 100
building tree 80 of 100
building tree 81 of 100
building tree 82 of 100
building tree 83 of 100
building tree 84 of 100
building tree 85 of 100
building tree 86 of 100
building tree 87 of 100
building tree 88 of 100
building tree 89 of 100
building tree 90 of 100
building tree 91 of 100
building tree 92 of 100
building tree 93 of 100
building tree 94 of 100
building tree 95 of 100
building tree 96 of 100
building tree 97 of 100
building tree 98 of 100
building tree 99 of 100
building tree 100 of 100
[Parallel(n_jobs=1)]: Done  40 tasks      | elapsed:    0.0s
verbose=2, MSE: 3873.3552, Training Time: 3.70 seconds

The key steps in this example are:

Generate a synthetic regression dataset
Split the data into train and test sets
Train ExtraTreesRegressor models with different verbose values
Measure the training time for each model
Evaluate the mean squared error of each model on the test set

Some tips for using the verbose parameter:

Use verbose=0 for production code to minimize unnecessary output
Set verbose=1 or higher when you need to monitor training progress
Higher verbose values are useful for debugging or detailed analysis

Issues to consider:

Increased verbosity can slightly slow down the training process
Very high verbose values may produce excessive output, which can be difficult to interpret
The appropriate verbose level depends on your specific needs and the size of your dataset

See Also