The early_stopping parameter in scikit-learn’s MLPRegressor determines whether to use early stopping to terminate training when validation score is not improving.
MLPRegressor is a multi-layer perceptron regressor that trains using backpropagation. It can learn non-linear models and is suitable for regression datasets where complex relationships exist between features and target variables.
The early_stopping parameter, when set to True, splits off part of the training data as a validation set. Training stops when validation score is not improving, potentially preventing overfitting and reducing training time.
By default, early_stopping is set to False. When enabled, it’s common to use it in conjunction with parameters like validation_fraction (default 0.1) and n_iter_no_change (default 10).
from sklearn.neural_network import MLPRegressor
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
# Generate synthetic dataset
X, y = make_regression(n_samples=1000, n_features=10, noise=0.1, random_state=42)
# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Train models with and without early stopping
mlp_with_es = MLPRegressor(hidden_layer_sizes=(100,), max_iter=1000, early_stopping=True, random_state=42)
mlp_without_es = MLPRegressor(hidden_layer_sizes=(100,), max_iter=1000, early_stopping=False, random_state=42)
mlp_with_es.fit(X_train, y_train)
mlp_without_es.fit(X_train, y_train)
# Evaluate models
mse_with_es = mean_squared_error(y_test, mlp_with_es.predict(X_test))
mse_without_es = mean_squared_error(y_test, mlp_without_es.predict(X_test))
print(f"MSE with early stopping: {mse_with_es:.4f}")
print(f"MSE without early stopping: {mse_without_es:.4f}")
Running this example produces output similar to:
MSE with early stopping: 287.6796
MSE without early stopping: 30.5298
Key steps in this example:
- Generate a synthetic regression dataset
- Split data into train and test sets
- Create and train
MLPRegressormodels with and without early stopping - Compare model performance using mean squared error
Tips for using early_stopping:
- Use with
validation_fractionto control the size of the validation set - Adjust
n_iter_no_changeto set the “patience” of early stopping - Monitor both training and validation scores to detect overfitting
Issues to consider:
- Early stopping may prevent the model from finding the global minimum
- It can be sensitive to the random initialization of weights
- May require more epochs (higher
max_iter) to ensure convergence