Configure MLPClassifier "n_iter_no_change" Parameter

The n_iter_no_change parameter in scikit-learn’s MLPClassifier controls early stopping during training.

MLPClassifier is a multi-layer perceptron neural network for classification tasks. It uses backpropagation to optimize the weights of the network.

The n_iter_no_change parameter sets the number of iterations with no improvement to wait before stopping training early. This helps prevent overfitting and reduces unnecessary computation.

The default value for n_iter_no_change is 10. In practice, values between 5 and 20 are commonly used, depending on the dataset’s complexity and the network’s architecture.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPClassifier
from sklearn.metrics import accuracy_score

# Generate synthetic dataset
X, y = make_classification(n_samples=1000, n_features=20, n_informative=10,
                           n_redundant=5, n_classes=3, random_state=42)

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train with different n_iter_no_change values
n_iter_values = [5, 10, 20, 50]
accuracies = []

for n in n_iter_values:
    mlp = MLPClassifier(hidden_layer_sizes=(100, 50), max_iter=1000,
                        n_iter_no_change=n, random_state=42)
    mlp.fit(X_train, y_train)
    y_pred = mlp.predict(X_test)
    accuracy = accuracy_score(y_test, y_pred)
    accuracies.append(accuracy)
    print(f"n_iter_no_change={n}, Accuracy: {accuracy:.3f}, Iterations: {mlp.n_iter_}")

Running the example gives an output like:

n_iter_no_change=5, Accuracy: 0.895, Iterations: 197
n_iter_no_change=10, Accuracy: 0.895, Iterations: 202
n_iter_no_change=20, Accuracy: 0.895, Iterations: 212
n_iter_no_change=50, Accuracy: 0.895, Iterations: 268

The key steps in this example are:

Generate a synthetic multi-class classification dataset
Split the data into train and test sets
Train MLPClassifier models with different n_iter_no_change values
Evaluate the accuracy and number of iterations for each model

Tips and heuristics for setting n_iter_no_change:

Start with the default value of 10 and adjust based on model performance
Smaller values may lead to early stopping, potentially underfitting
Larger values allow more iterations, potentially improving performance but increasing training time
Monitor both accuracy and number of iterations to find a good balance

Issues to consider:

The optimal value depends on the dataset complexity and network architecture
Too small a value may stop training prematurely, while too large a value may lead to overfitting
Consider using validation data to determine the best value for your specific problem

See Also