SKLearner Home | About | Contact | Examples

Configure SGDClassifier "n_iter_no_change" Parameter

The n_iter_no_change parameter in scikit-learn’s SGDClassifier controls the early stopping criterion based on the number of consecutive iterations without improvement.

Stochastic Gradient Descent (SGD) is an iterative optimization algorithm used for training various linear models. The SGDClassifier implements SGD for classification tasks, updating model parameters based on one sample at a time.

The n_iter_no_change parameter determines how many iterations to continue training without improvement before stopping. This helps prevent overfitting and reduces unnecessary computation when the model has converged.

The default value for n_iter_no_change is 5.

In practice, values between 2 and 10 are commonly used, depending on the dataset size and complexity.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import SGDClassifier
from sklearn.metrics import accuracy_score
import numpy as np

# Generate synthetic dataset
X, y = make_classification(n_samples=10000, n_features=20, n_informative=10,
                           n_redundant=5, n_classes=3, random_state=42)

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train with different n_iter_no_change values
n_iter_no_change_values = [2, 5, 10, 20]
accuracies = []

for n in n_iter_no_change_values:
    sgd = SGDClassifier(loss='log_loss', n_iter_no_change=n, random_state=42)
    sgd.fit(X_train, y_train)
    y_pred = sgd.predict(X_test)
    accuracy = accuracy_score(y_test, y_pred)
    accuracies.append(accuracy)
    print(f"n_iter_no_change={n}, Accuracy: {accuracy:.3f}, n_iter_: {sgd.n_iter_}")

Running the example gives an output like:

n_iter_no_change=2, Accuracy: 0.674, n_iter_: 72
n_iter_no_change=5, Accuracy: 0.669, n_iter_: 106
n_iter_no_change=10, Accuracy: 0.679, n_iter_: 165
n_iter_no_change=20, Accuracy: 0.674, n_iter_: 248

The key steps in this example are:

  1. Generate a synthetic multi-class classification dataset
  2. Split the data into train and test sets
  3. Train SGDClassifier models with different n_iter_no_change values
  4. Evaluate the accuracy of each model on the test set

Some tips and heuristics for setting n_iter_no_change:

Issues to consider:



See Also