Configure MLPRegressor "batch_size" Parameter

The batch_size parameter in scikit-learn’s MLPRegressor controls the number of samples used in each iteration during training.

MLPRegressor is a multi-layer perceptron regressor that uses backpropagation for training. It’s a type of neural network capable of learning non-linear relationships in data.

The batch_size parameter determines how many samples are used to estimate the gradient at each step. It affects both the model’s learning dynamics and computational efficiency.

By default, batch_size is set to ‘auto’, which uses the minimum of 200 or the number of samples. Common values range from 32 to 256, but can be larger for bigger datasets.

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPRegressor
from sklearn.metrics import mean_squared_error
import numpy as np

# Generate synthetic dataset
X, y = make_regression(n_samples=1000, n_features=10, noise=0.1, random_state=42)

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train with different batch_size values
batch_sizes = ['auto', 32, 64, 128, 256]
mse_scores = []

for batch_size in batch_sizes:
    mlp = MLPRegressor(hidden_layer_sizes=(100,), max_iter=1000, random_state=42, batch_size=batch_size)
    mlp.fit(X_train, y_train)
    y_pred = mlp.predict(X_test)
    mse = mean_squared_error(y_test, y_pred)
    mse_scores.append(mse)
    print(f"batch_size={batch_size}, MSE: {mse:.3f}")

# Find best batch_size
best_batch_size = batch_sizes[np.argmin(mse_scores)]
print(f"Best batch_size: {best_batch_size}")

Running the example gives an output like:

batch_size=auto, MSE: 30.530
batch_size=32, MSE: 1.046
batch_size=64, MSE: 1.372
batch_size=128, MSE: 4.541
batch_size=256, MSE: 29.503
Best batch_size: 32

The key steps in this example are:

Generate a synthetic regression dataset
Split the data into train and test sets
Train MLPRegressor models with different batch_size values
Evaluate the mean squared error of each model on the test set
Identify the best performing batch_size

Some tips and heuristics for setting batch_size:

Smaller batch sizes often lead to faster initial learning but more noisy updates
Larger batch sizes provide more stable gradient estimates but may converge slower
Try powers of 2 (32, 64, 128, 256) as they can be more computationally efficient

Issues to consider:

The optimal batch size can depend on the dataset size and complexity
Very small batch sizes may lead to unstable training
Very large batch sizes may cause the model to generalize poorly
There’s often a trade-off between training speed and model performance

See Also