Configure MLPClassifier "max_fun" Parameter

The max_fun parameter in scikit-learn’s MLPClassifier controls the maximum number of function evaluations allowed during training.

MLPClassifier is a multi-layer perceptron neural network for classification. It uses backpropagation with gradient descent or variants for optimization. The max_fun parameter limits the total number of loss function evaluations.

Setting max_fun can help control computational resources and prevent excessively long training times. It’s particularly useful when working with large datasets or complex network architectures.

The default value for max_fun is 15000. In practice, values between 5000 and 50000 are common, depending on the dataset size and model complexity.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPClassifier
from sklearn.metrics import accuracy_score
import time

# Generate synthetic dataset
X, y = make_classification(n_samples=1000, n_features=20, n_informative=10,
                           n_redundant=5, n_classes=3, random_state=42)

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train with different max_fun values
max_fun_values = [5000, 15000, 30000, 50000]
accuracies = []
training_times = []

for max_fun in max_fun_values:
    start_time = time.time()
    mlp = MLPClassifier(hidden_layer_sizes=(100,50), max_fun=max_fun, random_state=42)
    mlp.fit(X_train, y_train)
    training_time = time.time() - start_time

    y_pred = mlp.predict(X_test)
    accuracy = accuracy_score(y_test, y_pred)

    accuracies.append(accuracy)
    training_times.append(training_time)

    print(f"max_fun={max_fun}, Accuracy: {accuracy:.3f}, Training Time: {training_time:.2f} seconds")

Running the example gives an output like:

max_fun=5000, Accuracy: 0.895, Training Time: 0.73 seconds
max_fun=15000, Accuracy: 0.895, Training Time: 0.74 seconds
max_fun=30000, Accuracy: 0.895, Training Time: 0.73 seconds
max_fun=50000, Accuracy: 0.895, Training Time: 0.73 seconds

The key steps in this example are:

Generate a synthetic multi-class classification dataset
Split the data into train and test sets
Train MLPClassifier models with different max_fun values
Measure training time and evaluate accuracy for each model
Compare the results to understand the impact of max_fun

Some tips and heuristics for setting max_fun:

Start with the default value and adjust based on model convergence and training time
Increase max_fun if the model hasn’t converged and you have computational resources
Decrease max_fun if training time is too long or to prevent overfitting

Issues to consider:

Setting max_fun too low may result in underfitting if the model doesn’t have enough iterations to converge
Very high max_fun values may lead to overfitting or unnecessarily long training times
The optimal max_fun value depends on dataset complexity, model architecture, and other hyperparameters

See Also