Configure SVR "tol" Parameter

The tol parameter in scikit-learn’s SVR class controls the tolerance for the optimization convergence.

Support Vector Regression (SVR) is a regression algorithm that tries to find a function that approximates the training data with a given tolerance. The tol parameter determines the tolerance on the optimization process.

A smaller tol leads to tighter convergence criteria and potentially longer training times. Conversely, a larger tol allows for looser approximations and faster training.

The default value for tol is 0.001.

In practice, values between 0.0001 and 0.01 are commonly used depending on the desired precision and computational resources available.

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.svm import SVR
from sklearn.metrics import mean_squared_error

# Generate synthetic dataset
X, y = make_regression(n_samples=1000, n_features=10, noise=0.1, random_state=42)

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train with different tol values
tol_values = [0.1, 0.01, 0.001, 0.0001]
mse_scores = []

for tol in tol_values:
    svr = SVR(tol=tol)
    svr.fit(X_train, y_train)
    y_pred = svr.predict(X_test)
    mse = mean_squared_error(y_test, y_pred)
    mse_scores.append(mse)
    print(f"tol={tol}, MSE: {mse:.3f}")

Running the example gives an output like:

tol=0.1, MSE: 12758.146
tol=0.01, MSE: 12758.146
tol=0.001, MSE: 12758.146
tol=0.0001, MSE: 12758.146

The key steps in this example are:

Generate a synthetic regression dataset with noise
Split the data into train and test sets
Train SVR models with different tol values
Evaluate the mean squared error (MSE) of each model on the test set

Some tips and heuristics for setting tol:

Smaller tol leads to stricter convergence criteria
Smaller tol can lead to longer training times
Larger tol may result in looser approximations

Issues to consider:

Setting tol too small may cause the optimizer to run for a long time
Setting tol too large may yield a suboptimal model
The effect of tol depends on the scale of the target values

See Also