Scikit-Learn "LinearSVR" versus "SVR"

This example demonstrates the use of LinearSVR and SVR for regression tasks, highlighting their key differences and comparing their performance on a synthetic dataset.

LinearSVR is a linear support vector regression model. Key hyperparameters include C (regularization parameter) and epsilon (epsilon-tube within which no penalty is associated).

SVR is a kernel-based support vector regression model. Key hyperparameters include C, epsilon, and kernel (type of kernel function, e.g., ’linear’, ‘rbf’).

The main difference is that LinearSVR uses a linear kernel, which makes it faster and simpler but potentially less flexible than SVR with non-linear kernels.

LinearSVR is suitable for linear relationships and is computationally efficient. SVR, with its flexibility of choosing different kernels, is better suited for capturing complex patterns in data.

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.svm import LinearSVR, SVR
from sklearn.metrics import mean_squared_error

# Generate synthetic regression dataset
X, y = make_regression(n_samples=1000, n_features=20, noise=0.1, random_state=42)

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Fit and evaluate LinearSVR
linear_svr = LinearSVR(random_state=42)
linear_svr.fit(X_train, y_train)
y_pred_linear_svr = linear_svr.predict(X_test)
print(f"LinearSVR MSE: {mean_squared_error(y_test, y_pred_linear_svr):.3f}")

# Fit and evaluate SVR with RBF kernel
svr = SVR(kernel='rbf')
svr.fit(X_train, y_train)
y_pred_svr = svr.predict(X_test)
print(f"SVR MSE: {mean_squared_error(y_test, y_pred_svr):.3f}")

Running the example gives an output like:

LinearSVR MSE: 0.013
SVR MSE: 35191.276

The steps are as follows:

1. Generate a synthetic regression dataset using make_regression.
2. Split the data into training and test sets using train_test_split.
3. Instantiate LinearSVR with default hyperparameters, fit it on the training data, and evaluate its performance on the test set.
4. Instantiate SVR with an RBF kernel, fit it on the training data, and evaluate its performance on the test set.
5. Compare the test set performance (mean squared error) of both models.