Configure SVR "kernel" Parameter

The kernel parameter in scikit-learn’s SVR (Support Vector Regression) class determines the type of kernel function used to transform the input data into a higher-dimensional space.

SVR is a regression algorithm that tries to find a hyperplane in a high-dimensional space that has at most a given deviation (epsilon) from the target values for all training data points.

The kernel parameter specifies the kernel type to be used in the algorithm. It can be set to ’linear’, ‘poly’, ‘rbf’, or ‘sigmoid’. The choice of kernel function can significantly impact the model’s performance.

The default value for kernel is ‘rbf’, which stands for Radial Basis Function. This kernel is a good default choice as it can handle non-linear relationships between features and target.

Other commonly used kernels are ’linear’ for linearly separable data and ‘poly’ for polynomial kernel.

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.svm import SVR
from sklearn.metrics import mean_squared_error

# Generate synthetic regression dataset
X, y = make_regression(n_samples=100, n_features=1, noise=10, random_state=42)

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train with different kernel values
kernels = ['linear', 'poly', 'rbf', 'sigmoid']
mses = []

for kernel in kernels:
    svr = SVR(kernel=kernel)
    svr.fit(X_train, y_train)
    y_pred = svr.predict(X_test)
    mse = mean_squared_error(y_test, y_pred)
    mses.append(mse)
    print(f"kernel={kernel}, MSE: {mse:.3f}")

Running the example gives an output like:

kernel=linear, MSE: 173.525
kernel=poly, MSE: 663.069
kernel=rbf, MSE: 901.656
kernel=sigmoid, MSE: 437.687

The key steps in this example are:

Generate a synthetic regression dataset with one feature
Split the data into train and test sets
Train SVR models with different kernel values
Evaluate the mean squared error of each model on the test set

Some tips and heuristics for setting kernel:

‘rbf’ is a good default choice as it can handle non-linear relationships
’linear’ works well when the relationship between features and target is linear
‘poly’ can be useful for modeling polynomial relationships
‘sigmoid’ behaves similarly to ‘rbf’ but is less commonly used

Issues to consider:

The choice of kernel should be based on prior knowledge about the data if available
Using an inappropriate kernel can lead to poor model performance
Some kernels, like ‘rbf’ and ‘poly’, have additional parameters that may need tuning

See Also