Configure SVC "kernel" Parameter

The SVC (Support Vector Classifier) is a powerful algorithm for binary and multi-class classification tasks in scikit-learn. The kernel parameter of SVC determines the type of kernel function used to transform the input space, enabling the algorithm to learn non-linear decision boundaries.

The kernel parameter accepts four main options: 'linear', 'poly' (polynomial), 'rbf' (radial basis function), and 'sigmoid'. Each kernel function has its own characteristics and effects on the decision boundary.

The default value for the kernel parameter is 'rbf', which is a good default choice for many datasets. The 'linear' kernel is often used for linearly separable data, while the 'poly' and 'sigmoid' kernels are less commonly used.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score

# Generate synthetic dataset with overlapping classes
X, y = make_classification(n_samples=1000, n_features=10, n_informative=5,
                           n_redundant=0, n_clusters_per_class=2, random_state=42)

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train with different kernel values
kernels = ['linear', 'poly', 'rbf', 'sigmoid']
accuracies = []

for kernel in kernels:
    svc = SVC(kernel=kernel, random_state=42)
    svc.fit(X_train, y_train)
    y_pred = svc.predict(X_test)
    accuracy = accuracy_score(y_test, y_pred)
    accuracies.append(accuracy)
    print(f"kernel='{kernel}', Accuracy: {accuracy:.3f}")

Running the example gives an output like:

kernel='linear', Accuracy: 0.780
kernel='poly', Accuracy: 0.830
kernel='rbf', Accuracy: 0.905
kernel='sigmoid', Accuracy: 0.645

The key steps in this example are:

Generate a synthetic binary classification dataset with overlapping classes
Split the data into train and test sets
Train SVC models with different kernel values
Evaluate the accuracy of each model on the test set

Some tips and heuristics for choosing the appropriate kernel:

Use the linear kernel for linearly separable data
The RBF kernel is a good default choice for non-linear decision boundaries
The polynomial kernel can capture feature interactions
The sigmoid kernel has similarities to neural networks

Issues to consider when selecting the kernel:

The choice of kernel depends on the data distribution and problem complexity
RBF and polynomial kernels require tuning additional parameters (gamma, degree)
The linear kernel is computationally efficient but may underfit complex datasets
The sigmoid kernel is less commonly used and may not always converge

See Also