Configure SVC "coef0" Parameter

The coef0 parameter in scikit-learn’s SVC class controls the independent term in the kernel function for polynomial and sigmoid kernels.

Support Vector Machines (SVMs) are powerful classification algorithms that can handle complex non-linear decision boundaries by using kernel functions to transform the input space.

The coef0 parameter is used with the polynomial (poly) and sigmoid (sigmoid) kernels. It represents the trade-off between the influence of higher-order versus lower-order terms in the polynomial.

The default value for coef0 is 0.0.

In practice, values between 0.0 and 1.0 are commonly used, with higher values giving more weight to higher-order polynomial terms.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score

# Generate synthetic dataset
X, y = make_classification(n_samples=1000, n_features=10, n_informative=5,
                           n_redundant=0, n_clusters_per_class=1, random_state=42)

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train with different coef0 values
coef0_values = [0.0, 0.5, 1.0]
accuracies = []

for coef0 in coef0_values:
    svc = SVC(kernel='poly', degree=3, coef0=coef0, random_state=42)
    svc.fit(X_train, y_train)
    y_pred = svc.predict(X_test)
    accuracy = accuracy_score(y_test, y_pred)
    accuracies.append(accuracy)
    print(f"coef0={coef0}, Accuracy: {accuracy:.3f}")

Running the example gives an output like:

coef0=0.0, Accuracy: 0.970
coef0=0.5, Accuracy: 0.970
coef0=1.0, Accuracy: 0.980

The key steps in this example are:

Generate a synthetic binary classification dataset with informative features and non-linearly separable classes
Split the data into train and test sets
Train SVC models with polynomial kernel and different coef0 values
Evaluate the accuracy of each model on the test set

Some tips and heuristics for setting coef0:

The choice between polynomial and sigmoid kernel depends on the nature of the data and should be experimented with
Start with the default value of 0.0 and tune it if the model performance is not satisfactory
Try values between 0.0 and 1.0 to adjust the influence of higher-order polynomial terms
Use cross-validation to select the optimal coef0 value for your dataset

Issues to consider:

Higher coef0 values can lead to overfitting, especially with high-degree polynomials
The computational cost of training and prediction increases with higher degree polynomials
The polynomial kernel degree and coef0 interact with each other and should be tuned together for best results

See Also