SKLearner Home | About | Contact | Examples

Scikit-Learn Gaussian Process with "Sum" Kernel

Gaussian Process (GP) is a powerful probabilistic model used for regression and classification tasks. It is particularly useful when dealing with small datasets or when a measure of uncertainty is required for predictions.

The “Sum” kernel is a composite covariance function used in GP that combines two or more kernels to capture different aspects of the data. For instance, combining an RBF kernel with a WhiteKernel allows modeling both smooth variations and noise in the data. This kernel is versatile and can handle more complex relationships between inputs and outputs.

Key hyperparameters for the “Sum” kernel include the length scale of the RBF kernel, which controls the smoothness of the function, and the noise level of the WhiteKernel, which accounts for observation noise. Common values for the length scale range from 1 to 10, and the noise level is typically set to a small value like 1e-2.

The “Sum” kernel is appropriate for both regression and classification problems where multiple aspects of the data need to be modeled simultaneously.

from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RBF, WhiteKernel, Sum
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
import numpy as np

# Prepare a synthetic dataset
X = np.random.uniform(low=-5, high=5, size=(100, 3))
y = np.sin(X[:, 0]) + X[:, 1]**2 - 0.5 * X[:, 2] + np.random.normal(loc=0, scale=0.1, size=(100,))

# Create an instance of GaussianProcessRegressor with Sum kernel
kernel = Sum(RBF(length_scale=1.0), WhiteKernel(noise_level=1e-2))
gp = GaussianProcessRegressor(kernel=kernel, random_state=0)

# Split the dataset into train and test portions
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

# Fit the model on the training data
gp.fit(X_train, y_train)

# Evaluate the model's performance using mean squared error
y_pred = gp.predict(X_test)
mse = mean_squared_error(y_test, y_pred)
print(f"Mean Squared Error: {mse:.2f}")

# Make a prediction using the fitted model on a test sample
test_sample = np.array([[1, -2, 3]])
pred = gp.predict(test_sample)
print(f"Predicted value for test sample: {pred[0]:.2f}")

Running the example gives an output like:

Mean Squared Error: 65.81
Predicted value for test sample: 4.27

The key steps in this code example are:

  1. Dataset preparation: A synthetic dataset is generated where the target variable is a combination of nonlinear relationships and noise.

  2. Model instantiation and configuration: An instance of GaussianProcessRegressor is created with the Sum kernel, combining RBF and WhiteKernel, and relevant hyperparameters are set.

  3. Model training: The dataset is split into train and test portions, and the model is fitted on the training data.

  4. Model evaluation: The model’s performance is evaluated using mean squared error on the test set.

  5. Inference on test sample(s): A prediction is made using the fitted model on one test sample, demonstrating how the model can be used for inference on new data.



See Also