Scikit-Learn QuantileRegressor Model

Quantile regression is used for predicting specific quantiles in a regression problem, helping to understand the variability and distribution of the target variable.

The QuantileRegressor class in scikit-learn estimates the conditional quantiles of a response variable distribution in the linear model.

Key hyperparameters include:

• quantile: The quantile to be predicted, between 0 and 1.
• alpha: Regularization strength; a positive float.

This algorithm is appropriate for regression problems where predicting different quantiles of the target variable is important.

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.linear_model import QuantileRegressor
from sklearn.metrics import mean_absolute_error

# generate regression dataset
X, y = make_regression(n_samples=100, n_features=5, noise=0.1, random_state=1)

# split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)

# create model
model = QuantileRegressor(quantile=0.5, alpha=1.0)

# fit model
model.fit(X_train, y_train)

# evaluate model
yhat = model.predict(X_test)
mae = mean_absolute_error(y_test, yhat)
print('Mean Absolute Error: %.3f' % mae)

# make a prediction
row = [[0.5, -0.5, 0.3, -0.2, 0.1]]
yhat = model.predict(row)
print('Predicted: %.3f' % yhat[0])

Running the example gives an output like:

Mean Absolute Error: 59.300
Predicted: 9.967

The steps are as follows:

1. Generate a synthetic regression dataset using the make_regression() function, which creates a dataset with specified samples (n_samples), features (n_features), and noise level (noise). The dataset is split into training and test sets using train_test_split().

2. Instantiate a QuantileRegressor model with the quantile parameter set to 0.5 (median) and the alpha parameter for regularization strength.

3. Fit the model on the training data using the fit() method.

4. Evaluate the model’s performance by predicting on the test set and calculating the mean absolute error (MAE) using the mean_absolute_error() function.

5. Make a single prediction by passing a new data sample to the predict() method.

This example demonstrates how to set up and use a QuantileRegressor for predicting specific quantiles in regression tasks, providing insights into the distribution of the target variable.