SKLearner Home | About | Contact | Examples

Scikit-Learn "ElasticNet" versus "ElasticNetCV"

ElasticNet is a linear regression model that combines L1 (Lasso) and L2 (Ridge) regularization to handle high-dimensional datasets with correlated features.

The key hyperparameters of ElasticNet are l1_ratio (the mixing parameter between L1 and L2 penalties) and alpha (the strength of the regularization).

ElasticNetCV extends ElasticNet by automatically tuning the alpha parameter using cross-validation. Its key hyperparameters include l1_ratio, alphas (the list of alpha values to try), and cv (the number of folds for cross-validation).

The main difference is that ElasticNetCV automates the alpha tuning process, while ElasticNet requires manual tuning. This automation comes at a computational cost, as ElasticNetCV trains multiple models during cross-validation.

ElasticNet is preferred for quick prototyping or when you have prior knowledge of a good alpha value. ElasticNetCV is ideal when you need to tune alpha and perform model selection, especially with new datasets.

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.linear_model import ElasticNet, ElasticNetCV
from sklearn.metrics import mean_squared_error, r2_score

# Generate synthetic regression dataset
X, y = make_regression(n_samples=1000, n_features=100, noise=0.5, random_state=42)

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Fit and evaluate ElasticNet with default hyperparameters
en = ElasticNet(random_state=42)
en.fit(X_train, y_train)
y_pred_en = en.predict(X_test)
print(f"ElasticNet MSE: {mean_squared_error(y_test, y_pred_en):.3f}")
print(f"ElasticNet R^2: {r2_score(y_test, y_pred_en):.3f}")

# Fit and evaluate ElasticNetCV with cross-validation
encv = ElasticNetCV(cv=5, random_state=42)
encv.fit(X_train, y_train)
y_pred_encv = encv.predict(X_test)
print(f"\nElasticNetCV MSE: {mean_squared_error(y_test, y_pred_encv):.3f}")
print(f"ElasticNetCV R^2: {r2_score(y_test, y_pred_encv):.3f}")
print(f"Best hyperparameters: {encv.get_params()}")

Running the example gives an output like:

ElasticNet MSE: 4075.345
ElasticNet R^2: 0.865

ElasticNetCV MSE: 238.377
ElasticNetCV R^2: 0.992
Best hyperparameters: {'alphas': None, 'copy_X': True, 'cv': 5, 'eps': 0.001, 'fit_intercept': True, 'l1_ratio': 0.5, 'max_iter': 1000, 'n_alphas': 100, 'n_jobs': None, 'positive': False, 'precompute': 'auto', 'random_state': 42, 'selection': 'cyclic', 'tol': 0.0001, 'verbose': 0}

The key steps are:

  1. Generate a synthetic regression dataset with correlated features using make_regression.
  2. Split the data into training and test sets using train_test_split.
  3. Instantiate ElasticNet with default hyperparameters, fit it on the training data, and evaluate its performance on the test set.
  4. Instantiate ElasticNetCV with 5-fold cross-validation, fit it on the training data, and evaluate its performance on the test set.
  5. Compare the test set performance (MSE and R^2) of both models and print the best hyperparameters found by ElasticNetCV.


See Also