ElasticNet is a linear regression model that combines L1 (Lasso) and L2 (Ridge) regularization to handle high-dimensional datasets with correlated features.
The key hyperparameters of ElasticNet are l1_ratio
(the mixing parameter between L1 and L2 penalties) and alpha
(the strength of the regularization).
ElasticNetCV extends ElasticNet by automatically tuning the alpha
parameter using cross-validation. Its key hyperparameters include l1_ratio
, alphas
(the list of alpha
values to try), and cv
(the number of folds for cross-validation).
The main difference is that ElasticNetCV automates the alpha
tuning process, while ElasticNet requires manual tuning. This automation comes at a computational cost, as ElasticNetCV trains multiple models during cross-validation.
ElasticNet is preferred for quick prototyping or when you have prior knowledge of a good alpha
value. ElasticNetCV is ideal when you need to tune alpha
and perform model selection, especially with new datasets.
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.linear_model import ElasticNet, ElasticNetCV
from sklearn.metrics import mean_squared_error, r2_score
# Generate synthetic regression dataset
X, y = make_regression(n_samples=1000, n_features=100, noise=0.5, random_state=42)
# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Fit and evaluate ElasticNet with default hyperparameters
en = ElasticNet(random_state=42)
en.fit(X_train, y_train)
y_pred_en = en.predict(X_test)
print(f"ElasticNet MSE: {mean_squared_error(y_test, y_pred_en):.3f}")
print(f"ElasticNet R^2: {r2_score(y_test, y_pred_en):.3f}")
# Fit and evaluate ElasticNetCV with cross-validation
encv = ElasticNetCV(cv=5, random_state=42)
encv.fit(X_train, y_train)
y_pred_encv = encv.predict(X_test)
print(f"\nElasticNetCV MSE: {mean_squared_error(y_test, y_pred_encv):.3f}")
print(f"ElasticNetCV R^2: {r2_score(y_test, y_pred_encv):.3f}")
print(f"Best hyperparameters: {encv.get_params()}")
Running the example gives an output like:
ElasticNet MSE: 4075.345
ElasticNet R^2: 0.865
ElasticNetCV MSE: 238.377
ElasticNetCV R^2: 0.992
Best hyperparameters: {'alphas': None, 'copy_X': True, 'cv': 5, 'eps': 0.001, 'fit_intercept': True, 'l1_ratio': 0.5, 'max_iter': 1000, 'n_alphas': 100, 'n_jobs': None, 'positive': False, 'precompute': 'auto', 'random_state': 42, 'selection': 'cyclic', 'tol': 0.0001, 'verbose': 0}
The key steps are:
- Generate a synthetic regression dataset with correlated features using
make_regression
. - Split the data into training and test sets using
train_test_split
. - Instantiate
ElasticNet
with default hyperparameters, fit it on the training data, and evaluate its performance on the test set. - Instantiate
ElasticNetCV
with 5-fold cross-validation, fit it on the training data, and evaluate its performance on the test set. - Compare the test set performance (MSE and R^2) of both models and print the best hyperparameters found by
ElasticNetCV
.