Scikit-Learn multilabel_confusion_matrix() Metric

Multilabel classification models can be challenging to evaluate because each sample can have multiple labels. The multilabel_confusion_matrix() function in scikit-learn provides a way to compute confusion matrices for each label, helping to understand the performance of such models.

The multilabel_confusion_matrix() function returns a confusion matrix for each label, showing the counts of true positives, false positives, true negatives, and false negatives. This allows for a detailed analysis of how well the model predicts each label.

The calculation involves creating a separate confusion matrix for each label. A good performance is indicated by a high number of true positives and true negatives, whereas high numbers of false positives or false negatives suggest poor performance.

This metric is used for multilabel classification problems where each sample can have multiple labels. It is not suitable for single-label classification problems or when only overall performance metrics are needed.

from sklearn.datasets import make_multilabel_classification
from sklearn.model_selection import train_test_split
from sklearn.multioutput import MultiOutputClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import multilabel_confusion_matrix

# Generate synthetic multilabel dataset
X, y = make_multilabel_classification(n_samples=1000, n_classes=3, random_state=42)

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train a Logistic Regression classifier
clf = MultiOutputClassifier(LogisticRegression(max_iter=1000, random_state=42))
clf.fit(X_train, y_train)

# Predict on test set
y_pred = clf.predict(X_test)

# Calculate multilabel confusion matrix
mcm = multilabel_confusion_matrix(y_test, y_pred)
print(f"Multilabel Confusion Matrix:\n{mcm}")

Running the example gives an output like:

Multilabel Confusion Matrix:
[[[100  15]
  [ 25  60]]

 [[ 64  10]
  [ 17 109]]

 [[ 69  20]
  [ 17  94]]]

• Generate a synthetic multilabel classification dataset using make_multilabel_classification().
• Split the dataset into training and test sets using train_test_split().
• Train a LogisticRegression classifier wrapped in a MultiOutputClassifier on the training set.
• Use the trained classifier to make predictions on the test set with predict().
• Calculate the multilabel confusion matrix using multilabel_confusion_matrix() by comparing the predicted labels to the true labels.

1. Generate a synthetic multilabel classification dataset with 1000 samples and 3 labels using make_multilabel_classification().
2. Split the dataset into training and test sets using train_test_split(), with 80% of the data for training and 20% for testing.
3. Train a LogisticRegression classifier using the LogisticRegression class from scikit-learn, setting max_iter to 1000 to ensure convergence.
4. Use the trained classifier to make predictions on the test set by calling the predict() method with X_test.
5. Calculate the multilabel confusion matrix using the multilabel_confusion_matrix() function, passing in the true labels (y_test) and predicted labels (y_pred). The resulting confusion matrix for each label is printed, providing a detailed view of the classifier’s performance on each label.