Confusion Matrix

A Confusion Matrix is a powerful tool used to evaluate the performance of a classification model. While a single metric like "accuracy" can be misleading, a confusion matrix provides a detailed breakdown of a model's correct and incorrect predictions, offering a much clearer picture of its strengths and weaknesses. This interactive lab allows you to build your own confusion matrix and see how different types of errors affect key performance metrics.

The Four Quadrants of Prediction

For a binary classification problem (e.g., "spam" or "not spam"), the confusion matrix is a 2x2 table that organizes predictions into four categories:

True Positive (TP)

Correctly Identified. The model correctly predicted the positive class. (e.g., correctly identified an email as spam).

True Negative (TN)

Correctly Rejected. The model correctly predicted the negative class. (e.g., correctly identified an email as not spam).

False Positive (FP) - Type I Error

False Alarm. The model incorrectly predicted the positive class. (e.g., a legitimate email was flagged as spam).

False Negative (FN) - Type II Error

A Miss. The model incorrectly predicted the negative class. (e.g., a spam email was missed and went to the inbox).

Key Performance Metrics

From these four values, we can calculate several important metrics, which you can see updating live in the "Metrics" panel.

Accuracy: The percentage of total predictions that were correct. It's a good general measure, but can be very misleading if your classes are imbalanced.
(TP + TN) / (Total)
Precision: Of all the times the model predicted "Positive," how often was it correct? High precision is crucial when the cost of a False Positive is high (e.g., flagging a clean file as a virus).
TP / (TP + FP)
Recall (Sensitivity): Of all the actual "Positive" cases, how many did the model correctly identify? High recall is vital when the cost of a False Negative is high (e.g., failing to detect a fraudulent transaction).
TP / (TP + FN)
F1-Score: The harmonic mean of Precision and Recall. It provides a single score that balances both concerns, and is often a better measure than accuracy on imbalanced datasets.
2 * (Precision * Recall) / (Precision + Recall)

Guided Experiments

Use the controls to understand the impact of different error types.

The Problem with Accuracy: Set TP=10, TN=90, FP=0, and FN=0. Accuracy is 100%. Now, change FN to 90. The model is now missing 90% of all positive cases! But the accuracy is still 50%, which sounds okay. Notice how the Recall has plummeted to 10%, correctly telling you the model is failing at its main job. This shows why accuracy is not enough.
Precision vs. Recall Tradeoff: Imagine a medical test. Set TP=95, TN=95, FP=5, FN=5. Precision and Recall are both high (95%). Now, imagine you want to be absolutely sure you don't miss any sick patients (minimize False Negatives). Decrease FN to 0 and increase TP to 100. Your Recall is now 100%! But to achieve this, your model became more aggressive, and FP might have increased to 15. Now your Precision has dropped because you have more false alarms. This is the classic Precision-Recall tradeoff.
Use the Scenarios: Click the "Perf" (Perfect), "Bias", and "Poor" scenario buttons. Observe how the matrix and the metrics change for each one. A "Biased" model might always predict one class, leading to strange metric values.

Key Takeaways

Accuracy Isn't Everything: Always look at the full confusion matrix, especially when you have imbalanced classes or different costs for different errors.
Choose the Right Metric for the Job: If the cost of False Positives is high, optimize for Precision. If the cost of False Negatives is high, optimize for Recall.
F1-Score is a Balanced View: When you need a balance between Precision and Recall, the F1-Score is your go-to metric.
Context is Crucial: The "best" model depends entirely on the problem you are trying to solve. A spam filter can tolerate some False Negatives (spam in your inbox), but a medical diagnostic tool cannot.

85 TP Correctly predicted Positive	10 FN Missed Positive (Type II Error)
15 FP False Alarm (Type I Error)	90 TN Correctly predicted Negative

Confusion Matrix

Configuration

Visualization

Metrics

Decoding the Confusion Matrix