The two errors are not interchangeable, and which one hurts more is a property of the problem rather than the model. A cancer screen missing a tumour (FN) is far worse than flagging a healthy patient for a follow-up (FP). A spam filter deleting a real email (FP) is far worse than letting spam through (FN).
Configuration
Visualization
N=200
Predicted Class
PositiveNegative
Actual Class
Positive
Negative
85
TP
Correctly predicted Positive
10
FN
Missed Positive (Type II Error)
15
FP
False Alarm (Type I Error)
90
TN
Correctly predicted Negative
Metrics
Accuracy--
Precision--
TP / (TP + FP)
Recall--
TP / (TP + FN)
F1 Score--
Error Rate: --
Specificity: --
Confusion Matrix Analysis: A Practical Guide
Four numbers - TP, FP, FN, TN - and every classification metric is a ratio of them. Knowing which ratio to care about is the actual skill; accuracy is usually the wrong one.
Read them as questions. Precision: of everything I flagged, how much was real? Recall: of everything real, how much did I catch? Precision is about the cost of false alarms; recall is about the cost of misses.
F1 is their harmonic mean, which — unlike an ordinary average — is dragged down hard by the smaller of the two. Precision 1.0 with recall 0.0 gives F1 = 0, not 0.5, which is the behaviour you want from a summary that should not reward ignoring one side entirely.
Why accuracy misleads
Take a dataset that is 99% negative — fraud detection, rare disease, defect inspection. A model that predicts “negative” for every single input scores 99% accuracy and has no value whatsoever: TP = 0, so precision and recall are both zero.
This is the accuracy paradox, and it is why accuracy is a poor default on any imbalanced problem. Precision, recall and F1 all ignore TN, which is exactly the cell that inflates accuracy when negatives dominate.
A worked example with 1,000 emails
Numbers make this concrete faster than definitions do. Say a spam filter is tested on 1,000 emails, of which 100 are genuinely spam.
Predicted spam
Predicted not spam
Actually spam
TP = 80
FN = 20
Actually not spam
FP = 30
TN = 870
Read the four cells as sentences and they stop being jargon:
Precision = 80 / (80 + 30) = 73%. Of everything sent to the spam folder, roughly a quarter was innocent.
Recall = 80 / (80 + 20) = 80%. Of all real spam, a fifth got through.
F1 = 2 × (0.73 × 0.80) / (0.73 + 0.80) = 76%.
One model, four numbers between 73% and 95%, and only one of them is comfortable. Which one you quote decides whether this filter sounds finished or unfinished — and for a spam filter, the 30 false positives are the number that matters, because losing an invoice costs more than deleting a junk email.
The threshold is a dial, not a fact
A classifier does not really output "spam" or "not spam". It outputs a score between 0 and 1, and somebody chose to call anything above 0.5 spam. That 0.5 is a business decision wearing a mathematical costume.
Slide the threshold up to 0.9 and the model only flags what it is very sure about: false positives fall, precision rises, and more spam slips through, so recall drops. Slide it down to 0.2 and the opposite happens.
This is why precision and recall almost always move in opposite directions, and why quoting one alone is meaningless — you can drive either to 100% by ignoring the other. A model that predicts "spam" for everything has 100% recall and useless precision.
Set the threshold from the cost of each mistake:
Cancer screening. A missed tumour is far worse than an unnecessary follow-up scan. Lower the threshold, accept false positives, protect recall.
Automated bank account closures. Wrongly freezing a legitimate customer is expensive and public. Raise the threshold, protect precision.
Spam. Somewhere in the middle, leaning towards precision, because a lost invoice beats a junk email in the inbox.
Which metric for which job
Metric
Formula
Ask it when
Blind to
Accuracy
(TP+TN)/all
Classes are balanced and errors cost the same
Rare classes entirely
Precision
TP/(TP+FP)
A false alarm is expensive
Everything you missed
Recall
TP/(TP+FN)
A miss is expensive
Every false alarm
F1
harmonic mean of the two
You need one number and both matter
The true negatives
Specificity
TN/(TN+FP)
You care about correctly clearing the negatives
The positive class
ROC-AUC
area under TPR/FPR curve
Comparing models across all thresholds
Severe imbalance can flatter it
PR-AUC
area under precision/recall curve
The positive class is rare
Nothing much — prefer it when imbalanced
The harmonic mean in F1 is not decoration. It punishes imbalance: precision 1.0 with recall 0.0 gives an ordinary average of 0.5 but an F1 of 0. That is the correct verdict on a model that never fires.
Beyond two classes
With three or more classes the matrix grows to N×N: rows are the truth, columns are the prediction, and the diagonal is what went right. Everything off the diagonal is a specific, nameable confusion — "sneakers predicted as sandals, 140 times" is a far more actionable finding than "87% accuracy".
Per-class precision and recall are computed by treating each class as the positive one in turn. Combining them into a single score has three common conventions:
Macro average — the plain mean across classes. Every class counts equally, so a rare class can drag the score down. Usually what you want when the rare classes matter.
Weighted average — weighted by how many examples each class has. Closer to accuracy, and it hides poor performance on small classes.
Micro average — pool all the TP, FP and FN first. In single-label problems this equals accuracy.
If a heatmap of the matrix shows one bright off-diagonal square, you have found a pair of classes the model cannot tell apart. That is a labelling or feature problem, and no amount of extra training rounds fixes it.
Read the four cells
Accuracy hides which mistakes a model makes. The confusion matrix does not, and on imbalanced data the difference is the whole story.
example_01.pyscikit-learn
import numpy as np
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix, accuracy_score
X, y = make_classification(n_samples=4000, n_features=12, n_informative=4,
weights=[0.94, 0.06], flip_y=0.03, random_state=0)
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0,
stratify=y)
print("test set: %d negatives, %d positives" % ((yte == 0).sum(), (yte == 1).sum()))
print()
pred_all_zero = np.zeros_like(yte)
model = LogisticRegression(max_iter=2000).fit(Xtr, ytr)
pred_model = model.predict(Xte)
for name, pred in (("predict 0 always", pred_all_zero),
("logistic regression", pred_model)):
tn, fp, fn, tp = confusion_matrix(yte, pred).ravel()
print("%s" % name)
print(" accuracy %.4f" % accuracy_score(yte, pred))
print(" predicted 0 predicted 1")
print(" actual 0 %12d %12d" % (tn, fp))
print(" actual 1 %12d %12d" % (fn, tp))
print(" true positives %d, false negatives %d" % (tp, fn))
print()
print("the do-nothing model scores well on accuracy and catches zero positives.")
print("that is why accuracy alone is not a report -- the matrix is.")
print()
print("the four cells, named:")
print(" TN correctly said no")
print(" FP false alarm (said yes, was no)")
print(" FN miss (said no, was yes)")
print(" TP correctly said yes")
print()
print("every other metric is built from these four numbers.")
Output
Experiments to try
Build the paradox. Set True Positives (TP) to 0, False Negatives (FN) to 10, False Positives (FP) to 0 and True Negatives (TN) to 990. Accuracy reads 99% while precision and recall are zero.
Trade one for the other. Raise False Positives (FP) while lowering False Negatives (FN). Recall climbs and precision falls — the trade-off you make by lowering a decision threshold.
Find where F1 peaks. Adjust until precision and recall are close. F1 is highest when they are balanced, and collapses when either is small.
Randomize and read. Press Randomize and predict which metric will look best before checking. Whichever cell is largest drives it.
Traps worth knowing
Reporting accuracy on imbalanced data. The single most common evaluation error.
Optimising precision or recall alone. Either can be made perfect trivially — predict positive for nothing, or for everything. They are only meaningful together.
Leaving the threshold at 0.5. The confusion matrix describes one threshold. Moving it moves every metric, and the right threshold comes from the relative cost of FP and FN.
Transposing the matrix. Libraries differ on whether rows are true or predicted labels. Check the axis before reading off FP and FN.
Using F1 when the errors have very different costs. F1 weights precision and recall equally; when they are not equally important, use Fβ or state the costs directly.
What to remember
The confusion matrix holds the four counts every classification metric is built from, and the choice of metric is a statement about which error costs more. Precision asks how many flagged items were real, recall asks how many real items were caught, and F1 balances them harmonically so neither can be ignored. Accuracy includes true negatives, which is why it looks excellent on imbalanced data where the model has learned nothing.
Questions people ask
Which comes first, precision or recall? Whichever error costs more. Write both mistakes as a sentence about a real person — "a customer is wrongly refused a loan" versus "a fraudster is approved" — and the priority usually decides itself.
My dataset is 99% negative. Is 99% accuracy good? It is exactly what a model that predicts "negative" for everything achieves. Ignore accuracy entirely on imbalanced data and read precision, recall and PR-AUC instead.
Can I have high precision and high recall at once? Only if the model is genuinely good and the classes are separable. Otherwise the threshold just moves the error from one column to the other. When both are stuck low, the fix is better features or more data, not a different threshold.
What is a good F1 score? There is no universal number. Compare against a baseline: the F1 of always predicting the positive class, and the F1 of the simplest sensible rule you can write by hand. A model that cannot beat a two-line rule is not ready.
Should the test set be balanced? No. The test set should look like production, imbalance and all, or your metrics describe a world that does not exist. Balance the training data if you must, never the evaluation data.
What does a confusion matrix on the training set tell me? Mostly how much the model memorised. Always read the matrix on held-out data; a perfect training matrix beside a mediocre test matrix is the signature of overfitting.
Recap in one screen
Four cells: correct positives, correct negatives, false alarms, misses. Every metric is a ratio of them.
Precision answers "when it fires, is it right?" Recall answers "does it catch everything?"
The threshold trades one for the other. Choose it from the cost of each mistake, not from the default 0.5.
Accuracy is safe only when the classes are balanced and the two errors cost about the same.
For rare positives, read PR-AUC and per-class recall. For many classes, read the off-diagonal cells by name.
Check yourself
0 of 3
Answer without scrolling back up.
A disease affects 1% of people. A model that always predicts 'healthy' scores 99% accuracy. What does the confusion matrix show?
The whole top row of the matrix is empty. Accuracy hides this completely, which is exactly why the matrix is worth reading on any imbalanced problem.
For a spam filter, which error is usually more costly?
A missed spam is an annoyance; a lost job offer is a disaster. This asymmetry is why you tune the threshold toward precision here, and toward recall for something like cancer screening.
Recall answers which question?
Recall divides by the actual positives, so it measures coverage of the true cases. Precision divides by your predicted positives and measures how trustworthy a flag is.
Cheat sheet
Confusion Matrix Analysis
The two errors are not interchangeable, and which one hurts more is a property of the problem rather than the model. A cancer screen missing a tumour (FN) is far worse than flagging a healthy patient for a follow-up (FP). A spam filter deleting a real email (FP) is far worse than letting spam through (FN).
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.