Precision, Recall and F1

Move the threshold and watch the confusion matrix rearrange itself. One model, and every score in the table changes.

Overview

One model, many scores

A classifier does not output a class. It outputs a score — how positive it thinks each case is — and a class only appears once you compare that score against a threshold.

The histogram shows both distributions: actual negatives in grey, actual positives in orange, overlapping in the middle because the model is good and not perfect. The dashed line is the threshold.

Move it. Every number below the chart changes, and the model has not been retrained. The scores are properties of the threshold as much as of the model, which is the single most useful thing to understand here.

Precision, Recall and F1

This module needs JavaScript: the numbers are computed in the page rather than recorded.

Worth knowing

Precision: of the things you flagged, how many were right. It punishes false positives.
Recall: of the things that were there, how many you found. It punishes false negatives.
They trade off. Lower the threshold and recall rises while precision falls; raise it and the reverse.
F1 is their harmonic mean, which stays low unless both are high. An arithmetic mean would not.

Precision, Recall and F1

Four counts in a table, three ratios drawn from them, and one slider that moves all of it.

The four counts

Predicted positivePredicted negative
Actually positiveTrue positiveFalse negative
Actually negativeFalse positiveTrue negative

Everything else is a ratio of these. The two that matter most divide by different denominators, which is exactly why they disagree.

Precision = TP / (TP + FP) — of everything you flagged, what fraction was right. The denominator is what *you* predicted, so precision is damaged by false alarms.

Recall = TP / (TP + FN) — of everything that was actually there, what fraction you caught. The denominator is what *reality* contained, so recall is damaged by misses.

Why they trade off

Drag the threshold to 0.10. Almost everything is flagged, so almost every real positive is caught and recall approaches 100%. But the flagged set is now full of negatives, and precision collapses.

Drag it to 0.90. Only the most confident cases are flagged, nearly all of them correct, and precision approaches 100%. But most real positives are below the line, and recall collapses.

You cannot maximise both by moving the threshold. The threshold only chooses where on the trade-off you sit. Improving both at once requires a better model — one whose two distributions overlap less.

Which one you want depends on the cost

The question is never "which metric is best". It is which mistake is more expensive.

Recall matters more when a miss is costly: screening for a serious disease, detecting fraud, finding safety defects. A false alarm costs a second look. A miss costs the thing you were trying to prevent.

Precision matters more when a false alarm is costly: flagging accounts for suspension, recommending content, sending an alert that wakes someone. Missing one is a shame. Being wrong repeatedly destroys trust in the system.

F1, and why it is a harmonic mean

F1 combines them:

F1 = 2 * precision * recall / (precision + recall)

That is the harmonic mean, and the choice of mean is the point. Take precision of 1.0 and recall of 0.01 — a model that flags exactly one case and gets it right. The arithmetic mean is 0.505, which sounds respectable. F1 is 0.0198.

The harmonic mean is pulled toward the smaller number, so it stays low unless both are high. That makes it hard to game with a degenerate model, which is precisely what a single summary number needs to be.

F1 weights the two equally, which is a decision and often the wrong one. F-beta lets you weight recall β times as much as precision; F2 favours recall, F0.5 favours precision. If you have a reason to prefer one, use it.

Why accuracy is on the readout and not in the headline

Accuracy is (TP + TN) / everything. It is the most intuitive metric and it is close to useless when the classes are unbalanced.

If 1% of transactions are fraudulent, predicting "never fraud" scores 99% accuracy while catching nothing at all. Precision and recall are both zero, which is the honest description. [Imbalanced data](precision_recall_vs_roc.html) is where this matters most.

Where it goes wrong

Reporting one number without the threshold. "Precision 0.9" is uninterpretable alone; at some threshold nearly every model achieves it.

Optimising F1 when the costs are not equal. F1 assumes they are.

Comparing models at 0.5. Two models can rank cases equally well and differ only in calibration. Compare across thresholds, or tune each one.

Accuracy on imbalanced data. It measures the class balance more than the model.

Check yourself

0 of 3

Answer without scrolling back up.

  1. What happens to precision and recall as the threshold falls?

  2. Why is F1 a harmonic rather than arithmetic mean?

  3. Why is accuracy misleading when 1% of cases are positive?

Cheat sheet

Precision, Recall and F1

A classifier does not output a class. It outputs a score — how positive it thinks each case is — and a class only appears once you compare that score against a threshold.

MACHINE LEARNING · vizlearn.in/machine_learning/precision_recall_and_f1.html

About the author

Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.