vizlearn

Precision-Recall against ROC

Machine LearningBy Updated

The same model on the same scores. Drag the positive rate down and watch ROC stay flattering while the PR curve tells the truth.

Overview

Start balanced

Leave the slider at 0.5. Half the rows are positive, and the two curves broadly agree: ROC bows toward the top left, PR bows toward the top right, and both say the model is decent.

Now drag the slider down toward 0.02, which is a realistic rate for fraud, disease screening or ad clicks. Watch the two charts diverge.

ROC barely moves. The AUC in the readout stays high.

PR collapses. The precision that was achievable at any useful recall falls away, and the best F1 falls with it.

The scores did not change. The model did not change. Only the proportion of positives changed, and the two curves disagree about whether that matters.

Precision-Recall against ROC

This module needs JavaScript: the numbers are computed in the page rather than recorded.

Worth knowing

ROC plots true positive rate against false positive rate. Both are computed within a class, so neither depends on the balance.
That independence is the problem: on imbalanced data ROC looks good for a model that is not useful.
The PR curve uses precision, whose denominator mixes both classes, so it falls when positives are rare.
The no-skill baseline is the diagonal for ROC and a horizontal line at the positive rate for PR.

Precision-Recall against ROC

Two curves over identical scores, and the reason one of them flatters a model that cannot be deployed.

Why ROC does not notice

ROC plots true positive rate against false positive rate:

TPR = TP / (TP + FN)     denominator: all actual positives
FPR = FP / (FP + TN)     denominator: all actual negatives

Each is computed entirely within one class. TPR asks what fraction of positives were caught; FPR asks what fraction of negatives were wrongly flagged. Adding a million more negatives leaves TPR untouched and changes FPR only through its own denominator, which grows in proportion.

That class-independence is often described as a virtue, and for some purposes it is. It also means ROC cannot see the thing that makes rare-positive problems hard.

Why PR does notice

Precision is different:

precision = TP / (TP + FP)     denominator: everything you flagged

The denominator mixes both classes. When negatives outnumber positives a hundred to one, even a small false positive *rate* produces a large *number* of false positives, and those go straight into precision's denominator.

Concretely: 10,000 rows, 100 positive. A model at 90% recall and 5% FPR catches 90 real positives and flags 495 negatives. FPR of 5% sounds excellent and ROC records it as such. Precision is 90 / 585 = 15%. Five out of six flagged cases are wrong, and every one costs somebody an investigation.

Reading the baselines

Each chart has a dashed no-skill line, and they differ.

For ROC it is the diagonal, always. A random classifier gets AUC 0.5 regardless of balance.

For PR it is a horizontal line at the positive rate. At 50% positives, random scores 0.5 precision. At 2%, random scores 0.02. Drag the slider and watch it drop.

This is what makes PR honest and slightly harder to read: there is no fixed scale. A PR AUC of 0.4 is poor on balanced data and outstanding at a 2% base rate. The number must always be compared against the baseline, never quoted alone.

Which to use

SituationCurveReason
Roughly balancedeitherthey broadly agree
Rare positivesPRROC hides the false-positive volume
You care about ranking overallROCit is what AUC measures
Someone acts on each flagPRprecision is that person's experience
Comparing across datasetsROC, carefullyPR baselines differ by base rate

The practical rule: if a human or a process has to act on every positive prediction, precision is the thing they experience, and the PR curve is the one to look at.

Where it goes wrong

Reporting ROC AUC alone on imbalanced data. It is the standard way to make a model that will drown a team in false positives look ready to ship.

Comparing PR AUC across datasets with different base rates. The baselines are different, so the numbers are not comparable.

Forgetting that both curves span all thresholds. Neither tells you which threshold to deploy — that is a separate decision, made with [costs](threshold_tuning.html).

Check yourself

0 of 3

Answer without scrolling back up.

  1. Why does ROC stay flattering as positives become rare?

  2. 10,000 rows, 100 positive, 90% recall and 5% false positive rate. What is precision?

  3. What is the no-skill baseline on a PR curve?

Cheat sheet

Precision-Recall against ROC

Leave the slider at 0.5. Half the rows are positive, and the two curves broadly agree: ROC bows toward the top left, PR bows toward the top right, and both say the model is decent.

MACHINE LEARNING · vizlearn.in/machine_learning/precision_recall_vs_roc.html

More in Machine Learning

Machine LearningThreshold TuningMachine LearningDecision Tree AnalysisMachine LearningRandom Forest and BaggingMachine LearningGradient Boosting

About the author

Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.

About VizLearnGitHubLinkedInContact