predict() makes a decision you did not authorise. Taking it back is the cheapest improvement available, and it needs no refitting.
Overview
The hidden default
predict() calls predict_proba() and applies a threshold of 0.5. That is the whole of it, and the first editor confirms the two are identical.
There is nothing principled about 0.5. It is the point where the two classes are equally likely, which would be the right cut only if the classes were equally frequent *and* the two kinds of mistake cost the same. Neither is usually true.
The consequence on imbalanced data can be severe. The second editor fits a model on data with 15% positives and reports what happens at each threshold: at 0.5 the model flags 5 cases out of 900 and catches 2.2% of the positives. It is not a broken model — at a threshold of 0.1 the same fitted model catches 91.6% — it is a model being asked the wrong question.
Worth knowing
predict() is predict_proba() >= 0.5 - a decision the library made, not one the problem did.
Moving the threshold trades precision against recall without refitting, which makes it the cheapest adjustment available.
precision_recall_curve gives every threshold at once, so you can choose one to meet a requirement.
Choose the threshold on a validation set, not on the test set - it is a parameter like any other.
ROC AUC stays high on imbalanced data; average precision is the honest summary when positives are rare.
Probabilities are only meaningful if the model is calibrated; CalibratedClassifierCV fixes one that is not.
Probabilities and Thresholds: A Practical Guide
Every classifier in this track has been making a decision on your behalf at 0.5. Taking that decision back costs nothing and often improves the model more than tuning does.
predict is proba plus a hidden decision
The two agree exactly, because one is defined as the other.
example_01.pyscikit-learn
Output
One model, five classifiers
Look at what the default threshold does to recall on this data.
example_02.pyscikit-learn
Output
The curve is every threshold at once
Which is how you pick one to hit a requirement rather than by trying numbers.
example_03.pyscikit-learn
Output
ROC AUC flatters an imbalanced problem
The same model quality, two class balances, and only one of the two metrics notices.
example_04.pyscikit-learn
Output
Are the probabilities honest?
A calibrated model's 0.7 means seventy per cent of those cases turn out positive.
example_05.pyscikit-learn
Output
Choosing the threshold by cost
When a miss costs ten times a false alarm, the default is not close to optimal.
example_06.pyscikit-learn
Output
Moving it costs nothing
The threshold is applied *after* fitting, so changing it requires no refitting, no search, and no additional data. Get the probabilities once and every operating point is available.
That makes it the cheapest lever in the whole workflow, and it is routinely left untouched while considerable effort goes into tuning hyperparameters that move the score far less.
The mechanics are one line: (proba >= t).astype(int) instead of predict(X), having taken proba = model.predict_proba(X)[:, 1]. Column 1 is the positive class — the columns follow classes_, so with labels [0, 1] the second column is the probability of 1.
Choosing one deliberately
Three ways, in increasing order of rigour.
To meet a requirement. "We must catch 90% of cases" is a recall target, and precision_recall_curve says what threshold achieves it and what precision you pay. The third editor answers it directly: 90% recall needs a threshold of 0.105 and gives precision 0.207 — roughly five false alarms per real case, which the business either accepts or does not.
To balance the metric you chose. Sweep the threshold, compute F1 or balanced accuracy at each, take the best. Simple, and it optimises a proxy rather than the thing you care about.
To minimise cost. The honest version. Attach a price to a false negative and a false positive, compute the total at each threshold, take the minimum. The last editor does exactly this with a miss costing ten times a false alarm: the cheapest threshold is 0.15 with a cost of 50, against 90 at the default. Nearly half the cost, from changing one number.
Whichever you use, choose it on a validation set. The threshold is a parameter fitted to data, and choosing it on the test set is the same mistake as tuning on the test set — the resulting precision and recall are optimistic.
The curve, and its summary
precision_recall_curve returns arrays of precision, recall and the thresholds that produce them. The trailing element of precision and recall has no corresponding threshold, which is why indexing them together needs the [:-1] the editor uses.
average_precision_score summarises the whole curve as one number. roc_curve and roc_auc_score are the alternative pair, plotting the true positive rate against the false positive rate.
The choice between them matters on imbalanced data, and the fourth editor shows why. The same kind of model on data with 50% positives scores ROC AUC 0.935 and average precision 0.919 — close together. At 1% positives it scores ROC AUC 0.825 and average precision 0.134.
ROC AUC barely moved because its false-positive rate divides by the number of true negatives, and with 99% negatives that denominator is enormous, so even many false alarms barely register. Average precision ignores true negatives entirely and reports what actually happens to the rare class.
The rule: ROC AUC for roughly balanced problems, average precision when positives are rare. A high ROC AUC on a 1% problem is close to meaningless.
Whether the probabilities mean anything
A model outputting 0.7 is making a claim: among cases it scores 0.7, about 70% should be positive. A model whose output satisfies that is calibrated.
Logistic regression is usually well calibrated, because maximising the likelihood of the observed labels is close to what calibration asks for. Tree ensembles usually are not: a random forest's output is the proportion of trees voting positive, which ranks correctly and is pushed towards the middle — it rarely says 0.01 or 0.99 because rarely do all the trees agree. The fifth editor measures the gap and finds the forest roughly twice as far off as the linear model.
This matters whenever the probability is used as a number rather than as a ranking: expected-value calculations, risk scores shown to people, thresholds chosen by cost. It does not matter if you only ever rank and cut.
calibration_curve measures it by bucketing predictions and comparing the average predicted probability against the observed rate in each bucket. CalibratedClassifierCV fixes it by fitting a correction on held-out data, using either a sigmoid or an isotonic fit.
The threshold is part of the model
A subtle point that causes real problems in deployment: once you choose a threshold, it belongs to the model and has to travel with it.
A pipeline pickled without its threshold will be loaded somewhere else and used through predict(), which silently reverts to 0.5 — and on the imbalanced problem above that is the difference between catching 91% of cases and catching 2%. Nothing raises; the service simply flags almost nothing and everyone assumes the model is weak.
Three ways to keep them together. Record the threshold alongside the model file and apply it explicitly in the serving code, which is the most common and relies on discipline. Wrap the model in a small class whose predict applies your threshold, so the object carries it. Or, from recent scikit-learn versions, use FixedThresholdClassifier and TunedThresholdClassifierCV, which are estimators that hold a threshold and expose the ordinary interface — the second finds it by cross-validation against a metric or a cost function you supply.
The last of those is the tidiest answer, because the threshold is then chosen inside the folds like any other parameter and stored on the fitted object like any other learned value.
Multi-class is a different question
Everything on this page assumes two classes. With more, predict() takes the highest probability rather than comparing against a threshold, so there is no single number to move.
The equivalent adjustments are different. class_weight shifts the model's own preferences during fitting. Post-processing the probability matrix — multiplying each column by a factor and taking the argmax — is the closest analogue and has to be tuned per class. And a genuinely cost-sensitive multi-class decision means writing the expected cost of each prediction explicitly and choosing the minimum, which is a few lines and rarely done.
Where a binary threshold *does* reappear is one-versus-rest arrangements, where each class has its own binary decision and each can carry its own threshold — which is how multi-label problems are usually handled.
Which column of predict_proba is the positive class? The one matching classes_. With labels 0 and 1 that is column 1, and with string labels it is whichever sorts second - read classes_ rather than assuming.
What if my model has no predict_proba? Many have decision_function instead, which returns an unbounded score rather than a probability. Thresholding it works the same way; the numbers are just not probabilities.
Should I tune the threshold or use class_weight? Both, and they do different things. class_weight changes what the model optimises while fitting; the threshold changes the decision afterwards. Tuning the threshold is cheaper and reversible.
Does a better threshold improve AUC? No. AUC summarises every threshold, so it is unchanged by picking one. That is exactly why it cannot tell you which to pick.
Why the default is 0.5 at all
It is worth understanding rather than resenting, because the reasoning tells you exactly when it is right.
Under two assumptions, 0.5 is optimal. The first is that the two classes are equally frequent, so a probability above a half genuinely means "more likely than not". The second is that a false positive and a false negative cost the same, so the decision should go to whichever is more probable.
Both assumptions are defaults in the same sense the threshold is: they are what you get when nobody has said otherwise. And both are wrong on most real problems. Fraud, disease, churn, defects and failures are all rare, and in every one of them a miss and a false alarm have quite different prices.
So the library's choice is not a mistake — it is the only neutral answer available to something that knows nothing about your problem. What it cannot do is warn you, because from inside the estimator there is no way to tell a balanced problem from an imbalanced one that you happen to care about differently.
The practical version: on any problem where the classes are uneven or the mistakes cost differently, treat the threshold as unset rather than as set to 0.5. It is a parameter with a default, and the default was chosen in the absence of the information only you have.
A short procedure
Five steps that turn all of this into something routine.
Split off a validation set, or use cross-validation, keeping the test set untouched. Fit the model and take predict_proba on the validation part. Sweep thresholds from 0.01 to 0.99 and compute, at each, the quantity you actually care about — a cost, a metric, or the precision at a required recall. Choose the threshold that optimises it, and record it with the model. Then measure once on the test set, using that fixed threshold, and report those numbers.
The whole thing is about fifteen lines and it is the difference between a model whose reported performance is achievable and one whose numbers came from a decision nobody made.
Can the threshold be above 0.5? Yes, and it should be whenever false alarms are the expensive mistake - a spam filter that deletes mail, say. Nothing restricts it to the lower half.
Why does my precision-recall curve have one more point than thresholds? The curve includes the endpoint where everything is predicted negative, which has no threshold. Index with [:-1] when pairing them.
Things to try
Read the second editor properly. At the default threshold the model catches 2.2% of positives. It is a good model being asked a bad question.
Set a target. In the third editor, change the targets to 0.95 and 0.99 and watch the precision collapse.
Change the cost ratio. In the last editor, make a miss cost 100 and see where the threshold moves.
Calibrate the forest. Wrap it in CalibratedClassifierCV and re-measure the gap.
Where this leaves you
predict() hides a decision; predict_proba() gives it back. Choose the threshold from the cost of each mistake, on a validation set, using the precision-recall curve to see the options — and check that the probabilities are calibrated before treating any of them as a number.
Check yourself
0 of 4
Answer without scrolling back up.
What does predict() do that predict_proba() does not?
The two are identical by definition: predict is proba >= 0.5. The 0.5 is a library default, not a property of your problem.
Does changing the threshold require refitting?
That is what makes it the cheapest adjustment available, and why it is worth doing before tuning hyperparameters.
Positives are 1% of the data. Which summary should you trust?
ROC AUC read 0.825 where average precision read 0.134 on the same predictions. The false positive rate divides by an enormous number of true negatives.
What does it mean for a model to be calibrated?
It matters whenever the probability is used as a number. Tree ensembles rank well and are usually poorly calibrated; logistic regression usually is not.
Cheat sheet
Probabilities and Thresholds
Every classifier in this track has been making a decision on your behalf at 0.5. Taking that decision back costs nothing and often improves the model more than tuning does.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.