predict() makes a decision you did not authorise. Taking it back is the cheapest improvement available, and it needs no refitting.
Overview
The hidden default
Most classifiers in scikit-learn can output probabilities, not just class labels. predict_proba returns the estimated chance of each class for every row. predict takes those probabilities and applies a threshold to turn them into labels.
The default threshold is 0.5. If the predicted probability of the positive class is above 0.5, the model says positive. Otherwise it says negative. That threshold is baked into predict and there is no parameter to change it.
On balanced data with equal misclassification costs, 0.5 is reasonable. On imbalanced data, or when one kind of mistake is more expensive than the other, 0.5 is almost never the right choice. A fraud-detection model at 0.5 will miss most fraud. A medical-screening model at 0.5 will miss most disease. Lowering the threshold catches more positives at the cost of more false alarms.
Moving the threshold does not retrain the model. The fitted probabilities stay the same. You are just changing where the line is drawn between "flag" and "skip". The editors on this page show how the same model produces very different confusion matrices at different thresholds.
predict() calls predict_proba() and applies a threshold of 0.5. That is the whole of it. The first editor confirms the two are the same.
There's nothing principled about 0.5. It's the point where the two classes are equally likely. This Would be the right cut only if the classes were equally frequent *and* the two kinds of mistake cost the same. Neither is often true.
The consequence on imbalanced data can be severe. The second editor fits a model on data with 15% positives and reports what happens at each threshold. At 0.5 the model flags 5 cases out of 900 and catches 2.2% of the positives. It isn't a broken model. At a threshold of 0.1 the same fitted model catches 91.6%. It's a model being asked the wrong question.
Worth knowing
predict() is predict_proba() >= 0.5 - a decision the library made, not one the problem did.
Moving the threshold trades precision against recall without refitting, which makes it the cheapest adjustment available.
precision_recall_curve gives every threshold at once, so you can choose one to meet a requirement.
Choose the threshold on a validation set, not on the test set - it is a parameter like any other.
ROC AUC stays high on imbalanced data; average precision is the honest summary when positives are rare.
Probabilities are only meaningful if the model is calibrated; CalibratedClassifierCV fixes one that is not.
Probabilities and Thresholds: A Practical Guide
each classifier in this track has been making a choice on your behalf at 0.5. Taking that choice back costs nothing and often raises the model more than tuning does.
predict is proba plus a hidden decision
The two agree exactly, because one is defined as the other.
example_01.pyscikit-learn
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import make_classification
X, y = make_classification(n_samples=400, n_features=6, random_state=0)
model = LogisticRegression().fit(X, y)
proba = model.predict_proba(X)[:, 1]
manual = (proba >= 0.5).astype(int)
print("predict() equals proba >= 0.5 :", (manual == model.predict(X)).all())
print()
print("the 0.5 is a default, not a law - nothing about the problem")
print("said the boundary belonged there.")
Output
One model, five classifiers
Look at what the default threshold does to recall on this data.
example_02.pyscikit-learn
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_score, recall_score
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
# A harder problem, so both numbers actually move.
X, y = make_classification(n_samples=3000, n_features=10, n_informative=3,
weights=[0.85], flip_y=0.15, random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3,
random_state=0, stratify=y)
model = LogisticRegression(max_iter=1000).fit(X_tr, y_tr)
proba = model.predict_proba(X_te)[:, 1]
print("threshold precision recall flagged")
for t in (0.1, 0.2, 0.3, 0.5, 0.7):
pred = (proba >= t).astype(int)
print("%9.1f %10.3f %7.3f %8d"
% (t, precision_score(y_te, pred, zero_division=0),
recall_score(y_te, pred), int(pred.sum())))
print()
print("one fitted model, five different classifiers - and predict()")
print("silently chose the middle row for you.")
Output
The curve is every threshold at once
Which is how you pick one to hit a requirement rather than by trying numbers.
example_03.pyscikit-learn
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_recall_curve
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
import numpy as np
X, y = make_classification(n_samples=3000, n_features=10, n_informative=3,
weights=[0.85], flip_y=0.15, random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3,
random_state=0, stratify=y)
proba = LogisticRegression(max_iter=1000).fit(X_tr, y_tr).predict_proba(X_te)[:, 1]
precision, recall, thresholds = precision_recall_curve(y_te, proba)
for target in (0.70, 0.90):
ok = np.flatnonzero(recall[:-1] >= target)
i = ok[-1]
print("for %d%% recall: threshold <= %.3f, precision %.3f"
% (target * 100, thresholds[i], precision[i]))
print()
print("the curve is every threshold at once, which is what you read")
print("to pick one deliberately.")
Output
ROC AUC flatters an imbalanced problem
The same model quality, two class balances, and only one of the two metrics notices.
example_04.pyscikit-learn
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score, average_precision_score
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
for weight in (0.5, 0.99):
X, y = make_classification(n_samples=4000, n_features=10,
weights=[weight], random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3,
random_state=0, stratify=y)
m = LogisticRegression(max_iter=1000).fit(X_tr, y_tr)
p = m.predict_proba(X_te)[:, 1]
print("positives %5.1f%% ROC AUC %.3f average precision %.3f"
% (100 * (1 - weight), roc_auc_score(y_te, p),
average_precision_score(y_te, p)))
print()
print("ROC AUC stays high when positives are rare, because the huge")
print("number of true negatives keeps the false positive rate low.")
Output
Are the probabilities honest?
A calibrated model's 0.7 means seventy per cent of those cases turn out positive.
example_05.pyscikit-learn
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.calibration import calibration_curve
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
import numpy as np
X, y = make_classification(n_samples=3000, n_features=10, random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.4, random_state=0)
for name, m in [("LogisticRegression", LogisticRegression(max_iter=1000)),
("RandomForest", RandomForestClassifier(n_estimators=50,
random_state=0))]:
p = m.fit(X_tr, y_tr).predict_proba(X_te)[:, 1]
true, pred = calibration_curve(y_te, p, n_bins=5)
gap = np.abs(true - pred).mean()
print("%-20s mean |predicted - observed| = %.4f" % (name, gap))
print()
print("a calibrated model's predicted 0.7 means 70% of those cases")
print("are positive. one of these two is closer to that than the other.")
Output
Choosing the threshold by cost
When a miss costs ten times a false alarm, the default is not close to optimal.
example_06.pyscikit-learn
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
import numpy as np
X, y = make_classification(n_samples=2000, n_features=10, weights=[0.9],
random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3,
random_state=0, stratify=y)
proba = LogisticRegression(max_iter=1000).fit(X_tr, y_tr).predict_proba(X_te)[:, 1]
# A miss costs 10, a false alarm costs 1.
best = None
for t in np.arange(0.05, 0.96, 0.05):
tn, fp, fn, tp = confusion_matrix(y_te, (proba >= t).astype(int)).ravel()
cost = 10 * fn + 1 * fp
if best is None or cost < best[1]:
best = (t, cost, fn, fp)
print("cheapest threshold: %.2f (cost %d = %d misses x10 + %d false alarms)"
% best)
tn, fp, fn, tp = confusion_matrix(y_te, (proba >= 0.5).astype(int)).ravel()
print("the default 0.5 would cost:", 10 * fn + fp)
Output
Moving it costs nothing
The threshold is applied *after* fitting. Changing it needs no refitting, no search. No extra data. Get the chances once and each operating point is available.
That makes it the cheapest lever in the whole workflow. It's routinely left untouched while big effort goes into tuning settings that move the score far less.
The mechanics are one line: (proba >= t).astype(int) instead of predict(X), having taken proba = model.predict_proba(X)[:, 1]. Column 1 is the positive class. The columns follow classes_. With labels [0, 1] the second column is the chance of 1.
Choosing one deliberately
Three ways, in growing order of rigour.
To meet a need. "We must catch 90% of cases" is a recall target. precision_recall_curve says what threshold reaches it and what precision you pay. The third editor answers it right: 90% recall needs a threshold of 0.105 and gives precision 0.207. Roughly five false alarms per real case. This The business either accepts or doesn't.
To balance the metric you chose. Sweep the threshold, find F1 or weighed fit at each, take the best. Simple. It optimises a proxy rather than the thing you care about.
To cut cost. The honest version. Attach a price to a false minus and a false plus, find the total at each threshold, take the min. The last editor does just this with a miss costing ten times a false alarm. The cheapest threshold is 0.15 with a cost of 50, against 90 at the default. Nearly half the cost, from changing one number.
Whichever you use, choose it on a validation set. The threshold is a param fitted to data. Choosing it on the test set is the same mistake as tuning on the test set. The resulting precision and recall are optimistic.
The curve, and its summary
precision_recall_curve returns arrays of precision, recall and the thresholds that give them. The trailing element of precision and recall has no matching threshold. This Is why indexing them together needs the [:-1] the editor uses.
average_precision_score summarises the whole curve as one number. roc_curve and roc_auc_score are the option pair, plotting the true positive rate against the false positive rate.
The choice between them matters on imbalanced data. The fourth editor shows why. The same kind of model on data with 50% positives scores ROC AUC 0.935 and mean precision 0.919. Close together. At 1% positives it scores ROC AUC 0.825 and mean precision 0.134.
ROC AUC barely moved because its false-positive rate cuts by the number of true negatives. With 99% negatives that denominator is huge. Even many false alarms barely register. Mean precision skips true negatives fully and reports what really happens to the rare class.
The rule: ROC AUC for roughly weighed problems, mean precision when positives are rare. A high ROC AUC on a 1% problem is close to meaningless.
Whether the probabilities mean anything
A model outputting 0.7 is making a claim. Among cases it scores 0.7, about 70% should be plus. A model whose output meets that is calibrated.
Logistic regression is often well calibrated. Raising the likelihood of the seen labels is close to what calibration asks for. Tree ensembles often aren't. A random forest's output is the proportion of trees voting plus. This Ranks right and is pushed towards the middle. It rarely says 0.01 or 0.99 because rarely do all the trees agree. The fifth editor measures the gap and finds the forest roughly twice as far off as the linear model.
This matters when the chance is used as a number rather than as a ranking. Expected-value maths, risk scores shown to people, thresholds chosen by cost. It doesn't matter if you only ever rank and cut.
calibration_curve measures it by bucketing guesses and checking the mean predicted chance against the seen rate in each bucket. CalibratedClassifierCV fixes it by fitting a correction on held-out data, using either a sigmoid or an isotonic fit.
The threshold is part of the model
A subtle point that causes real problems in deployment. Once you choose a threshold, it belongs to the model and has to travel with it.
A pipeline pickled without its threshold will be loaded a place else and used through predict(). This Silently reverts to 0.5. And on the imbalanced problem above that is the difference between catching 91% of cases and catching 2%. Nothing raises. The service simply flags almost nothing and everyone assumes the model is weak.
Three ways to keep them together. Record the threshold alongside the model file and apply it by name in the serving code. This Is the most common and leans on discipline. Wrap the model in a small class whose predict applies your threshold. The object carries it. Or, from recent scikit-learn versions, use FixedThresholdClassifier and TunedThresholdClassifierCV. This Are estimators that hold a threshold and expose the plain interface. The second finds it by cross-validation against a metric or a cost function you supply.
The last of those is the tidiest answer. The threshold is then chosen inside the folds like any other param and stored on the fitted object like any other learned value.
Multi-class is a different question
All on this page assumes two classes. With more, predict() takes the highest chance rather than checking against a threshold. There's no single number to move.
The equal adjustments are different. class_weight shifts the model's own preferences during fitting. Post-processing the chance mat. Timesing each column by a factor and taking the argmax. Is the closest analogue and has to be tuned per class. And a truly cost-sensitive multi-class choice means writing the expected cost of each guess by name and choosing the min. This Is a few lines and rarely done.
Where a binary threshold *does* reappear is one-versus-rest setups. Here Each class has its own binary choice and each can carry its own threshold. Which is how multi-label problems are often handled.
Which column of predict_proba is the positive class? The one matching classes_. With labels 0 and 1 that is column 1. With string labels it's whichever sorts second - read classes_ rather than given.
What if my model has no predict_proba? Many have decision_function instead. This Returns an unbounded score rather than a chance. Thresholding it works the same way. The numbers are just not chances.
Should I tune the threshold or use class_weight? Both. They do different things. class_weight changes what the model optimises while fitting. The threshold changes the choice afterwards. Tuning the threshold is cheaper and reversible.
Does a better threshold improve AUC? No. AUC summarises each threshold. It's unchanged by picking one. That is just why it can't tell you which to pick.
Why the default is 0.5 at all
It's worth understanding rather than resenting. The thinking tells you just when it's right.
Under two assumptions, 0.5 is best. The first is that the two classes are equally frequent. A chance above a half truly means "more likely than not". The second is that a false plus and a false minus cost the same. So The choice should go to whichever is more probable.
Both assumptions are defaults in the same sense the threshold is. They're what you get when no one has said or else. And both are wrong on most real problems. Fraud, disease, churn, defects and failures are all rare. In each one of them a miss and a false alarm have quite different prices.
So the library's choice isn't a mistake. It's the only neutral answer available to a thing that knows nothing about your problem. What it can't do is warn you. From inside the estimator there's no way to tell a weighed problem from an imbalanced one that you happen to care about differently.
The hands-on version: on any problem where the classes are uneven or the mistakes cost differently, treat the threshold as unset rather than as set to 0.5. It's a param with a default. The default was chosen in the lack of the info only you've.
A short procedure
Five steps that turn all of this into a thing routine.
Split off a validation set, or use cross-validation, keeping the test set untouched. Fit the model and take predict_proba on the validation part. Sweep thresholds from 0.01 to 0.99 and find, at each, the quantity you really care about. A cost, a metric, or the precision at a needed recall. Choose the threshold that optimises it. Record it with the model. Then measure once on the test set, using that fixed threshold, and report those numbers.
The whole thing is about fifteen lines and it's the difference between a model whose reported speed is achievable and one whose numbers came from a choice no one made.
Can the threshold be above 0.5? Yes. It should be when false alarms are the costly mistake - a spam filter that deletes mail, say. Nothing restricts it to the lower half.
Why does my precision-recall curve have one more point than thresholds? The curve includes the endpoint where all is predicted minus. This Has no threshold. Index with [:-1] when pairing them.
Things to try with Probabilities and Thresholds
Read the second editor properly. At the default threshold the model catches 2.2% of positives. It's a good model being asked a bad question.
Set a target. In the third editor, change the targets to 0.95 and 0.99 and watch the precision collapse.
Change the cost ratio. In the last editor, make a miss cost 100 and see where the threshold moves.
Calibrate the forest. Wrap it in CalibratedClassifierCV and re-measure the gap.
Questions people ask about Probabilities and Thresholds
What threshold does predict use? For most binary classifiers it effectively uses 0.5 on the positive probability. You can choose a different cut-off yourself.
How do I choose a threshold? Use a precision-recall curve on validation data and pick the point that matches your costs or minimum recall.
Does changing the threshold retrain the model? No. It only changes how probabilities become labels, so it is cheap to try.
Are predict_proba values true probabilities? Not always. Check a calibration curve, and use CalibratedClassifierCV if they are off.
What does ROC AUC tell me? It is the chance that a random positive scores higher than a random negative, independent of any threshold.
Should the threshold be saved with the model? Yes. It is part of the decision rule, and a loaded model will otherwise fall back to the default.
Where this leaves you
predict() hides a choice. predict_proba() gives it back. Choose the threshold from the cost of each mistake, on a validation set, using the precision-recall curve to see the options. And check that the chances are calibrated before treating any of them as a number.
Check yourself
0 of 4
Answer without scrolling back up.
What does predict() do that predict_proba() does not?
The two are identical by definition: predict is proba >= 0.5. The 0.5 is a library default, not a property of your problem.
Does changing the threshold require refitting?
That is what makes it the cheapest adjustment available, and why it is worth doing before tuning hyperparameters.
Positives are 1% of the data. Which summary should you trust?
ROC AUC read 0.825 where average precision read 0.134 on the same predictions. The false positive rate divides by an enormous number of true negatives.
What does it mean for a model to be calibrated?
It matters whenever the probability is used as a number. Tree ensembles rank well and are usually poorly calibrated; logistic regression usually is not.
Cheat sheet
Probabilities and Thresholds
Move the classification threshold away from 0.5, plot a precision-recall curve, and decide the cut-off that matches your cost trade-off.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.