A linear model that outputs probabilities - the sensible first classifier for almost any problem, and the baseline every other one has to beat.
Overview
Why it is called regression
Logistic regression is the standard first classifier. It is fast to train, produces probabilities rather than just labels, and its coefficients are interpretable. For many tabular classification tasks it is the right starting point.
Despite the name, logistic regression is a classifier. It predicts which class a sample belongs to. The confusion is that the underlying arithmetic is a linear regression on a transformed target. The transformation is what makes it a classifier rather than a regression model.
The model also serves as a strong baseline. If a random forest or gradient-boosted tree barely beats logistic regression on your data, the extra complexity may not be worth the cost. Running logistic regression first tells you how much room there is for improvement.
This page covers the name, the probabilities, the coefficients, the scaling requirement and the parameters worth knowing. It also covers the multi-class case and the question of whether the probabilities the model outputs can be trusted.
The name is the first confusion and it has a real explanation. Logistic regression is a classifier. It predicts which class a sample belongs to. But the thing being fitted linearly isn't the class.
What it fits is the log-odds: the logarithm of the ratio between the chance of the positive class and the chance of the minus one. That quantity runs from minus infinity to plus infinity. This Makes it a thing a linear model can predict sensibly. The logistic function then squashes it back into the range 0 to 1, giving a chance.
So underneath it's a linear regression on a transformed target. The change is what turns an unbounded guess into a chance that can't escape its bounds. Predicting the chance right with linear regression would happily give 1.4 or -0.3. This Is why no one does that.
Worth knowing
Despite the name it is a classifier; the regression in it is on the log-odds, not on the class.
predict_proba returns one column per class, in classes_ order, and each row sums to 1.
predict is just predict_proba with a 0.5 threshold applied - which is a choice, not a law.
A coefficient is a change in log-odds per unit of the feature; exp(coef) is the odds ratio.
It is fitted by iterative optimisation, so unscaled features produce a ConvergenceWarning - scale them.
The decision boundary is linear, so genuinely curved problems need engineered features or a different model.
Logistic Regression: A Practical Guide
The classifier to try first, what the problem. It's fast, it makes chances rather than bare labels. It gives every other model a number to beat.
Fitting a classifier
The same three calls as a regressor, and a score that means accuracy rather than R-squared.
example_01.pyscikit-learn
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25,
random_state=0, stratify=y)
scaler = StandardScaler().fit(X_tr)
model = LogisticRegression(max_iter=1000).fit(scaler.transform(X_tr), y_tr)
print("classes_ :", model.classes_)
print("accuracy :", round(model.score(scaler.transform(X_te), y_te), 4))
print("first 8 predictions:", model.predict(scaler.transform(X_te))[:8])
Output
predict_proba is the useful output
predict throws away the confidence. The probabilities are what let you decide where the line should be.
example_02.pyscikit-learn
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import numpy as np
X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25,
random_state=0, stratify=y)
sc = StandardScaler().fit(X_tr)
model = LogisticRegression(max_iter=1000).fit(sc.transform(X_tr), y_tr)
proba = model.predict_proba(sc.transform(X_te))
pred = model.predict(sc.transform(X_te))
print("predict_proba shape:", proba.shape, "- one column per class")
print()
print(" P(class 0) P(class 1) predicted")
for p, k in list(zip(proba, pred))[:5]:
print("%11.4f %11.4f %10d" % (p[0], p[1], k))
print()
print("rows sum to 1:", np.allclose(proba.sum(axis=1), 1.0))
Output
What the coefficient means
Not a probability, and not a slope in the target - it is a change in the log of the odds.
example_03.pyscikit-learn
from sklearn.linear_model import LogisticRegression
import numpy as np
# One feature, so the coefficient is readable.
X = np.array([[1], [2], [3], [4], [5], [6], [7], [8]])
y = np.array([0, 0, 0, 0, 1, 1, 1, 1])
model = LogisticRegression().fit(X, y)
print("coef_ :", model.coef_[0].round(4))
print("intercept_:", model.intercept_.round(4))
print()
print("the coefficient is in log-odds per unit of x:")
print("odds multiply by exp(coef) = %.3f for each step of 1"
% np.exp(model.coef_[0][0]))
Output
It needs its features scaled
The same data twice. Only one of the two fits converges in the default number of iterations.
example_04.pyscikit-learn
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
import warnings
X, y = load_breast_cancer(return_X_y=True)
with warnings.catch_warnings(record=True) as caught:
warnings.simplefilter("always")
LogisticRegression(max_iter=100).fit(X, y)
names = [w.category.__name__ for w in caught]
print("unscaled, max_iter=100 ->", names or ["no warning"])
Xs = StandardScaler().fit_transform(X)
with warnings.catch_warnings(record=True) as caught:
warnings.simplefilter("always")
LogisticRegression(max_iter=100).fit(Xs, y)
names = [w.category.__name__ for w in caught]
print("scaled, max_iter=100 ->", names or ["no warning"])
Output
More than two classes, without changing anything
One row of coefficients per class, and probabilities that still sum to one across them.
example_05.pyscikit-learn
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
import numpy as np
X, y = load_iris(return_X_y=True)
Xs = StandardScaler().fit_transform(X)
model = LogisticRegression(max_iter=1000).fit(Xs, y)
print("classes_ :", model.classes_)
print("coef_ :", model.coef_.shape, "- one row per class")
print()
proba = model.predict_proba(Xs[:3])
print("probabilities for the first three rows:")
print(np.round(proba, 4))
print("each row sums to 1 and the winner is predict()'s answer")
Output
The boundary is a straight line
Which is the model's one real limitation, and it is easy to construct data that shows it.
example_06.pyscikit-learn
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
X, y = make_moons(n_samples=300, noise=0.2, random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0)
for model in [LogisticRegression(), DecisionTreeClassifier(random_state=0)]:
model.fit(X_tr, y_tr)
print("%-24s test accuracy %.3f"
% (type(model).__name__, model.score(X_te, y_te)))
print()
print("two interleaving crescents cannot be separated by a straight line,")
print("which is exactly what logistic regression is limited to.")
Output
Probabilities, not just labels
predict returns the class. predict_proba returns the chance of each class, as an array with one column per class in the order given by classes_.
The second is almost always more handy. predict throws away info you needed. A guess of "plus" with chance 0.51 and one with chance 0.99 are wildly different situations, and predict reports them the same way.
That matters wherever the two kinds of mistake have different costs. A medical screen that is 51% confident should probably be followed up. One that is 99% confident may warrant immediate action. A fraud check at 51% might ask for a second factor while one at 99% blocks the transaction. None of that is expressible if all you've is the label.
predict is just predict_proba with a threshold of 0.5. 0.5 is a default rather than a principle. Choosing a different threshold is one of the highest-value adjustments available. It gets its own module later in this track.
Reading the coefficients
coef_ has the same shape story as linear regression. One number per feature. But the units are different and the difference is the part people get wrong.
A coeff is the change in log-odds for a one-unit grow in the feature, holding the others fixed. Log-odds aren't natural. The conventional move is to exponentiate: exp(coef) is an odds ratio, the factor by which the odds times per unit of the feature.
A coeff of 1.17, as in the editor above, means the odds times by about 3.2 for each step of one in that feature. That is a statement about odds, not about chance. The two diverge. Timesing odds by 3.2 moves a chance of 0.1 to about 0.26. Moves 0.5 to about 0.76 and 0.9 to about 0.97. The same coeff has a different effect on chance based on where you start. This Is the whole point of the logistic curve being a curve.
The scale caveat from linear regression applies with equal force. A coeff's size depends on the feature's units. Coeffs are only close to each other when the features were standardised first. And since the model needs scaling anyway, this often comes for free.
It has to be scaled
Unlike plain linear regression, logistic regression is fitted by iterative optimisation. There's no formula. A solver takes steps towards a min and stops when it stops raising or when it runs out of iterations.
Features on wildly different scales make that surface awkward to descend. The solver takes far more steps and often hits max_iter first. When it does, scikit-learn issues a ConvergenceWarning and returns the half-finished model anyway. It doesn't raise. The coeffs you get are simply wherever the solver had reached.
The editor above shows both cases on the same data. Unscaled and it warns, scaled and it settles comfortably within the same budget.
The right fix is StandardScaler, not a larger max_iter. Raising the iteration limit makes the warning go away by letting the solver grind to the same answer more slowly. Scaling makes the problem easy. Raising max_iter is the correct response only when the features are already scaled and the model is truly large.
There's a second reason scaling matters here: LogisticRegression applies L2 smoothing by default, controlled by C. A fine on coeff size is meaningless when the coeffs are in incomparable units. An unscaled fit is smoothed unevenly across features without saying so.
C, and the regularisation nobody notices
LogisticRegression() is smoothed out of the box. This Surprises people who expect a plain max-likelihood fit.
The strength is set by C. The param is inverted. Small C means strong smoothing, large C means weak. C=1.0 is the default, C=0.01 shrinks the coeffs hard, and C=1000 is close to unregularised. The inversion catches everybody at least once. Every other fine param in the library. alpha in Ridge and Lasso. Runs the other way.
The default is often reasonable, and C is one of the first things worth tuning. With many features it's often the only setting that matters.
fine="l1" gives the Lasso-style action of driving some coeffs to just zero. This Selects features as it fits. It needs a compatible solver such as liblinear or saga.
More than two classes
Nothing changes at the call site. Fit on a target with three classes and predict returns one of the three, predict_proba returns three columns summing to one. coef_ gains a row per class.
Underneath, recent versions fit a genuine multinomial model when the solver supports it. This Guesses all the classes jointly rather than running a series of one-against-the-rest fits. The hands-on consequence is that the chances are properly normed across classes rather than being separate binary chances rescaled afterwards.
classes_ is worth reading rather than given. It holds the classes in sorted order. The columns of predict_proba follow it. So with string labels, ["cat", "dog", "fish"], column 0 is cat. Indexing that array the wrong way round is a quiet way to report the wrong chance.
What it cannot do
The choice boundary is a straight line. A plane in higher dims. And no amount of tuning changes that.
For data that truly needs a curved boundary, the model will do its honest best and be beaten by any of it that can bend. The editor above makes it concrete. On two interleaving crescents, logistic regression manages 0.84 and a choice tree 0.99. The gap is fully structural.
Two responses. Engineer features that make the problem linear. Adding squares and products via PolynomialFeatures lets a linear boundary in the expanded space be a curved one in the first. Or use a model that finds the shape itself. This Is what trees and their ensembles do and why they dominate on tabular data.
Neither makes logistic regression a poor first choice. It fits in milliseconds, needs almost no tuning, makes calibrated-ish chances and coeffs you can read. Tells you right away whether the problem is easy. A grad-boosted ensemble that beats it by two points may not be worth the cost. One that beats it by thirty tells you the structure is truly non-linear.
Are the probabilities trustworthy?
A model that outputs 0.7 is making a claim. Among the cases it labels 0.7, about 70% should turn out plus. A model whose chances meet that is calibrated, and not each classifier is.
Logistic regression is unusually good here. Because it's fitted by raising the likelihood of the seen labels, well-calibrated chances are what it's right optimising for. On reasonable data it makes them. That is a real edge over models that output a score which merely ranks right. A random forest's averaged votes and an SVM's distance from the boundary both rank well and are systematically off as chances, often pushed towards the middle or the extremes.
Two things spoil it even here. Heavy smoothing shrinks the coeffs and pulls the chances towards 0.5. And a badly imbalanced training set, or one resampled to fix imbalance, shifts them wholesale.
CalibratedClassifierCV wraps any classifier and fits a correction on held-out data. This Is how you get usable chances out of a model that doesn't give them on its own. Checking calibration is a matter of bucketing guesses and checking the mean predicted chance against the seen rate in each bucket. calibration_curve does it in a line. It's worth doing when a chance is going to be used as a number rather than as a ranking.
Which solver, and when it matters
LogisticRegression takes a solver, and the default handles most cases. The choice turns relevant in three situations.
lbfgs is the default: fast, handles multinomial fits right, supports only L2 and no fine. liblinear is the one to use for L1 on a small dataset. It fits one-against-the-rest rather than a true multinomial. saga supports L1, L2 and ElasticNet, handles sparse data well. Is the one to reach for on large or high-dimensional problems. Text, in given. newton-cholesky is fast when there are many samples and few features.
The hands-on rule: leave it alone until a thing makes you change it. The two things that will are asking for a fine the default can't do. This Raises a clear error naming the compatible solvers. A fit that is slow on a large sparse mat. Here saga is the answer.
Does it work with more than two classes? Yes, with no change at the call site. coef_ gains a row per class and predict_proba gains a column, in classes_ order.
Can the target be strings? Yes. classes_ holds them sorted. Guesses come back as the same strings.
Why is my fit 0.99 and the model useless? Almost certainly imbalance - if 99% of rows are one class, predicting that class always scores 0.99. The classification metrics module covers what to use instead.
Where it sits among the alternatives
Three models occupy nearly the same space. Knowing what separates them saves trying all three blindly.
Linear SVM draws a straight boundary too. Chooses it by raising the margin between the classes rather than by raising likelihood. It's often marginally more accurate and gives no chances without an extra calibration step. This Is often the deciding factor against it.
Naive Bayes is faster still and makes a strong freedom assumption between features. On text it works oddly well despite that assumption being obviously false. It's the right first try for a bag-of-words problem.
Grad boosting finds non-linear structure without being told where to look and often wins on tabular data by a clear margin. It costs tuning time, fits slower. Gives up the readable coeffs.
The sequence worth following is: logistic regression first for the number to beat, grad boosting second to find out whether the problem has structure a line can't see. Any of it else only if those two leave a exact gap.
Things to try with Logistic Regression
Look at the chances. In the second editor, find the rows nearest 0.5. Those are the ones where the label is close to any.
Break settling. In the fourth editor, drop max_iter to 20 on the scaled data and watch the warning return.
Turn smoothing up. Fit with C=0.001 and compare the coeffs to the default. They shrink towards zero together.
Bend the boundary. In the last editor, wrap the logistic model in a pipeline with PolynomialFeatures(degree=3) and see how much of the gap closes.
Questions people ask about Logistic Regression
Why is it called regression if it classifies? It fits a linear model to the log-odds of a class, then passes that through a sigmoid to get a probability. The class label comes from thresholding it.
Do I need to scale features for logistic regression? Yes in practice. The default solver converges poorly on unscaled data, and regularisation treats features unevenly.
What does C control? C is the inverse of regularisation strength. A smaller C pulls coefficients towards zero more firmly, and a larger C lets the model fit the training data closely.
Can it handle more than two classes? Yes. It supports multi-class problems directly with the default solvers, producing one probability per class.
Can it learn a non-linear boundary? Not by itself. Add polynomial or interaction features, or switch to a tree-based model.
Are predict_proba outputs reliable? Often fairly well calibrated, but not guaranteed. Check a calibration curve before treating them as true probabilities.
Where this leaves you
A fast linear classifier that returns chances, needs its features scaled, is smoothed by default with an inverted param. Draws a straight boundary. Fit it first on each classification problem. Treat its score as the number any of it more complicated has to justify itself against.
Check yourself
0 of 4
Answer without scrolling back up.
Why is it called regression when it classifies?
The linear part predicts log-odds, which is unbounded and so suitable for a linear fit. The logistic function converts that to a probability between 0 and 1.
What is predict() doing that predict_proba() is not?
predict is predict_proba plus a threshold. The 0.5 is a default choice, and changing it is often the highest-value adjustment available.
What does a ConvergenceWarning mean here?
It does not raise. You get whatever coefficients the solver had reached, which is why scaling the features is the fix rather than raising max_iter.
In LogisticRegression, what does a small C mean?
C is inverted relative to alpha in Ridge and Lasso: small C penalises coefficient size heavily. It catches almost everyone once.
Cheat sheet
Logistic Regression
Train a logistic regression classifier, interpret its coefficients as log-odds, and see why it is the model to try first on any problem.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.