Which models need it, which are entirely indifferent to it, and the one rule about when it is allowed to look at your data.
Overview
What scaling does
StandardScaler learns two numbers per column — the mean and the standard deviation — and transforms by subtracting the first and dividing by the second. The result has mean 0 and standard deviation 1 in every column.
That is the entire operation. The shape of the distribution is unchanged: a skewed column stays skewed, outliers stay outliers, and the relative positions of every value are preserved exactly. Scaling moves and stretches; it does not reshape.
The learned numbers live in mean_ and scale_, which is worth knowing because those two arrays are the entire fitted state and the thing that must not be learned from the test data.
Worth knowing
StandardScaler subtracts the mean and divides by the standard deviation, learning both from the data it is fitted on.
Models that measure distance or penalise coefficients need it: k-NN, SVM, PCA, k-means, and anything regularised.
Trees and their ensembles do not - they split one column at a time, so a monotonic rescaling changes nothing.
MinMaxScaler maps to a fixed range and is wrecked by outliers; RobustScaler uses the median and IQR instead.
Fit the scaler on the training data only, then transform both halves with it - the test set will not be centred, and should not be.
Put it in a Pipeline and that rule becomes impossible to break by accident.
Scaling Features: A Practical Guide
Half the models in the library are seriously damaged by unscaled features and half are completely indifferent. Knowing which is which saves both wasted work and silently bad results.
What StandardScaler learns
Two numbers per column, and the transform is a subtraction and a division.
example_01.pyscikit-learn
Output
What it is worth on a distance-based model
Same data, same model, one line of difference.
example_02.pyscikit-learn
Output
Three scalers, one outlier
The choice between them is a choice about what to do with extreme values.
example_03.pyscikit-learn
Output
Fit on train, transform on both
The test set does not end up centred on zero, and that is the point.
example_04.pyscikit-learn
Output
The pipeline does that for you
Which is why scaling and pipelines are taught together and used together.
example_05.pyscikit-learn
Output
Trees do not care at all
One of these two models moves. The other does not move by any amount at all.
example_06.pyscikit-learn
Output
Why some models need it
Two families of algorithm are affected, for two different reasons.
Anything that measures distance. k-nearest neighbours, SVMs with an RBF kernel, k-means, PCA. These compute how far apart two samples are, usually by summing squared differences across features. A column measured in thousands contributes thousands of times more to that sum than a column measured in single digits, so the distance is effectively decided by whichever feature happens to have the largest units. The model is not weighing the features by importance; it is weighing them by unit.
The editor above shows the cost: k-NN on the wine dataset scores 0.69 unscaled and 0.95 scaled. That is not a tuning improvement, it is the difference between a model that works and one that does not, and it comes from one column being measured in hundreds while others are measured in fractions.
Anything that penalises coefficient size. Ridge, Lasso, ElasticNet, and LogisticRegression, which is regularised by default. The penalty is on the magnitude of the coefficients, and a coefficient's magnitude depends on its feature's units. Without scaling, the penalty falls unevenly across features for reasons that have nothing to do with the data.
There is also a practical third reason: gradient-based solvers converge faster and more reliably on scaled features, which is why unscaled logistic regression produces ConvergenceWarning.
Why some models do not
Tree-based models — decision trees, random forests, gradient boosting — are completely unaffected, and the reason is structural rather than approximate.
A tree splits on one feature at a time, asking whether a value is above or below a threshold. Rescaling a column monotonically moves the threshold by the same transformation and leaves the ordering of every sample identical, so exactly the same split is chosen. The tree that results is the same tree.
The editor above measures it: the random forest changes by +0.0000. Not "a little"; not at all.
This matters practically because scaling a tree model is harmless but pointless, and adding a scaler to a pipeline that ends in a forest is a step that costs time and clarity for no benefit. It matters more because people who have learned "always scale" sometimes conclude that trees must therefore be doing something odd, when in fact this is one of the reasons trees are so convenient on messy tabular data: they need no preparation at all.
Choosing between the scalers
StandardScaler is the default and the right answer most of the time. It assumes nothing about the range and handles roughly symmetric data well.
MinMaxScaler maps each column onto a fixed range, usually 0 to 1. It is the choice when an algorithm requires bounded input, and its weakness is severe: a single outlier defines the range and compresses everything else against the opposite end. The editor above shows four ordinary values landing in the first hundredth of the range because one value is 100.
RobustScaler centres on the median and divides by the interquartile range. Because both statistics ignore the tails, extreme values do not distort the transformation of the ordinary ones — in the same demonstration, the four normal values keep a sensible spread and the outlier simply sits far away. It is the right default when the data has outliers you do not want to remove.
MaxAbsScaler divides by the largest absolute value and preserves zeros, which makes it the one to use on sparse data where centring would destroy the sparsity.
Normalizer is the odd one out and a frequent source of confusion: it scales each row to unit length rather than each column. It is for making samples comparable in direction rather than magnitude — text vectors, mostly — and is not a substitute for the column scalers.
The rule that makes it honest
The scaler must be fitted on the training data only, and then used to transform both halves.
The reason is the same as everywhere else in this track: mean_ and scale_ are learned from data, so a scaler fitted on everything has seen the test set, and the score afterwards is not a score on unseen data.
The consequence surprises people the first time. After scaling correctly, the test set is not centred on zero — its mean is whatever it happens to be relative to the training distribution. The editor above shows 0.0184 rather than 0. That is correct and it is the whole point: the test data is being put through exactly the transformation that new data would go through in production, where you do not get to recompute the mean using the future.
A test set that comes out perfectly centred is evidence that the scaler was fitted on it, which means the evaluation is compromised.
The same logic extends to deployment. The scaler is part of the model and has to be saved with it. A model deployed without its scaler, receiving raw features, will produce confident nonsense — which is one more argument for the pipeline, since saving the pipeline saves both.
Just use a pipeline
Every rule above is enforced automatically by putting the scaler in a Pipeline.
A pipeline is a single estimator. Its fit runs fit_transform on each transformer and then fit on the final model; its predict runs transform and then predict. So inside a cross-validation loop, the scaler is refitted on each training fold and the held-out fold is only ever transformed — which is exactly the correct behaviour, achieved without you having to remember it.
make_pipeline(StandardScaler(), LogisticRegression()) is two words longer than fitting the scaler by hand and removes an entire category of mistake. There is essentially no situation in which scaling outside a pipeline is preferable, and the habit is worth forming before the mistakes are.
Scaling is not the only transformation a column might want
Standardising fixes the *scale* of a feature and leaves its *shape* alone. Sometimes the shape is the problem.
A heavily skewed column — incomes, populations, response times, anything where a few values are orders of magnitude above the rest — stays heavily skewed after standardising. The mean and standard deviation are themselves distorted by the tail, so most of the data ends up squeezed just below zero with a few enormous values above it, which is exactly the situation that harmed the distance-based models in the first place.
Three responses. A log transform, via FunctionTransformer(np.log1p), compresses the tail and often turns a multiplicative relationship into an additive one that a linear model can fit. log1p rather than log because it handles zeros without producing negative infinity.
PowerTransformer finds a transformation that makes the column as close to normal as it can, and standardises as it goes. It handles the choice for you and is the reasonable default when you know a column is skewed and do not want to think about which transformation.
QuantileTransformer maps the column onto a uniform or normal distribution by rank. It is the most aggressive of the three: it will normalise anything, at the cost of discarding the actual spacing between values and of behaving unpredictably on values outside the training range.
All three are transformers with the usual fit/transform, so they go in a pipeline in the same place a scaler would, and they are subject to exactly the same training-data-only rule.
The order of the steps
When several transformations apply to the same column, the order is not arbitrary.
Imputation comes before scaling, because a scaler cannot compute a mean over missing values — it raises. Encoding comes before scaling for the same reason, since a scaler cannot subtract a mean from a string.
Skew correction comes before standardising, since the point is to fix the shape and then fix the location and spread of the fixed shape.
Feature selection is usually last among the transformers and before the model, so that the selection sees the features in the form the model will.
A Pipeline executes its steps in the order given, so writing them in the wrong order produces either an error or a silently worse model. The sequence that covers most cases is: impute, encode, transform skew, scale, select, fit.
Should I scale the target as well? Rarely for regression - it changes the units your errors are reported in. When it genuinely helps, TransformedTargetRegressor does it and inverts the transformation on predict, so the predictions come back in the original units.
Does scaling help accuracy on trees, ever? No. Exactly zero, which the last editor demonstrates. It is harmless and pointless.
What about one-hot columns - do those get scaled? They are already 0 and 1, so standardising them mostly adds noise to the interpretation. It is common to scale only the numeric columns, which is what a ColumnTransformer is for.
Categories of model, and what each one wants
A summary worth keeping, because "does this need scaling" comes up for every new estimator.
Needs it, badly. k-nearest neighbours, k-means, SVM with any kernel, PCA and anything built on it, and neural networks. All of these either measure distance or descend a gradient, and both are dominated by whichever feature has the largest units.
Needs it, for the penalty. Ridge, Lasso, ElasticNet, and LogisticRegression with its default L2. The fit itself would be unaffected, but the regularisation penalises coefficient magnitude, and magnitude depends on units.
Does not need it, and gains nothing. Decision trees, random forests, extra trees, gradient boosting in all its varieties, and naive Bayes. These either split one column at a time or treat the columns independently.
Does not need it, but converges faster with it. Plain LinearRegression has a closed-form solution and is genuinely indifferent; the iterative solvers used for very large problems are not.
The rule that covers the whole table: if the algorithm ever adds two different features together, compares them, or measures a distance across them, it needs them on a common scale. If it only ever looks at one feature at a time, it does not.
Things to try
Run the second editor. 0.69 against 0.95, from one line. That is the argument in a single number.
Watch a tree ignore it. The last editor reports a change of exactly zero for the forest. Try MinMaxScaler instead and confirm it is still zero.
Break the range. In the third editor, change the outlier to 10000 and see how much further MinMax compresses the ordinary values.
Cheat deliberately. In the fourth editor, fit the scaler on all of X and note that the test mean becomes 0 - the signature of a compromised evaluation.
Where this leaves you
Scale for distance-based and regularised models, do not bother for trees, prefer StandardScaler unless outliers argue for RobustScaler, and fit it inside a pipeline so that the training-data-only rule holds without anyone having to remember it.
Check yourself
0 of 4
Answer without scrolling back up.
Which of these is unaffected by feature scaling?
Trees split one column at a time on a threshold, and a monotonic rescaling moves the threshold identically. The measured change is exactly zero.
After scaling correctly, what should the test set's mean be?
A test set that comes out perfectly centred is evidence the scaler was fitted on it, which compromises the evaluation.
Why prefer RobustScaler when the data has outliers?
Median and IQR ignore the tails. MinMaxScaler does the opposite - one outlier defines the range and compresses everything else.
What does Normalizer do?
It works on rows rather than columns, making samples comparable in direction rather than magnitude. It is not a substitute for the column scalers.
Cheat sheet
Scaling Features
Half the models in the library are seriously damaged by unscaled features and half are completely indifferent. Knowing which is which saves both wasted work and silently bad results.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.