Which models need it, which are entirely indifferent to it, and the one rule about when it is allowed to look at your data.
Overview
What scaling does
Features in a dataset often have different units and different ranges. Age might run from 0 to 100. Income might run from 20,000 to 500,000. A distance-based model like k-nearest neighbours treats both columns equally. Without scaling, the column with the larger numbers dominates every distance calculation.
Scaling puts each column on a comparable range. It does not change the information in the data. It changes the numbers so that no single column drowns out the others by sheer magnitude.
Not every model needs scaling. Tree-based models split on one feature at a time and are unaffected by the scale of the others. Linear models, SVMs and neural networks all benefit from it. Knowing which models need scaling and which do not saves you from both unnecessary work and silent errors.
The most common mistake with scaling is fitting the scaler on the full dataset before splitting. The scaler then learns the mean and standard deviation of the test rows, and the test score is no longer honest. The scaler must be fitted on the training set only. A pipeline handles this automatically.
This page covers what the standard scaler does, why some models need it, and how to apply it without leaking information across the train-test boundary.
StandardScaler learns two numbers per column. The mean and the standard deviation. And transforms by subtracting the first and cutting by the second. The result has mean 0 and standard deviation 1 in each column.
That is the entire op. The shape of the spread is unchanged. A skewed column stays skewed, outliers stay outliers. The scaled spots of each value are preserved just. Scaling moves and stretches. It doesn't reshape.
The learned numbers live in mean_ and scale_. This Is worth knowing because those two arrays are the entire fitted state and the thing that must not be learned from the test data.
Worth knowing
StandardScaler subtracts the mean and divides by the standard deviation, learning both from the data it is fitted on.
Models that measure distance or penalise coefficients need it: k-NN, SVM, PCA, k-means, and anything regularised.
Trees and their ensembles do not - they split one column at a time, so a monotonic rescaling changes nothing.
MinMaxScaler maps to a fixed range and is wrecked by outliers; RobustScaler uses the median and IQR instead.
Fit the scaler on the training data only, then transform both halves with it - the test set will not be centred, and should not be.
Put it in a Pipeline and that rule becomes impossible to break by accident.
Scaling Features: A Practical Guide
Half the models in the library are seriously damaged by unscaled features and half are fully indifferent. Knowing which is which saves both wasted work and silently bad results.
What StandardScaler learns
Two numbers per column, and the transform is a subtraction and a division.
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_wine
X, y = load_wine(return_X_y=True)
print("largest feature range is %d times the smallest"
% int(X.max(axis=0).max() / X.max(axis=0).min()))
print()
raw = cross_val_score(KNeighborsClassifier(), X, y, cv=5)
scaled = cross_val_score(make_pipeline(StandardScaler(), KNeighborsClassifier()),
X, y, cv=5)
print("k-NN unscaled:", round(raw.mean(), 4))
print("k-NN scaled :", round(scaled.mean(), 4))
Output
Three scalers, one outlier
The choice between them is a choice about what to do with extreme values.
example_03.pyscikit-learn
from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler
import numpy as np
X = np.array([[1.0], [2.0], [3.0], [4.0], [100.0]]) # one outlier
for scaler in (StandardScaler(), MinMaxScaler(), RobustScaler()):
out = scaler.fit_transform(X).ravel()
print("%-16s %s" % (type(scaler).__name__, np.round(out, 3)))
print()
print("MinMax squashes the four ordinary values into a hundredth of the range.")
print("RobustScaler uses the median and IQR, so they keep a sensible spread")
print("and the outlier is simply far away.")
Output
Fit on train, transform on both
The test set does not end up centred on zero, and that is the point.
example_04.pyscikit-learn
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_wine
X, y = load_wine(return_X_y=True)
X_tr, X_te, _, _ = train_test_split(X, y, test_size=0.3, random_state=0)
scaler = StandardScaler().fit(X_tr) # training data only
print("train mean after scaling:", round(scaler.transform(X_tr).mean(), 10))
print("test mean after scaling:", round(scaler.transform(X_te).mean(), 4))
print()
print("the test set is NOT centred on zero, and should not be -")
print("it is transformed with the training set's numbers, not its own.")
Output
The pipeline does that for you
Which is why scaling and pipelines are taught together and used together.
example_05.pyscikit-learn
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LogisticRegression
pipe = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))
print("a pipeline is one estimator:")
print(" steps:", [name for name, _ in pipe.steps])
print()
print("calling pipe.fit(X, y) runs:")
print(" scaler.fit_transform(X) then model.fit(scaled, y)")
print("calling pipe.predict(X) runs:")
print(" scaler.transform(X) then model.predict(scaled)")
print()
print("which is why the scaler never sees the data it is tested on.")
Output
Trees do not care at all
One of these two models moves. The other does not move by any amount at all.
example_06.pyscikit-learn
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_wine
X, y = load_wine(return_X_y=True)
for name, model in [("LogisticRegression", LogisticRegression(max_iter=5000)),
("RandomForest", RandomForestClassifier(random_state=0))]:
raw = cross_val_score(model, X, y, cv=5).mean()
sca = cross_val_score(make_pipeline(StandardScaler(), model), X, y, cv=5).mean()
print("%-20s unscaled %.4f scaled %.4f change %+.4f"
% (name, raw, sca, sca - raw))
print()
print("trees split one column at a time, so a monotonic rescaling")
print("cannot change which split they choose.")
Output
Why some models need it
Two families of method are affected, for two different reasons.
any of it that measures distance. k-nearest neighbours, SVMs with an RBF kernel, k-means, PCA. These find how far apart two samples are, often by summing squared differences across features. A column measured in thousands adds thousands of times more to that sum than a column measured in single digits. The distance is in effect decided by whichever feature happens to have the largest units. The model isn't weighing the features by importance. It's weighing them by unit.
The editor above shows the cost: k-NN on the wine dataset scores 0.69 unscaled and 0.95 scaled. That isn't a tuning improvement, it's the difference between a model that works and one that doesn't. It comes from one column being measured in hundreds while others are measured in fractions.
any of it that punishes coeff size. Ridge, Lasso, ElasticNet. LogisticRegression. This Is smoothed by default. The fine is on the magnitude of the coeffs. A coeff's magnitude depends on its feature's units. Without scaling, the fine falls unevenly across features for reasons that have nothing to do with the data.
There's also a hands-on third reason: grad-based solvers settle faster and more well on scaled features. This Is why unscaled logistic regression makes ConvergenceWarning.
Why some models do not
Tree-based models — choice trees, random forests, grad boosting — are fully unaffected. The reason is structural rather than rough.
A tree splits on one feature at a time, asking whether a value is above or below a threshold. Rescaling a column monotonically moves the threshold by the same change and leaves the sorting of each sample the same. Just the same split is chosen. The tree that results is the same tree.
The editor above measures it: the random forest changes by +0.0000. Not "a little". Not at all.
This matters in practice because scaling a tree model is harmless but pointless. Adding a scaler to a pipeline that ends in a forest is a step that costs time and clarity for no gain. It matters more because people who have learned "always scale" at times conclude that trees must so be doing a thing odd, when this is one of the reasons trees are so convenient on messy tabular data: they need no preparation at all.
Choosing between the scalers
StandardScaler is the default and the right answer most of the time. It assumes nothing about the range and handles roughly even data well.
MinMaxScaler maps each column onto a fixed range, often 0 to 1. It's the choice when an method needs bounded input. Its weakness is severe. A single outlier defines the range and compresses all else against the opposite end. The editor above shows four plain values landing in the first hundredth of the range because one value is 100.
RobustScaler centres on the median and cuts by the interquartile range. Because both statistics skip the tails, extreme values don't distort the change of the plain ones. In the same demo, the four normal values keep a sensible spread and the outlier simply sits far away. It's the right default when the data has outliers you don't want to drop.
MaxAbsScaler cuts by the largest abs value and preserves zeros. This Makes it the one to use on sparse data where centring would destroy the sparsity.
Normalizer is the odd one out and a frequent source of confusion. It scales each row to unit length rather than each column. It's for making samples close in direction rather than magnitude. Text vectors, mostly. And isn't a substitute for the column scalers.
The rule that makes it honest
The scaler must be fitted on the training data only. Then used to transform both halves.
The reason is the same as all over else in this track: mean_ and scale_ are learned from data. A scaler fitted on all has seen the test set. The score afterwards isn't a score on unseen data.
The consequence surprises people the first time. After scaling right, the test set isn't centred on zero. Its mean is what it happens to be scaled to the training spread. The editor above shows 0.0184 rather than 0. That is correct and it's the whole point. The test data is being put through just the change that new data would go through in production. Here You don't get to recompute the mean using the future.
A test set that comes out just right centred is evidence that the scaler was fitted on it. This Means the test is compromised.
The same logic extends to deployment. The scaler is part of the model and has to be saved with it. A model deployed without its scaler, receiving raw features, will give confident nonsense. Which is one more arg for the pipeline. Saving the pipeline saves both.
Just use a pipeline
each rule above is enforced automatically by putting the scaler in a Pipeline.
A pipeline is a single estimator. Its fit runs fit_transform on each transformer and then fit on the final model. Its predict runs transform and then predict. So inside a cross-validation loop, the scaler is refitted on each training fold and the held-out fold is only ever transformed. Which is just the correct action, reached without you having to note it.
make_pipeline(StandardScaler(), LogisticRegression()) is two words longer than fitting the scaler by hand and drops an entire category of mistake. There's no situation in which scaling outside a pipeline is preferable. The habit is worth forming before the mistakes are.
Scaling is not the only transformation a column might want
Standardising fixes the *scale* of a feature and leaves its *shape* alone. at times the shape is the problem.
A heavily skewed column — incomes, groups, response times, any of it where a few values are orders of magnitude above the rest — stays heavily skewed after standardising. The mean and standard deviation are themselves distorted by the tail. Most of the data ends up squeezed just below zero with a few huge values above it. This Is just the situation that harmed the distance-based models in the first place.
Three responses. A log transform, via FunctionTransformer(np.log1p), compresses the tail and often turns a multiplicative tie into an additive one that a linear model can fit. log1p rather than log because it handles zeros without making minus infinity.
PowerTransformer finds a change that makes the column as close to normal as it can. Standardises as it goes. It handles the choice for you and is the reasonable default when you know a column is skewed and don't want to think about which change.
QuantileTransformer maps the column onto a uniform or normal spread by rank. It's the most aggressive of the three. It will norm any of it, at the cost of discarding the actual spacing between values and of behaving unpredictably on values outside the training range.
All three are transformers with the usual fit/transform. They go in a pipeline in the same place a scaler would. They're subject to just the same training-data-only rule.
The order of the steps
When a few changes apply to the same column, the order isn't any.
Imputation comes before scaling, because a scaler can't find a mean over missing values. It raises. Encoding comes before scaling for the same reason. A scaler can't subtract a mean from a string.
Skew correction comes before standardising, since the point is to fix the shape and then fix the location and spread of the fixed shape.
Feature pick is often last among the transformers and before the model. That the pick sees the features in the form the model will.
A Pipeline executes its steps in the order given. Writing them in the wrong order makes either an error or a silently worse model. The sequence that covers most cases is: impute, encode, transform skew, scale, select, fit.
Should I scale the target as well? Rarely for regression - it changes the units your errors are reported in. When it truly helps, TransformedTargetRegressor does it and inverts the change on predict. The guesses come back in the first units.
Does scaling help fit on trees, ever? No. just zero. This The last editor shows. It's harmless and pointless.
What about one-hot columns - do those get scaled? They're already 0 and 1. Standardising them mostly adds noise to the reading. It's common to scale only the numeric columns. This Is what a ColumnTransformer is for.
Categories of model, and what each one wants
A summary worth keeping, because "does this need scaling" comes up for each new estimator.
Needs it, badly. k-nearest neighbours, k-means, SVM with any kernel, PCA and any of it built on it. Neural networks. All of these either measure distance or descend a grad. Both are dominated by whichever feature has the largest units.
Needs it, for the fine. Ridge, Lasso, ElasticNet. LogisticRegression with its default L2. The fit itself would be unaffected. The smoothing punishes coeff magnitude, and magnitude depends on units.
Doesn't need it, and gains nothing. choice trees, random forests, extra trees, grad boosting in all its varieties, and naive Bayes. These either split one column at a time or treat the columns on its own.
Doesn't need it, but settles faster with it. Plain LinearRegression has a closed-form answer and is truly indifferent. The iterative solvers used for very large problems aren't.
The rule that covers the whole table: if the method ever adds two different features together, checks them, or measures a distance across them, it needs them on a common scale. If it only ever looks at one feature at a time, it doesn't.
Things to try with Scaling Features
Run the second editor. 0.69 against 0.95, from one line. That is the arg in a single number.
Watch a tree skip it. The last editor reports a change of just zero for the forest. Try MinMaxScaler instead and confirm it's still zero.
Break the range. In the third editor, change the outlier to 10000 and see how much further MinMax compresses the plain values.
Cheat on purpose. In the fourth editor, fit the scaler on all of X and note that the test mean turns 0 - the signature of a compromised test.
Questions people ask about Scaling Features
Which models need scaling? Distance- and gradient-based models such as KNN, SVM, logistic regression and neural networks. Tree-based models do not.
StandardScaler or MinMaxScaler? StandardScaler is the safe default. MinMaxScaler suits bounded ranges but is sensitive to outliers.
What if my data has outliers? Try a scaler that uses the median and interquartile range, found in sklearn.preprocessing, so extreme values distort the scale far less.
Do I scale the target too? Usually not for classification. For regression it is optional, and TransformedTargetRegressor handles it safely.
Why fit the scaler on training data only? Fitting on all rows lets test statistics leak into training. Fit on the training part and reuse it to transform the test part.
Should I scale one-hot columns? Normally not. They are already 0 or 1, and scaling makes them harder to read without much benefit.
Where this leaves you
Scale for distance-based and smoothed models, don't bother for trees, prefer StandardScaler unless outliers argue for RobustScaler. Fit it inside a pipeline so that the training-data-only rule holds without anyone having to note it.
Check yourself
0 of 4
Answer without scrolling back up.
Which of these is unaffected by feature scaling?
Trees split one column at a time on a threshold, and a monotonic rescaling moves the threshold identically. The measured change is exactly zero.
After scaling correctly, what should the test set's mean be?
A test set that comes out perfectly centred is evidence the scaler was fitted on it, which compromises the evaluation.
Why prefer RobustScaler when the data has outliers?
Median and IQR ignore the tails. MinMaxScaler does the opposite - one outlier defines the range and compresses everything else.
What does Normalizer do?
It works on rows rather than columns, making samples comparable in direction rather than magnitude. It is not a substitute for the column scalers.
Cheat sheet
Scaling Features
Scale features with StandardScaler and MinMaxScaler, and learn which models break without scaling and which ignore it entirely.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.