The object that makes leakage impossible by construction, because every fold fits its own preprocessing.
Overview
The same demonstration, one level up
The chart is the one from the leakage module, relabelled: 300 rows of pure noise, five columns chosen by their link with the label, and a choice about *when* that choice is made.
By hand, before cross-validation, the reported fit sits well above the 50% the data deserves. Inside a pipeline. Here Each fold selects its own columns from its own training rows, it falls to the truth.
The model is the same in both. What changed is which object owns the fitting, and that is the entire arg for pipelines. It isn't about tidiness.
Pipelines
This module needs JavaScript: the
numbers are computed in the page rather than recorded.
Worth knowing
A pipeline chains transformers and a final estimator into one object with one fit and one predict.
Cross-validating the pipeline refits every step on each fold's training portion. That is what makes leakage structurally impossible.
It also removes the train/serve gap: the same object that was fitted is the thing you save and deploy.
ColumnTransformer applies different steps to different columns, which is how real tables get handled.
Pipelines
Not a convenience wrapper. The setup in which the order of ops can't be got wrong.
What goes wrong by hand
The by-hand workflow looks reasonable written out:
X = impute(X)
X = scale(X)
X = select_features(X, y)
scores = cross_val_score(model, X, y, cv=5)
each one of those three steps has been fitted on all the data, and cross_val_score then splits data that has already been contaminated. Each fold was scaled using a mean that included its own validation rows, imputed using a median that included them, and reduced to features chosen partly because they linked with their labels.
Nothing errors. The score is simply too high, by an amount that depends on the data and can't be guessed after the fact.
cross_val_score now receives one estimator. On each fold it calls fit on the pipeline with that fold's training rows, and the pipeline fits the imputer, then the scaler, then the feature selector, then the model. All on those rows only. Scoring calls transform on the validation rows using the params just learned.
The correct order isn't a thing you remembered to do. It's the only thing the object can do.
The three other things it buys
Setting search over prep. Because the steps are part of one estimator, a grid can range over them.
Whether the median beats the mean turns a question the search answers, on the same footing as the model's own params. Doing this by hand right is possible and no one does it.
No train/serve gap. The fitted pipeline is one object. Save it, load it in the service, call predict on raw input. There's no second setup of the prep to drift out of step with the first. Which is a real and common production failure, and a silent one.
Different columns, different treatment.ColumnTransformer routes numeric columns to an imputer and scaler, cat ones to a different imputer and a one-hot encoder, and text to a vectoriser, then hands the mixed result to the model. That is what an actual table needs, and it stays one object.
Cross-validation still needs care
A pipeline stops leakage between prep and test. It doesn't fix splits that were wrong to begin with.
If rows are grouped — a few scans per patient — use GroupKFold so a patient can't appear on both sides. If the data is a time series, use TimeSeriesSplit so training always precedes validation. And if you tune settings on the same folds you report, that number is optimistic too; nested cross-validation is the honest version.
The size of the leak, in accuracy points
The reason to insist on a pipeline is that the leak is large and silent. Take 300 rows of pure noise — features with no tie to the label at all — and select the 20 columns most linked with the label, then cross-validate:
when the columns are chosen
reported accuracy
truth
once, on all the data, before CV
~65%
50%
inside each fold, on its training rows only
~50%
50%
The data can't support better than chance, and the honest pipeline says so. The by-hand version reports 65% because the feature pick has already seen each fold's validation rows: it picked the columns that happened to correlate with those labels too, so each fold is scored on features chosen partly *because* they match its answers. Nothing errors, and the inflation — fifteen points here, more with more pick features. Can't be guessed after the fact, because it depends on data the leak already consumed. The pipeline makes the correct order the only order the object can execute. Each fold fits the selector, the scaler and the model on its own training rows and merely transforms the validation rows. The number it reports is a real guess.
Beyond stopping the leak, the pipeline buys three things that are awkward by hand. Setting search can range over the prep — whether the median beats the mean turns a tuned choice on the same footing as the model's own settings. There's no train/serve gap, because the fitted pipeline is one object saved and loaded whole, so there's no second copy of the prep to drift out of step. And ColumnTransformer routes numeric columns to a scaler, cats to an encoder, and text to a vectoriser, all inside the same object. What a pipeline can't do is validate the splits themselves: if rows are grouped by patient or sorted in time, it still needs GroupKFold or TimeSeriesSplit to keep the folds honest.
One special case catches people: resampling. SMOTE and other over-samplers must run on the training fold only, and scikit-learn's own Pipeline doesn't enforce that — imblearn's pipeline does. A log transform, by contrast, has no fitted params and can't leak. So It need not live in the pipeline at all, though keeping it there's still worth the tidiness of one place for the prep.
Where it goes wrong with Pipelines
**Calling fit on the test data.** Use transform. fit_transform on the test set refits all and undoes the whole point.
Steps that aren't fitted. A log transform has no params and can't leak, so it doesn't have to be in the pipeline. But putting it there keeps the prep in one place. This Is worth more than the exemption.
Resampling inside a plain pipeline. SMOTE and friends must run on the training fold only. imblearn's pipeline handles this. Scikit-learn's doesn't.
given a pipeline validates your splits. It fits each fold right. It can't know that your folds mix one patient across both sides.
Questions people ask about Pipelines
Why use a pipeline instead of separate steps? A pipeline guarantees every transform is fit only on training data inside each fold. It also makes the whole workflow one object you can save and reuse.
Does a pipeline prevent leakage? It prevents the preprocessing kind, when you cross-validate the whole pipeline. It cannot catch target leakage or time-based leakage.
How do I tune hyperparameters of pipeline steps? Use the step name, a double underscore, and the parameter, such as svc__C. Pass these to GridSearchCV or RandomizedSearchCV.
What is ColumnTransformer for? It applies different preprocessing to different columns, for example scaling numeric columns and one-hot encoding categorical ones, then joins the results.
Can I put custom code in a pipeline? Yes. Wrap a function in FunctionTransformer, or write a class with fit and transform methods.
How do I save a pipeline? Serialise the fitted pipeline with joblib. Loading it gives you the preprocessing and model together, so production predictions match training.
The same workflow, wrong then right
prep outside the pipeline scores higher and is a lie. Here are both versions side by side, on the same data.
example_01.pyscikit-learn
import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import cross_val_score, StratifiedKFold
rng = np.random.default_rng(0)
X = rng.normal(size=(200, 800))
y = rng.integers(0, 2, 200)
cv = StratifiedKFold(5, shuffle=True, random_state=0)
print("200 rows, 800 noise columns, a random label. nothing is learnable.")
print()
Xs = StandardScaler().fit_transform(X)
Xk = SelectKBest(f_classif, k=25).fit_transform(Xs, y)
print("preprocessing first, then cross-validating the model:")
print(" cv %.4f" % cross_val_score(LogisticRegression(max_iter=2000),
Xk, y, cv=cv).mean())
pipe = make_pipeline(StandardScaler(), SelectKBest(f_classif, k=25),
LogisticRegression(max_iter=2000))
print("the same three steps, inside a Pipeline:")
print(" cv %.4f" % cross_val_score(pipe, X, y, cv=cv).mean())
print()
print("identical code, identical data. the first number is a lie because the")
print("selector saw the test rows' labels. cross_val_score refits every step")
print("of a Pipeline inside each fold, which is the whole reason it exists.")
print()
print("a Pipeline is just a list of (name, step) pairs:")
for name, step in pipe.steps:
print(" %-16s %s" % (name, type(step).__name__))
print(" every step but the last needs fit_transform. the last needs fit.")
Output
The next block shows how to tune parameters through a pipeline and how ColumnTransformer handles mixed column types.
example_02.pyscikit-learn
import numpy as np
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.compose import ColumnTransformer
from sklearn.model_selection import StratifiedKFold, GridSearchCV
rng = np.random.default_rng(0)
X = rng.normal(size=(200, 800))
y = rng.integers(0, 2, 200)
cv = StratifiedKFold(5, shuffle=True, random_state=0)
pipe = make_pipeline(StandardScaler(), SelectKBest(f_classif, k=25),
LogisticRegression(max_iter=2000))
print("you can reach into it to tune any step's parameters:")
grid = GridSearchCV(pipe, {"selectkbest__k": [5, 25, 100],
"logisticregression__C": [0.01, 1.0]},
cv=cv, n_jobs=1)
grid.fit(X, y)
print(" best params:", grid.best_params_)
print(" best cv score: %.4f <- still noise, as it should be" % grid.best_score_)
print(" the double underscore is 'step name, then parameter name'.")
print()
print("ColumnTransformer handles mixed column types.")
data = np.column_stack([rng.normal(size=(300, 3)),
rng.integers(0, 3, 300).astype(float)])
data[rng.random(300) < 0.08, 0] = np.nan
pre = ColumnTransformer([
("numbers", make_pipeline(SimpleImputer(), StandardScaler()), [0, 1, 2]),
("category", OneHotEncoder(handle_unknown="ignore"), [3]),
])
out = pre.fit_transform(data)
print(" %d input columns -> %d output columns" % (data.shape[1], out.shape[1]))
print(" numeric: imputed and scaled. category: three indicator columns.")
print(" each transformer only touches its own columns.")
print()
print("one object holds the whole recipe, so the thing you cross-validate")
print("is the thing you deploy. that is the argument -- not tidiness.")
Output
Check yourself
0 of 3
Answer without scrolling back up.
What does cross-validating a pipeline do that cross-validating a model does not?
Preprocessing fitted by hand beforehand has already seen every validation row. Inside a pipeline the correct order is the only thing the object can do.
Why does a pipeline remove the train/serve gap?
Reimplementing preprocessing in the serving code is a common production failure and a silent one - the model receives differently-prepared inputs and quietly degrades.
What does a pipeline NOT protect you from?
It fits each fold correctly but cannot know your folds are badly formed. Grouped data needs GroupKFold and time series need TimeSeriesSplit.
Cheat sheet
Pipelines
Chain preprocessing and modelling into one pipeline object that prevents data leakage and locks every step into the correct order.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.