Shuffle one column, re-score the model, and see how much it mattered. Simple, model-agnostic, and easy to misread.
Overview
The idea
If a feature matters, destroying the info in it should hurt the model. So:
Score the model on a dataset.
Shuffle one column, breaking its tie with the target while leaving its spread untouched.
Score again.
The drop is that feature's importance.
Repeat for each column.
Shuffling rather than deleting is the key detail. Deleting a column means retraining, which gives a different model and answers a different question. Shuffling keeps the same fitted model and the same input shape, so the only thing that changed is whether that column carries signal.
Because it needs nothing but guesses, it works on any model at all — linear, forest, grad-boosted, neural network.
Feature and Permutation Importance
This module needs JavaScript: the
numbers are computed in the page rather than recorded.
Worth knowing
Permutation importance breaks the link between one column and the target by shuffling it, then measures the damage.
It works on any fitted model, because it only needs predictions.
Measure it on held-out data. On training data it reports what the model memorised.
Correlated features hide each other: shuffle one and the model leans on its twin, so the measured drop is far below what that column is worth.
Feature and Permutation Importance
The most handy model-agnostic importance measure, and the exact way it lies to you.
Reading the chart
Four features are fed to a logistic model: a strong signal, a weak one, pure noise, and a near-duplicate copy of the strong signal.
noise scores about zero, which is correct and reassuring.
Now compare signal with copy of signal. They carry the same info — either would serve the model about equally well on its own — and copy of signal scores roughly half what signal does, landing at about the level of weak, a feature that is truly much less helpful.
Why: correlated features hide each other
The reason is just the method. The model here is a small random forest, and each split considers a random subset of the features. So Some trees split on signal and others on its copy. Shuffling the copy leaves all the trees that used signal working just right. So The measured drop is much smaller than the column's actual worth.
The metric is measured one column at a time, and with linked features that is the wrong unit of analysis. The honest reading isn't "the copy doesn't matter" but "either one largely substitutes for the other, and this method can't say that".
The size of the effect depends on the model. A linear model given two duplicate columns splits its weight between them. So Shuffling one still destroys half the part and both score moderately high. A forest can substitute one for the other outright, which is why the discount is so visible here — and why the trap is worst on just the tree models where importance charts are most often drawn.
This is the single most key thing to know about permutation importance, because the failure is silent and the chart looks just right reasonable.
What to do about it:
Cluster linked features first and permute whole groups together. The group's importance is then real though its members' aren't.
Check the link mat before reading any importance chart.
Use if-based permutation, which shuffles within strata of the linked features rather than globally.
Held-out data, not training data
Permutation importance found on the training set measures what the model *used to fit*, including what it stored. A feature the model overfitted to will look highly key because shuffling it destroys stored training speed.
On held-out data the same method measures what the feature adds to speed that generalises. That is almost always the question, and it's why scikit-learn's docs is emphatic about it.
Checking the two is diagnostic in itself: a feature that is very key on training data and unimportant on validation data is one the model overfitted.
Against a tree's built-in importance
Random forests and grad-boosted trees expose feature_importances_, often mean fall in impurity. It's free, since it's built up during training, and it has a known bias: it favours high-cardinality features. A column with many distinct values offers more possible split points, so it's chosen more often and builds up more credit. Even a random ID column will score above a handy binary flag.
Permutation importance doesn't have that bias, costs a re-scoring pass per feature, and applies to models that have no built-in measure at all.
Repeats
Shuffling is random, so a single shuffle is a noisy guess. The repeats control above means a few, and the wobble as you change the seed at one repeat shrinks visibly as you raise it. Five to ten is common.
Two copies of one signal, and the discount
The central trap is measurable. Feed a model four features — a strong signal, a weak one, pure noise, and a near-duplicate copy of the strong signal — and read the permutation importances:
feature
importance
signal
high
copy of signal
~half of signal, level with weak
weak
low
noise
~0
noise scoring zero is correct and reassuring. But copy of signal carries the same info as signal — either alone would serve the model — and it scores about half, down at the level of a truly weak feature. The cause is the method: importance is measured one column at a time. So When the copy is shuffled each tree that split on the first still works, and the measured drop is small. The honest reading isn't "the copy doesn't matter" but "either substitutes for the other, and this method can't say so". And the effect is worst on the tree models where these charts are drawn most. A forest can swap one linked column for the other outright. The fixes are to cluster linked features and permute the group, or to check the link mat before trusting any importance bar. Find it on held-out data, too: on training data it rewards what the model stored.
Where it goes wrong with Feature and Permutation Importance
Reading it on linked features. The central trap.
finding it on training data. Measures memorisation.
Treating it as causal. It says the model used a feature, not that the feature causes the outcome.
Checking across models. Importance is scaled to one fitted model. A different model can rank the same features differently and both be right about themselves.
Questions people ask about Feature and Permutation Importance
How does permutation importance work? Shuffle one column in held-out data and measure how much the score drops. A large drop means the model relied on that feature.
Why use held-out data? On training data, an overfit model can rely on features that carry only noise. Held-out data shows which features help predictions that generalise.
Why do correlated features get low scores? When two columns carry the same signal, shuffling one leaves the other to cover for it. Both look unimportant, so group correlated features or drop one.
How is it different from tree feature importance? Tree importance counts how much splits reduce impurity and favours high-cardinality features. Permutation importance measures the effect on actual predictions and works with any model.
How many repeats should I use? Ten or more shuffles per feature, so you can see the spread. A feature whose drop is within the noise is not clearly useful.
Can importance be negative? Yes, slightly. It means shuffling the feature happened to improve the score, which suggests the feature is not helping.
Break one column and see what it costs
Permutation importance shuffles a column and measures the damage. It works on any fitted model. It disagrees with a tree's built-in importances in ways worth understanding.
example_01.pyscikit-learn
import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.inspection import permutation_importance
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(0)
X, y = make_classification(n_samples=3000, n_features=6, n_informative=3,
n_redundant=0, flip_y=0.08, shuffle=False,
random_state=0)
X = np.column_stack([X, rng.normal(size=len(X)), X[:, 0]])
names = ["real_0", "real_1", "real_2", "filler_3", "filler_4", "filler_5",
"pure_noise", "copy_of_real_0"]
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0)
rf = RandomForestClassifier(n_estimators=250, random_state=0).fit(Xtr, ytr)
print("baseline test accuracy %.4f" % rf.score(Xte, yte))
print()
print("do it by hand on one column -- shuffle it and re-score:")
col = 0
scores = []
for rep in range(10):
Xs = Xte.copy()
rng.shuffle(Xs[:, col])
scores.append(rf.score(Xs, yte))
print(" shuffling %s ten times: accuracy %.4f +/- %.4f"
% (names[col], np.mean(scores), np.std(scores)))
print(" the drop is %.4f. that is its permutation importance."
% (rf.score(Xte, yte) - np.mean(scores)))
print()
r = permutation_importance(rf, Xte, yte, n_repeats=10, random_state=0)
print("%-16s %12s %10s %14s" % ("feature", "permutation", "sd", "tree built-in"))
order = np.argsort(-r.importances_mean)
for i in order:
print("%-16s %12.4f %10.4f %14.4f"
% (names[i], r.importances_mean[i], r.importances_std[i],
rf.feature_importances_[i]))
print()
print("filler columns and pure_noise carry nothing. the tree's built-in")
print("importance still scores them above zero because a continuous column")
print("offers many split points. permutation importance, on held-out rows,")
print("puts them at zero where they belong.")
Output
The next block shows how correlated features split their importance and how to read the result honestly.
example_02.pyscikit-learn
import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.inspection import permutation_importance
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(0)
X, y = make_classification(n_samples=3000, n_features=6, n_informative=3,
n_redundant=0, flip_y=0.08, shuffle=False,
random_state=0)
X = np.column_stack([X, rng.normal(size=len(X)), X[:, 0]])
names = ["real_0", "real_1", "real_2", "filler_3", "filler_4", "filler_5",
"pure_noise", "copy_of_real_0"]
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0)
rf = RandomForestClassifier(n_estimators=250, random_state=0).fit(Xtr, ytr)
r = permutation_importance(rf, Xte, yte, n_repeats=10, random_state=0)
print("the trap. real_0 and copy_of_real_0 are the same column:")
for i in (0, 7):
print(" %-16s permutation %.4f" % (names[i], r.importances_mean[i]))
print(" each scores about half. shuffling one leaves the other standing.")
print()
print("drop the copy and refit:")
keep = [i for i in range(8) if i != 7]
rf2 = RandomForestClassifier(n_estimators=250, random_state=0).fit(Xtr[:, keep], ytr)
r2 = permutation_importance(rf2, Xte[:, keep], yte, n_repeats=10, random_state=0)
print(" real_0 importance is now %.4f, up from %.4f."
% (r2.importances_mean[0], r.importances_mean[0]))
print(" the column did not become more useful. it stopped sharing credit.")
print()
print("correlated features SPLIT their importance. a zero means 'the model")
print("can get by without it', not 'this column is useless'.")
print()
print("measure on held-out data and use the metric you care about:")
r3 = permutation_importance(rf, Xte, yte, n_repeats=10, random_state=0,
scoring="roc_auc")
order = np.argsort(-r.importances_mean)
print(" scored by ROC AUC instead of accuracy:")
for i in order[:4]:
print(" %-16s %.4f" % (names[i], r3.importances_mean[i]))
Output
Check yourself
0 of 3
Answer without scrolling back up.
Why shuffle a column rather than delete it?
Shuffling keeps the same fitted model and the same distribution, so the only thing that changed is whether that column still carries signal.
Two near-identical informative features both show low importance. Why?
The metric is measured one column at a time. The honest reading is 'either one suffices', which this method cannot express - permute correlated groups together instead.
What bias does a tree's built-in impurity importance have?
A random ID column can score above a useful binary flag. Permutation importance does not have this bias, at the cost of a re-scoring pass per feature.
Cheat sheet
Feature and Permutation Importance
Shuffle one feature at a time and measure the drop in accuracy. See why correlated features split the importance and how to handle it.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.