Text columns have to become numbers, and the obvious way of doing it invents an ordering that is not there.
Overview
Why it cannot be skipped
scikit-learn's estimators operate on numeric arrays. A column of strings raises rather than being converted automatically, and the error — could not convert string to float — is one of the first anyone meets on real data.
The conversion is not mechanical, because there is more than one way to do it and they encode different claims about the data. Getting it wrong does not raise; it produces a model that has been told something false.
Worth knowing
OneHotEncoder makes one column per category and is the safe default for nominal data - categories with no order.
OrdinalEncoder maps categories to integers, which implies an ordering; use it only when one genuinely exists, and pass categories= to say what it is.
handle_unknown="ignore" encodes an unseen category as all zeros instead of raising - almost always what you want in production.
get_feature_names_out() recovers the invented column names, which is how feature importances stay readable.
High-cardinality columns explode into thousands of near-empty columns; group the rare values or use a different encoding.
LabelEncoder is for the target, not for features - it accepts only 1-D input.
Encoding Categories: A Practical Guide
Every estimator in scikit-learn takes numbers. Turning "london" into a number is easy; doing it without inventing a fact that is not in the data takes one decision.
One column per category
The safe default, and the names it generates.
example_01.pyscikit-learn
Output
The category it has never seen
The default raises. Which of the two behaviours you want is a real decision.
example_02.pyscikit-learn
Output
Ordinal encoding invents an order
Left to itself it uses alphabetical order, which here makes small the largest.
example_03.pyscikit-learn
Output
drop='first', and when it matters
Two columns for a yes/no answer carry the information of one.
example_04.pyscikit-learn
Output
What high cardinality does to the matrix
Three cities is three columns. Sixteen hundred users is sixteen hundred.
example_05.pyscikit-learn
Output
Encoding belongs inside the pipeline
The label here is the city, so a perfect score is expected - the point is that the folds do not crash.
example_06.pyscikit-learn
Output
The mistake to avoid
The obvious approach is to number the categories: london 0, paris 1, berlin 2. It is one line and it is wrong for most categorical columns.
Numbering asserts an order and a spacing. It says berlin is greater than paris, that paris sits exactly between london and berlin, and that the distance from london to berlin is twice the distance from london to paris. None of that is true of cities, and a model that computes with those numbers will use every one of those false relationships.
For a linear model the damage is direct: a single coefficient is fitted for the column, so the model can only express "more city is more target", which is meaningless. For a distance-based model, berlin and london are computed as far apart while london and paris are close. For a tree the damage is milder — a tree can split the numbers into groups and partially recover — but it still has to spend splits undoing an ordering that was invented.
One-hot encoding
The safe answer is one column per category, each holding 1 when the row is that category and 0 otherwise.
No ordering is implied, because no two categories share a column. Each gets its own coefficient in a linear model, so the model can learn that paris raises the target and berlin lowers it, which numbering could never express.
OneHotEncoder learns the categories during fit and stores them in categories_, sorted. get_feature_names_out() returns the generated names — city_london, city_paris — which is what keeps coefficients and feature importances readable after the transformation.
By default it returns a sparse matrix, because with many categories almost every entry is zero. sparse_output=False gives a dense array, which is convenient for looking at and a bad idea when the number of categories is large.
The unseen category
The most important argument on the encoder is handle_unknown, and the default is the strict one.
A category present at prediction time but absent during training raises ValueError by default. That is the right behaviour while you are developing — it tells you the data has changed — and usually the wrong behaviour in production, where a new city appearing should not take the service down.
handle_unknown="ignore" encodes the unknown value as all zeros: not any of the known categories, which is an honest representation of "something I have not seen". The model then predicts from the remaining features.
This matters inside cross-validation too. Each fold refits the encoder on its own training part, so a category appearing only in the held-out fold is unknown to that fold's encoder. Without handle_unknown="ignore", cross-validation on a dataset with rare categories crashes on whichever fold happens to isolate one.
min_frequency and max_categories, added more recently, are the built-in answer to the same problem from the other end: they fold rare categories into a single "infrequent" column during fitting, so the rare values are handled by design rather than by exception.
When ordering is real
Some categories genuinely are ordered. Small, medium, large. Bronze, silver, gold. Disagree, neutral, agree. Here an integer encoding is not a lie, and it is better than one-hot because it lets the model use the ordering with a single coefficient instead of learning three independent ones.
OrdinalEncoder does it — and left to itself it sorts alphabetically, which is almost never the order you meant. The editor above shows the damage: with small, medium and large, alphabetical order produces large 0, medium 1, small 2, so the encoding says small is the biggest.
categories=[["small", "medium", "large"]] states the order explicitly, and passing it is not optional. The list is nested because the encoder handles several columns at once, one list per column.
The test for whether a column is ordinal: is there a defensible answer to "which is bigger", and is the gap between consecutive values roughly comparable? Sizes pass the first and often fail the second, which is why ordinal encoding of survey scales is common and mildly wrong.
drop, and the collinearity question
With k categories, one-hot produces k columns of which any one is determined by the others — if it is not london and not paris, it is berlin. That redundancy is called collinearity.
drop="first" removes one column per feature. For a plain linear regression this matters: with an intercept, perfectly collinear columns make the solution non-unique and the individual coefficients unstable. For anything regularised — Ridge, Lasso, LogisticRegression with its default penalty — it does not, because the penalty resolves the ambiguity, and dropping a column makes the remaining coefficients harder to interpret since they become relative to the dropped baseline.
drop="if_binary" is the sensible middle: it drops the redundant column only for two-category features, where two columns are obviously one too many, and leaves multi-category features alone.
For trees, dropping is actively unhelpful — it removes a column the tree might have split on and gains nothing.
High cardinality
One-hot encoding a column with thousands of distinct values produces thousands of columns, almost all zero, and the editor above shows 1639 distinct user ids becoming 1639 columns.
Three problems follow. The matrix becomes enormous and mostly empty. Most columns are 1 for only a handful of rows, so any coefficient fitted to them is fitted to almost no data. And the model can memorise individual ids, which looks like skill on the training data and is worthless on anyone new.
Three responses, in increasing order of effort. Group the rare values into an "other" category, which min_frequency does for you. Encode something about the category rather than its identity — for a postcode, the region; for a product, its price band. Or target encoding, replacing each category with the mean target for that category, which is compact and powerfully prone to leakage: computed on the whole dataset it tells the model the answer, so it must be computed inside the folds, which is what TargetEncoder handles.
The question worth asking first is whether the column belongs in the model at all. A user id usually carries no generalisable information; what you want is what the user *did*.
LabelEncoder is not for this
LabelEncoder appears in a great deal of example code applied to feature columns, and it is documented for encoding the target.
It accepts only 1-D input, so using it on features means looping over columns and applying it one at a time. That works, produces an ordinal encoding with all the problems above, and — the real objection — it has no handle_unknown, so a new category at prediction time raises with no option to do otherwise.
Use OrdinalEncoder for ordered features, OneHotEncoder for unordered ones, and LabelEncoder only when converting string class labels into integers for y, which most estimators do not even require.
Categories that arrive as numbers
The hardest categorical columns to spot are the ones already stored as integers, because nothing raises and nothing looks wrong.
A postcode district, a product code, a department id, a day-of-week stored as 0 to 6, a survey answer coded 1 to 5. Each is a category wearing a number, and a model handed it raw will treat department 7 as greater than department 3 and exactly twice department 3.5.
Nothing in the data announces this. The column is numeric, the estimator accepts it, the fit succeeds, and the model quietly uses an ordering that means nothing. It is the mirror image of the ordinal mistake: there, an order was invented where none existed; here, an order that happens to exist in the encoding is mistaken for one that exists in the world.
The check is to ask what the arithmetic would mean. If the average of two values is meaningless — the average of department 2 and department 8 — the column is categorical regardless of its dtype. If it is meaningful, as with an age or a price, it is genuinely numeric.
Day-of-week is the interesting middle case. It has an order, the gaps are equal, and it wraps around: Sunday is adjacent to Monday, which no integer encoding captures. The usual fix is a pair of features, the sine and cosine of the angle around the week, which makes the wrap-around explicit and is the standard trick for any cyclical quantity — hours, months, compass bearings.
Reading the model afterwards
One-hot encoding multiplies the columns, and a fitted model reports one number per column. Keeping the two aligned is what makes the result readable.
get_feature_names_out() on the encoder returns the generated names in order, and on a Pipeline or ColumnTransformer it returns the names for everything the whole chain produced. Zipping that against coef_ or feature_importances_ gives a labelled result rather than an array of numbers whose positions you have to reconstruct.
Two things to remember when reading them. The importance of a category is spread across its columns, so a feature with twenty categories has its influence divided twenty ways and will look weaker than a single numeric column carrying the same information — which is a known bias in tree-based feature importances, not a fact about the data. And with drop="first", every remaining coefficient is relative to the dropped category, so a positive coefficient means "higher than the baseline", not "high".
Do trees need one-hot encoding? scikit-learn's do, because they cannot take strings. HistGradientBoostingClassifier accepts declared categorical features natively, which is both faster and better than one-hot for high-cardinality columns.
Should I one-hot encode before or after splitting? After, and inside a pipeline - otherwise the encoder learns the category list from the test rows too.
What about missing values in a categorical column?OneHotEncoder treats NaN as its own category by default, which is often the right behaviour: "not stated" is frequently informative rather than merely absent.
Things to try
Read the alphabetical disaster. The third editor encodes small as 2 and large as 0. Look at the numbers before reading the fix.
Meet the unknown category. In the second editor, remove handle_unknown="ignore" from the safe encoder and watch it raise on tokyo.
Explode a column. In the fifth editor, raise the number of distinct users and watch the column count follow it exactly.
Group the rare ones. Add min_frequency=50 to that encoder and see how many columns survive.
Where this leaves you
One-hot for unordered categories with handle_unknown="ignore", ordinal with an explicit categories= when the order is real, drop="if_binary" as a reasonable default, and a plan for any column with more than a few dozen distinct values before it becomes a few dozen columns of nearly nothing.
Check yourself
0 of 4
Answer without scrolling back up.
Why is numbering cities 0, 1, 2 a problem?
The numbers claim berlin > paris > london and that the gaps are equal. A linear model fits one coefficient to that invented scale.
What does handle_unknown="ignore" do?
All zeros honestly represents "none of the categories I know". Without it, cross-validation crashes on any fold that isolates a rare category.
When is OrdinalEncoder the right choice?
Left to itself it sorts alphabetically, which turned small into 2 and large into 0 in the editor above.
What is LabelEncoder for?
It takes only 1-D input and has no handle_unknown, so applying it to features gives an ordinal encoding that raises on any new category.
Cheat sheet
Encoding Categories
Every estimator in scikit-learn takes numbers. Turning "london" into a number is easy; doing it without inventing a fact that is not in the data takes one decision.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.