Text columns have to become numbers, and the obvious way of doing it invents an ordering that is not there.
Overview
Why it cannot be skipped
Real-world data almost always contains categorical columns. City names, product types, colour labels, yes/no flags. These columns carry useful information, but scikit-learn cannot accept strings as input. Every column must be numeric before it reaches fit.
There are several ways to turn a category into a number. Each one makes a different claim about the data. Setting London=0, Paris=1, Berlin=2 says there is an order. One-hot encoding avoids that by making a new binary column for each category.
The wrong encoding does not raise an error. It runs and gives a model false data. A linear model trained on ordinal city names will treat Berlin as twice Paris. That is nonsense, and the model has no way to know it.
The right choice depends on the column and the model. Ordinal encoding is right for groups that have a real order, like size. One-hot is right for groups with no order. Target encoding helps when there are many groups and one-hot would make too many columns.
This page covers each encoding, the mistakes each one stops, and how to put them in a pipeline. It also covers the drop setting in one-hot encoding and why it matters for linear models.
scikit-learn's estimators operate on numeric arrays. A column of strings raises rather than being converted automatically. The error. couldn't convert string to float. Is one of the first anyone meets on real data.
The conversion isn't mechanical, because there's more than one way to do it and they encode different claims about the data. Getting it wrong doesn't raise. It makes a model that has been told a thing false.
Worth knowing
OneHotEncoder makes one column per category and is the safe default for nominal data - categories with no order.
OrdinalEncoder maps categories to integers, which implies an ordering; use it only when one genuinely exists, and pass categories= to say what it is.
handle_unknown="ignore" encodes an unseen category as all zeros instead of raising - almost always what you want in production.
get_feature_names_out() recovers the invented column names, which is how feature importances stay readable.
High-cardinality columns explode into thousands of near-empty columns; group the rare values or use a different encoding.
LabelEncoder is for the target, not for features - it accepts only 1-D input.
Encoding Categories: A Practical Guide
each estimator in scikit-learn takes numbers. Turning "london" into a number is easy. Doing it without inventing a fact that isn't in the data takes one choice.
The default raises. Which of the two behaviours you want is a real decision.
example_02.pyscikit-learn
from sklearn.preprocessing import OneHotEncoder
import pandas as pd
train = pd.DataFrame({"city": ["london", "paris"]})
new = pd.DataFrame({"city": ["london", "tokyo"]})
strict = OneHotEncoder(sparse_output=False).fit(train[["city"]])
try:
strict.transform(new[["city"]])
except ValueError as e:
print("default:", str(e).split("\n")[0][:90])
print()
safe = OneHotEncoder(sparse_output=False, handle_unknown="ignore").fit(train[["city"]])
print("handle_unknown='ignore' gives all zeros for the unseen value:")
print(safe.transform(new[["city"]]).astype(int))
Output
Ordinal encoding invents an order
Left to itself it uses alphabetical order, which here makes small the largest.
example_03.pyscikit-learn
from sklearn.preprocessing import OrdinalEncoder
import pandas as pd
df = pd.DataFrame({"size": ["small", "large", "medium", "small"]})
# Wrong: alphabetical order has no meaning here.
auto = OrdinalEncoder().fit(df[["size"]])
print("default (alphabetical):", auto.categories_[0].tolist())
print(" encoded as:",
auto.transform(df[["size"]]).ravel().astype(int).tolist())
print()
order = [["small", "medium", "large"]]
told = OrdinalEncoder(categories=order).fit(df[["size"]])
print("with categories= given:", told.categories_[0].tolist())
print(" encoded as:",
told.transform(df[["size"]]).ravel().astype(int).tolist())
Output
drop='first', and when it matters
Two columns for a yes/no answer carry the information of one.
example_04.pyscikit-learn
from sklearn.preprocessing import OneHotEncoder
import pandas as pd
df = pd.DataFrame({"answer": ["yes", "no", "yes"]})
full = OneHotEncoder(sparse_output=False).fit(df[["answer"]])
dropped = OneHotEncoder(sparse_output=False, drop="first").fit(df[["answer"]])
print("all columns :", list(full.get_feature_names_out()))
print(full.transform(df[["answer"]]).astype(int))
print()
print("drop='first':", list(dropped.get_feature_names_out()))
print(dropped.transform(df[["answer"]]).astype(int))
print()
print("the dropped column carried no information the other did not:")
print("'not no' and 'yes' are the same statement.")
Output
What high cardinality does to the matrix
Three cities is three columns. Sixteen hundred users is sixteen hundred.
example_05.pyscikit-learn
from sklearn.preprocessing import OneHotEncoder
import pandas as pd
import numpy as np
rng = np.random.RandomState(0)
ids = ["user_%04d" % i for i in rng.randint(0, 5000, size=2000)]
df = pd.DataFrame({"user": ids, "city": rng.choice(["a", "b", "c"], 2000)})
for col in ("city", "user"):
enc = OneHotEncoder(sparse_output=False).fit(df[[col]])
print("%-6s %5d distinct -> %5d columns"
% (col, df[col].nunique(), enc.transform(df[[col]]).shape[1]))
print()
print("one-hot encoding a high-cardinality column produces a matrix")
print("that is almost entirely zeros, and a model that memorises ids.")
Output
Encoding belongs inside the pipeline
The label here is the city, so a perfect score is expected - the point is that the folds do not crash.
example_06.pyscikit-learn
from sklearn.preprocessing import OneHotEncoder
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import cross_val_score
import pandas as pd
import numpy as np
rng = np.random.RandomState(0)
n = 400
city = rng.choice(["london", "paris", "berlin"], n)
y = (city == "paris").astype(int) # the answer IS the city
X = pd.DataFrame({"city": city})
pipe = make_pipeline(
OneHotEncoder(handle_unknown="ignore"),
LogisticRegression(max_iter=1000))
print("cross-validated accuracy:", round(cross_val_score(pipe, X, y, cv=5).mean(), 4))
print("(1.0 by construction - the task is trivial on purpose)")
print()
print("the encoder is refitted inside each fold, so a category that")
print("appears only in the held-out part is handled, not crashed on.")
Output
The mistake to avoid
The obvious approach is to number the categories. London 0, paris 1, berlin 2. It's one line and it's wrong for most cat columns.
Numbering asserts an order and a spacing. It says berlin is greater than paris, that paris sits just between london and berlin. That the distance from london to berlin is twice the distance from london to paris. None of that is true of cities. A model that finds with those numbers will use each one of those false ties.
For a linear model the damage is direct. A single coeff is fitted for the column. The model can only express "more city is more target". This Is meaningless. For a distance-based model, berlin and london are found as far apart while london and paris are close. For a tree the damage is milder. A tree can split the numbers into groups and in part recover. But it still has to spend splits undoing an sorting that was invented.
One-hot encoding
The safe answer is one column per category, each holding 1 when the row is that category and 0 or else.
No sorting is implied, because no two categories share a column. Each gets its own coeff in a linear model. The model can learn that paris raises the target and berlin lowers it. This Numbering could never express.
OneHotEncoder learns the categories during fit and stores them in categories_, sorted. get_feature_names_out() returns the made names. city_london, city_paris. Which is what keeps coeffs and feature importances readable after the change.
By default it returns a sparse mat. With many categories almost each entry is zero. sparse_output=False gives a dense array. This Is convenient for looking at and a bad idea when the number of categories is large.
The unseen category
The most key arg on the encoder is handle_unknown. The default is the strict one.
A category present at guess time but absent during training raises ValueError by default. That is the right action while you're developing. It tells you the data has changed. And often the wrong action in production. Here A new city appearing shouldn't take the service down.
handle_unknown="skip" codes the unknown value as all zeros. Not any of the known categories. This Is an honest form of "a thing I haven't seen". The model then predicts from the left features.
This matters inside cross-validation too. Each fold refits the encoder on its own training part. A category appearing only in the held-out fold is unknown to that fold's encoder. Without handle_unknown="skip", cross-validation on a dataset with rare categories crashes on whichever fold happens to isolate one.
min_frequency and max_categories, added more recently, are the built-in answer to the same problem from the other end. They fold rare categories into a single "infrequent" column during fitting. The rare values are handled by design rather than by exception.
When ordering is real
Some categories truly are sorted. Small, medium, large. Bronze, silver, gold. Disagree, neutral, agree. Here an int encoding isn't a lie. It's better than one-hot because it lets the model use the sorting with a single coeff instead of learning three free ones.
OrdinalEncoder does it — and left to itself it sorts alphabetically. This Is almost never the order you meant. The editor above shows the damage. With small, medium and large, alphabetical order makes large 0, medium 1, small 2. The encoding says small is the biggest.
categories=[["small", "medium", "large"]] states the order by name. Passing it isn't optional. The list is nested because the encoder handles a few columns at once, one list per column.
The test for whether a column is ordinal. Is there a defensible answer to "which is bigger". Is the gap between back to back values roughly close? Sizes pass the first and often fail the second. This Is why ordinal encoding of survey scales is common and mildly wrong.
drop, and the collinearity question
With k categories, one-hot makes k columns of which any one is found by the others. If it isn't london and not paris, it's berlin. That redundancy is called collinearity.
drop="first" drops one column per feature. For a plain linear regression this matters. With an intercept, just right collinear columns make the answer non-unique and the single coeffs unstable. For any of it smoothed. Ridge, Lasso, LogisticRegression with its default fine. It doesn't. The fine resolves the ambiguity. Dropping a column makes the left coeffs harder to read since they become scaled to the dropped baseline.
drop="if_binary" is the sensible middle: it drops the redundant column only for two-category features. Here Two columns are obviously one too many, and leaves multi-category features alone.
For trees, dropping is actively unhelpful. It drops a column the tree might have split on and gains nothing.
High cardinality
One-hot encoding a column with thousands of distinct values makes thousands of columns, almost all zero. The editor above shows 1639 distinct user ids turning 1639 columns.
Three problems follow. The mat turns huge and mostly empty. Most columns are 1 for only a handful of rows. Any coeff fitted to them is fitted to almost no data. And the model can store single ids. This Looks like skill on the training data and is worthless on anyone new.
Three responses, in growing order of effort. Group the rare values into an "other" category. This min_frequency does for you. Encode a thing about the category rather than its identity. For a postcode, the region. For a product, its price band. Or target encoding, replacing each category with the mean target for that category. This Is compact and powerfully prone to leakage. Found dataset it tells the model the answer. It must be found inside the folds, which is what TargetEncoder handles.
The question worth asking first is whether the column belongs in the model at all. A user id often carries no generalisable info. What you want is what the user *did*.
LabelEncoder is not for this
LabelEncoder appears in a great deal of example code applied to feature columns. It's documented for encoding the target.
It accepts only 1-D input. So Using it on features means looping over columns and applying it one at a time. That works, makes an ordinal encoding with all the problems above, and. The real objection. It has no handle_unknown, so a new category at guess time raises with no option to do or else.
Use OrdinalEncoder for sorted features, OneHotEncoder for unordered ones. LabelEncoder only when converting string class labels into ints for y. This Most estimators don't even need.
Categories that arrive as numbers
The hardest cat columns to spot are the ones already stored as ints. Nothing raises and nothing looks wrong.
A postcode district, a product code, a department id, a day-of-week stored as 0 to 6, a survey answer coded 1 to 5. Each is a category wearing a number. A model handed it raw will treat department 7 as greater than department 3 and just twice department 3.5.
Nothing in the data announces this. The column is numeric, the estimator accepts it, the fit succeeds. The model quietly uses an sorting that means nothing. It's the mirror image of the ordinal mistake. There, an order was invented where none existed. Here, an order that happens to exist in the encoding is mistaken for one that exists in the world.
The check is to ask what the math would mean. If the mean of two values is meaningless. The mean of department 2 and department 8. The column is cat even so of its dtype. If it's real, as with an age or a price, it's truly numeric.
Day-of-week is the notable middle case. It has an order, the gaps are equal. It wraps around: Sunday is adjacent to Monday. This No int encoding captures. The usual fix is a pair of features, the sine and cosine of the angle around the week. This Makes the wrap-around clear and is the standard trick for any cyclical quantity. Hours, months, compass bearings.
Reading the model afterwards
One-hot encoding times the columns, and a fitted model reports one number per column. Keeping the two aligned is what makes the result readable.
get_feature_names_out() on the encoder returns the made names in order. On a Pipeline or ColumnTransformer it returns the names for all the whole chain made. Zipping that against coef_ or feature_importances_ gives a labelled result rather than an array of numbers whose spots you've to rebuild.
Two things to note when reading them. The importance of a category is spread across its columns. A feature with twenty categories has its sway cut twenty ways and will look weaker than a single numeric column carrying the same info. Which is a known bias in tree-based feature importances, not a fact about the data. And with drop="first", each left coeff is scaled to the dropped category. So A plus coeff means "higher than the baseline", not "high".
Do trees need one-hot encoding? scikit-learn's do. They can't take strings. HistGradientBoostingClassifier accepts declared cat features natively. This Is both faster and better than one-hot for high-cardinality columns.
Should I one-hot encode before or after splitting? After. Inside a pipeline - or else the encoder learns the category list from the test rows too.
What about missing values in a cat column?OneHotEncoder treats NaN as its own category by default. This Is often the right action: "not stated" is often helpful rather than merely absent.
Things to try with Encoding Categories
Read the alphabetical disaster. The third editor codes small as 2 and large as 0. Look at the numbers before reading the fix.
Meet the unknown category. In the second editor, drop handle_unknown="skip" from the safe encoder and watch it raise on tokyo.
Explode a column. In the fifth editor, raise the number of distinct users and watch the column count follow it just.
Group the rare ones. Add min_frequency=50 to that encoder and see how many columns live.
Questions people ask about Encoding Categories
OneHotEncoder or OrdinalEncoder? Use one-hot for unordered categories like colour. Use ordinal only when the categories have a real order, such as small, medium and large.
What does handle_unknown='ignore' do? It encodes a category not seen in training as all zeros instead of raising an error at prediction time.
Why not use LabelEncoder on features? It is meant for the target. On features it invents an order that models will treat as meaningful.
What if a column has thousands of categories? One-hot gets wide. Consider grouping rare values with min_frequency, or target encoding done inside cross-validation.
Should I drop one column from one-hot output? Only when the model suffers from collinearity, such as unregularised linear regression. Trees and regularised models do not need it.
Do tree models need encoding? Most still need numeric input, though HistGradientBoostingClassifier can take categorical columns natively.
Where this leaves you
One-hot for unordered categories with handle_unknown="skip", ordinal with an clear categories= when the order is real, drop="if_binary" as a reasonable default. A plan for any column with more than a few dozen distinct values before it turns a few dozen columns of nearly nothing.
Check yourself
0 of 4
Answer without scrolling back up.
Why is numbering cities 0, 1, 2 a problem?
The numbers claim berlin > paris > london and that the gaps are equal. A linear model fits one coefficient to that invented scale.
What does handle_unknown="ignore" do?
All zeros honestly represents "none of the categories I know". Without it, cross-validation crashes on any fold that isolates a rare category.
When is OrdinalEncoder the right choice?
Left to itself it sorts alphabetically, which turned small into 2 and large into 0 in the editor above.
What is LabelEncoder for?
It takes only 1-D input and has no handle_unknown, so applying it to features gives an ordinal encoding that raises on any new category.
Cheat sheet
Encoding Categories
Turn categorical columns into numbers with OrdinalEncoder and OneHotEncoder, and learn which choice invents a fact and which does not.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.