A small example
For a colour feature with values red, green and blue, create three columns: is_red, is_green and is_blue. A green row becomes [0, 1, 0]. This avoids the misleading numeric order created by assigning red=0, green=1 and blue=2.
The visualiser applies the same representation to words in a text corpus. Each unique token acts like a category: it receives one vocabulary position, and each token’s vector has a 1 at that position. This is a small example of one-hot representation; production text models often use counts, weighted counts or learned embeddings instead.
Explore the vocabulary
- Click Encode Corpus and inspect Vocabulary and Vec Length. Each token maps to a vector whose length is the number of unique vocabulary entries.
- Add a word that is not in the current text and encode again. The vocabulary and vector length grow; repeating an existing word adds another token but does not add a new vocabulary column.
The trade-offs
One-hot features are easy to interpret and work well for low-cardinality categories. They can be sparse: each row has only a small number of ones in a wide matrix. Sparse storage helps, but a feature with thousands of distinct values can still increase memory and make estimation difficult, especially when many categories appear only a few times.
For a categorical feature with k possible values, a full encoding uses k columns. In a linear model with an intercept, one column is often dropped to avoid exact collinearity; the remaining coefficients are interpreted relative to that reference category. Whether to drop a category depends on the estimator and regularisation—do not apply it blindly to every model.
| Situation | Common choice |
|---|
| A few unordered categories | One-hot encoding |
| Categories with a real order | Ordinal encoding, with the order stated |
| Very many categories | Native categorical handling, feature hashing or carefully validated target encoding |
| Text with a large vocabulary | Sparse count or TF–IDF features; embeddings for suitable neural models |
Unseen categories at prediction time need an explicit policy. Scikit-learn’s OneHotEncoder documentation describes options such as handling unknown values and sparse output. Fit the encoder on training data only, then reuse it for validation and test data; a pipeline helps prevent leakage. See label encoding for the alternative that maps categories to integers.