vizlearn
Modules / NLP Encoding / Label Encoding

Label Encoding

Convert categorical text into unique numerical labels. Each unique word is assigned a distinct integer ID.

Encoded Sequence

0 TOKENS
Sequence List Scroll View

Decoding Label Encoding

Machine learning models are mathematical functions; they understand numbers, not text. Before we can train a model on categorical data (like city names, product types, or sentiments), we must convert that text into a numerical format. Label Encoding is one of the simplest techniques to achieve this. It works by assigning a unique integer to each unique category or "label" in your dataset.

How It Works: Building the Vocabulary

The process is straightforward and consists of two main steps, which you can see in action by clicking the "Encode Corpus" button above.

1. Create a Vocabulary

First, the encoder scans the entire input text (the "corpus") to find all unique words (or "tokens"). It then sorts these unique words alphabetically to create a consistent vocabulary. This vocabulary acts as a dictionary or a look-up table.

2. Assign Integer IDs

Once the vocabulary is built, the encoder assigns a unique integer ID to each word, starting from 0. The first word in the sorted vocabulary gets ID 0, the second gets ID 1, and so on. The final output is a sequence of these integer IDs, representing the original text.

Guided Experiments

Use the interactive panel to understand the nuances of this technique.

  1. Observe the Default Text: Click the "Encode Corpus" button with the default text. The vocabulary is created from the unique words: `['deep', 'fascinating', 'is', 'learning', 'machine']`. Notice that "machine" and "learning" appear multiple times in the input, but only once in the vocabulary. The encoded sequence then maps each word to its corresponding index in this vocabulary (e.g., 'machine' becomes 4, 'learning' becomes 3).
  2. Add a New Word: Add the word "new" to the end of the text area and click "Encode Corpus" again. The vocabulary will be rebuilt and sorted alphabetically: `['deep', 'fascinating', 'is', 'learning', 'machine', 'new']`. Observe that 'new' is assigned the ID 5. All the original words keep their IDs because 'new' comes last alphabetically.
  3. Add a Word at the Beginning (Alphabetically): Now, add the word "a" to the text. Click "Encode Corpus". The new vocabulary is `['a', 'deep', 'fascinating', 'is', 'learning', 'machine', 'new']`. Notice what happened: 'a' is now assigned ID 0, and the ID for every other word has shifted up by one ('deep' is now 1, 'fascinating' is 2, etc.). This highlights a key property: the numerical assignments depend entirely on the complete, sorted vocabulary.

The Big Drawback: Unintended Ordinality

While simple, Label Encoding has a major flaw. Most machine learning models will interpret the assigned integers as having an order or rank. For example, if you have categories 'Cat', 'Dog', and 'Fish', they might be encoded as 0, 1, and 2.

The model might incorrectly assume that 'Dog' (1) is "greater" than 'Cat' (0), or that the difference between 'Fish' (2) and 'Cat' (0) is twice the difference between 'Dog' (1) and 'Cat' (0). This is an arbitrary ordinal relationship that doesn't exist in the original data, and it can mislead your model.

  • When is it okay? Label Encoding is suitable for categorical features that have a clear, inherent order (ordinal features), such as 'Low', 'Medium', 'High' or 'First Class', 'Second Class', 'Third Class'.
  • When should it be avoided? It should generally be avoided for features without a natural order (nominal features), like 'Country' or 'Color'. For these cases, One-Hot Encoding is a much better choice as it creates a new binary feature for each category, avoiding any implied ranking.
Cheat sheet

Label Encoding Process

Machine learning models are mathematical functions; they understand numbers, not text. Before we can train a model on categorical data (like city names, product types, or sentiments), we must convert that text into a numerical format. Label Encoding is one of the simplest techniques to achieve this. It works by assigning a unique integer to each unique category or "label" in your dataset.

MACHINE LEARNING · vizlearn.in/machine_learning/label_encoding.html

More in Machine Learning

Machine LearningLinear Regression with OLSMachine LearningNaive Bayes ClassifierMachine LearningOne-Hot EncodingMachine LearningSliding Window for Time Series