N-Gram

Interactive visualization of how text is broken down into contiguous sequences of N items for language modeling.

Source Text

N-gram Size (N)

Value 2

Statistics

Total N-grams

Unique

Word Count

Current N

Generated N-grams

0 ITEMS

Enter text to generate sequences

Tip: Use the slider to scroll through the list if it overflows.

Understanding N-grams: The Building Blocks of Language Models

See how text is broken down into overlapping sequences to help machines understand context and predict the next word.

What are N-grams?

An n-gram is a contiguous sequence of 'n' items from a given sample of text or speech. The "items" can be characters, syllables, or, most commonly, words. N-grams are a simple yet powerful way for machines to capture the context and statistical properties of a language.

By analyzing which words tend to appear together, n-gram models can form a probabilistic understanding of a language. This is the foundation for many Natural Language Processing (NLP) tasks.

Types of N-grams

The 'n' in n-gram determines the size of the sequence. The most common types are:

Unigram (1-gram): A single word. This tells you about word frequency but nothing about context. (e.g., "The", "quick", "brown")
Bigram (2-gram): A sequence of two words. This starts to capture basic context. (e.g., "The quick", "quick brown", "brown fox")
Trigram (3-gram): A sequence of three words, capturing more specific context. (e.g., "The quick brown", "quick brown fox")

Interactive Exploration Guide

Use the N-gram Explainer above to see how this works in practice:

Enter Text: The default text is "The quick brown fox jumps over the lazy dog".
Select N-gram Size: The default is set to "Bi (2)". Look at the "Generated N-grams" panel. You'll see a list of all the two-word sequences: "The quick", "quick brown", "brown fox", and so on. This is a "sliding window" of size 2 moving across the text.
Change to Unigrams: Click the "Uni (1)" button or set the slider to 1. The panel now shows each individual word. This is a simple bag-of-words representation.
Change to Trigrams: Click the "Tri (3)" button or set the slider to 3. The panel now shows three-word sequences like "The quick brown". Notice how much more specific the context becomes.
Check the Statistics: As you change the value of 'n', observe the "Total N-grams" count. For a text with 'k' words, the number of n-grams is `k - n + 1`. This is why the total count decreases as 'n' gets larger.

Why are N-grams Important in NLP?

N-grams are fundamental to many classic NLP applications:

Language Modeling: N-grams are used to build statistical models that can predict the next word in a sentence. For example, after seeing the bigram "peanut butter", the model knows there's a high probability the next word is "and" or "sandwich". This is how smartphone keyboards suggest the next word.
Text Classification: The presence of certain n-grams can be a strong signal for classifying documents. For instance, a movie review containing bigrams like "highly recommend" or "waste of time" is a good indicator of its sentiment.
Machine Translation: N-gram models help ensure that translated sentences are fluent and grammatically correct by comparing the probability of different word sequences.

Key Takeaway

N-grams are a simple but effective method for capturing local context in text by breaking it into small, overlapping chunks. While modern deep learning models use more sophisticated techniques, n-grams remain a foundational concept for understanding how machines begin to process and find patterns in human language.

Cheat sheet

N-gram Explainer

An n-gram is a contiguous sequence of 'n' items from a given sample of text or speech. The "items" can be characters, syllables, or, most commonly, words. N-grams are a simple yet powerful way for machines to capture the context and statistical properties of a language.

NLP · vizlearn.in/natural_language_processing/n_gram.html