Encoder-only vs Decoder-only
BERT and GPT are the same transformer block with one difference: whether a token is allowed to look to its right. Everything else — the training objective, what the model is good for, whether it can write at all — follows from that one triangle of zeros.
The Architecture
which token is doing the looking
BERT's training objective, in one checkbox
Can It Write?
What Position 5 Can See
—Blue is the token doing the looking. Green is what it is allowed to attend to.
The Attention Mask
The Objective That Allows
Counting It
What Each Is For
Encoder-only — classification, retrieval embeddings, named entities, reranking. Anything where you have the whole text and want to understand it.
Decoder-only — chat, completion, summarising, code. Anything where the output is text you do not have yet.
Encoder-decoder — translation and other strict input-to-output mappings: read bidirectionally, write causally.
BERT vs GPT: A Practical Guide
One triangle of zeros, and everything that follows from it.
Quick Context
Both families are stacks of the same block: multi-head attention and a feed-forward layer, repeated. The difference is a mask applied inside self-attention — a matrix of minus infinities that deletes some of the attention scores before the softmax.
Delete nothing and every token sees the whole sentence: that is an encoder, and BERT is the famous one. Delete everything above the diagonal and each token sees only itself and what came before: that is a decoder, and GPT is the famous one.
Why the mask decides the objective
A model that can see the future cannot be trained to predict it. If position 4 can attend to position 5, then "predict the token at position 5" is answered by copying it, and nothing is learned. So the two masks admit different training tasks.
- Bidirectional → masked language modelling. Hide 15% of the tokens and predict them from both sides. Nothing is being copied because the answer has been removed from the input entirely.
- Causal → next-token prediction. Every position predicts the token after it, and the mask guarantees it cannot cheat. One pass over a sentence supplies a training signal at every position at once, which is what makes it so efficient at scale.
The consequence is the interesting part. Masked language modelling produces excellent representations and no way to generate: there is no "next" for a model where every position already saw everything. Next-token prediction produces a generator whose representations are weaker per parameter, because each token was built without the right-hand context.
Interactive Exploration Guide
- Look around from the middle. With bidirectional attention selected and position 5, the whole sentence lights up: 8 tokens visible, 2 of them to the right.
- Switch to causal. The same position now sees 6 tokens and none to the right, and the mask grid loses its whole upper triangle — 36 visible pairs out of 64 instead of all 64.
- Walk the query along. Drag Look From Position from 0 to 7 with causal attention on. Position 0 sees exactly one token — itself. That is why the first token of a prompt is the hardest thing a decoder ever has to represent.
- Try the masked objective. Switch back to bidirectional, put the query on position 5 and tick Mask That Token Out. The word is replaced by [MASK] and the model has "on the ___ mat" to work with — both sides. That is BERT's whole training task.
- Now mask under causal attention. Same checkbox, causal mode: the model has "The cat sat on the ___" and no idea that "mat" follows. Same sentence, strictly less information, which is exactly the trade.
- Try to generate. Press Generate Next in causal mode and the sentence extends one token at a time, each new token attending only to what precedes it. Switch to bidirectional and the button refuses: an encoder has no notion of next.
Which one to reach for
- Classifying, tagging, reranking, embedding for search? Encoder. You already have the whole text; a model that reads it bidirectionally will represent it better and is far cheaper to run — a BERT-sized encoder is hundreds of times smaller than a frontier decoder.
- Producing text? Decoder. Nothing else can.
- Strict input-to-output rewriting, like translation? Encoder-decoder, which reads with full attention and writes causally, plus cross-attention between the two. T5 and the original transformer are this shape.
The industry has drifted decisively towards decoder-only for general systems, because one model that can be prompted to do anything beats a family of specialists — but for a high-volume classifier or a retrieval encoder, the small bidirectional model is still usually the right engineering answer.
Key Takeaway
Encoder-only and decoder-only transformers differ by one thing: whether the attention mask deletes the upper triangle. Bidirectional attention lets every token see the whole sequence, which rules out next-token prediction and demands a masked objective instead, and yields strong representations with no ability to generate. Causal attention lets each token see only what precedes it, which makes next-token prediction both possible and enormously efficient, and yields a generator. Pick by the task rather than by fashion: text out means a decoder, and understanding text you already have is still cheaper and often better with an encoder.