Quick Context
A vector is a list of numbers that also has a geometric meaning: a direction and a length. The dot product combines two vectors into a single number that measures how much they point the same way.
Two Definitions, One Answer
The dot product can be computed two completely different ways, and they always agree:
algebraic: a · b = a₁b₁ + a₂b₂
geometric: a · b = |a| |b| cos θ
The first is what a computer does — multiply matching components, add them up. The second explains what the answer means. Setting them equal is how we recover the angle between two vectors, which is exactly the cosine similarity used throughout NLP.
The Sign Is the Whole Story
Because cos θ is positive below 90°, zero at exactly 90°, and negative above it, the dot product acts as an agreement meter:
- Positive — the vectors broadly point the same way.
- Zero — they are perpendicular, or orthogonal. They share nothing.
- Negative — they point in opposing directions.
Load the Perpendicular preset and watch the dot product sit at exactly 0. Then drag one arrowhead slowly past the right angle and watch the sign flip.
Projection: The Shadow Interpretation
The dashed grey line on the plot is the projection of b onto a — the shadow b casts if light shines perpendicular to a. Its signed length is:
projₜ b = (a · b) / |a|
So the dot product is really asking: how far along a does b reach? That framing is why it appears everywhere — a neuron computing w · x is measuring how strongly an input aligns with a learned pattern.
Where You Have Already Met It
- Every neuron computes w · x + b before its activation function.
- Attention in transformers scores tokens with q · k, scaled by √d.
- Cosine similarity is the dot product of two unit-length vectors — the angle alone, with magnitude divided out.
- Matrix multiplication is nothing but a grid of dot products, which is the next module.
Interactive Exploration Guide
- Drag b in a circle around a at constant length. The dot product traces a cosine wave: maximum when aligned, zero at right angles, most negative when opposed.
- Load Parallel. The angle reads 0° and the projection equals the full length of b — total agreement.
- Double every component of b. The angle is unchanged but the dot product doubles — proof it measures magnitude and direction. Cosine similarity removes that magnitude effect.
- Set a to (0, 0). The dot product collapses to 0 and the angle becomes undefined — a zero vector has no direction to agree with.
Key Takeaway
Multiply matching components and add. The result is positive when two vectors agree, zero when they are unrelated, and negative when they oppose — a single number capturing alignment, and the atom from which matrix multiplication, attention and similarity search are all built.