Drag two vectors around the plane and watch their dot product change. This one number — the most-used operation in machine learning — tells you how much two directions agree.
Controls
a₁
a₂
b₁
b₂
Presets
Tip: you can also drag the arrowheads directly on the plot.
Vector Space
drag the arrowheads
Live Calculation
|a|
0
|b|
0
angle θ
0°
a · b
0
Vectors and the Dot Product
One multiplication-and-add that quietly powers every neural network on this site.
The problem it solves
A vector is a list of numbers that also has a geometric meaning: a direction and a length. The dot product combines two vectors into a single number that measures how much they point the same way.
Two Definitions, One Answer
The dot product can be computed two completely different ways, and they always agree:
The first is what a computer does — multiply matching components, add them up. The second explains what the answer means. Setting them equal is how we recover the angle between two vectors, which is exactly the cosine similarity used throughout NLP.
The Sign Is the Whole Story
Because cos θ is positive below 90°, zero at exactly 90°, and negative above it, the dot product acts as an agreement meter:
Positive — the vectors broadly point the same way.
Zero — they are perpendicular, or orthogonal. They share nothing.
Negative — they point in opposing directions.
Load the Perpendicular preset and watch the dot product sit at exactly 0. Then drag one arrowhead slowly past the right angle and watch the sign flip.
Projection: The Shadow Interpretation
The dashed grey line on the plot is the projection of b onto a — the shadow b casts if light shines perpendicular to a. Its signed length is:
So the dot product is really asking: how far along a does b reach? That framing is why it appears everywhere — a neuron computing w · x is measuring how strongly an input aligns with a learned pattern.
Where You Have Already Met It
Every neuron computes w · x + b before its activation function.
Attention in transformers scores tokens with q · k, scaled by √d.
Cosine similarity is the dot product of two unit-length vectors — the angle alone, with magnitude divided out.
Matrix multiplication is nothing but a grid of dot products, which is the next module.
A vector is a list of numbers with a direction
In machine learning a vector is simply an ordered list of numbers — and each number is a measurement of something.
house = [120, 3, 2, 1995] (sq m, bedrooms, bathrooms, year)
Geometrically, that list is an arrow from the origin to a point in four-dimensional space. Nobody can picture four dimensions, and it does not matter: every operation is defined arithmetically and behaves the same way in 2 dimensions as in 768.
Two operations are all you need to start:
import numpy as np
a = np.array([3, 4])
b = np.array([1, 2])
a + b # [4, 6] - add corresponding entries
3 * a # [9, 12] - scale every entry
np.linalg.norm(a) # 5.0 - the length, sqrt(3^2 + 4^2)
The length (or norm) is Pythagoras extended to any number of dimensions: square every entry, add them, take the square root.
The dot product, and the two ways to read it
a · b = a₁b₁ + a₂b₂ + … + aₙbₙ
For [3, 4] and [1, 2]: 3×1 + 4×2 = 3 + 8 = 11. Multiply matching entries, add them all up, get a single number.
The second reading is geometric, and it is where the meaning lives:
a · b = |a| |b| cosθ
The dot product is the product of the two lengths times the cosine of the angle between them. That gives an immediate interpretation of its sign:
Dot product
Angle
Meaning
Large positive
Near 0°
Pointing the same way
Zero
Exactly 90°
Perpendicular — no shared direction
Large negative
Near 180°
Pointing opposite ways
The zero case is the most useful fact in the whole topic: a dot product of zero means the vectors are at right angles. That single test underpins orthogonality, projections, PCA components and the independence of basis directions.
Where it appears in machine learning
A neuron.z = w · x + b is a dot product between the weights and the inputs. Everything a network computes is built from this.
Similarity. Cosine similarity is the dot product with the lengths divided out, which is how search, recommendation and retrieval systems compare embeddings.
Projection. How much of one vector lies along another is a dot product.
Matrix multiplication. Every entry of a matrix product is a dot product between a row and a column.
The scale of it is worth noticing: a single forward pass of a large language model is billions of dot products, and nothing more exotic.
Normalising, and why it matters
Dividing a vector by its length gives a unit vector — same direction, length exactly 1.
unit = a / np.linalg.norm(a) # [0.6, 0.8], length 1
This matters because raw dot products mix two things: how aligned the vectors are, and how long they are. A long vector scores highly against everything. Normalise first and the dot product measures only alignment — which is exactly cosine similarity.
That is why embedding libraries normalise vectors on the way in and then use plain dot products: on unit vectors the two are the same number, and a dot product is faster.
Run the dot product
The sign is the whole story: positive means roughly the same direction, zero means perpendicular, negative means opposing.
example_01.pyNumPy
import numpy as np
a = np.array([3.0, 4.0])
b = np.array([4.0, 3.0])
dot = a @ b
print("a . b =", dot, " (3*4 + 4*3)")
print("|a| =", np.linalg.norm(a), " |b| =", np.linalg.norm(b))
cos = dot / (np.linalg.norm(a) * np.linalg.norm(b))
print("cos(angle) = %.4f -> %.1f degrees" % (cos, np.degrees(np.arccos(cos))))
print()
for label, v in [("same direction", np.array([6.0, 8.0])),
("perpendicular ", np.array([-4.0, 3.0])),
("opposite ", np.array([-3.0, -4.0]))]:
print("%s a . v = %7.2f" % (label, a @ v))
print()
print("zero means perpendicular; the sign says which side.")
Output
Experiments to try
Drag b in a circle around a at constant length. The dot product traces a cosine wave: maximum when aligned, zero at right angles, most negative when opposed.
Load Parallel. The angle reads 0° and the projection equals the full length of b — total agreement.
Double every component of b. The angle is unchanged but the dot product doubles — proof it measures magnitude and direction. Cosine similarity removes that magnitude effect.
Set a to (0, 0). The dot product collapses to 0 and the angle becomes undefined — a zero vector has no direction to agree with.
Worth remembering
Multiply matching components and add. The result is positive when two vectors agree, zero when they are unrelated, and negative when they oppose — a single number capturing alignment, and the atom from which matrix multiplication, attention and similarity search are all built.
Vectors in two, three and seven hundred dimensions
Everything above is defined by arithmetic, not by pictures, which is why it survives the jump to high dimensions unchanged. A sentence embedding with 768 entries has a length, an angle to every other embedding, and a dot product with each of them, computed by exactly the formulas above.
What does change is intuition, and two effects are worth knowing.
Distances concentrate. In very high dimensions, the distance between the nearest pair of random points and the furthest pair becomes proportionally similar. "Nearest neighbour" gets less meaningful, which is why KNN degrades and why dimensionality reduction before distance-based methods is standard practice.
Random vectors are nearly orthogonal. Pick two random directions in 1000 dimensions and their dot product is almost certainly close to zero. There is simply an enormous amount of room, which is part of why high-dimensional embeddings can store so many distinguishable concepts.
Other products, and what they are for
Operation
Input
Output
Use
Dot product
Two vectors
One number
Similarity, projection, neurons
Elementwise product
Two vectors
A vector
Masking, gating in LSTMs
Cross product
Two 3D vectors
A vector at right angles to both
Geometry, graphics, physics
Outer product
Two vectors
A matrix
Building rank-1 updates, attention maths
The cross product only exists in three dimensions (and seven, for exotic reasons), which is why it appears in graphics and almost never in machine learning.
Questions people ask
Is a vector the same as an array? In code, effectively yes — a 1-D NumPy array. Mathematically a vector also carries the idea of direction and magnitude.
Row vector or column vector? Maths convention is columns; NumPy's 1-D arrays are neither, and broadcast sensibly in both roles. It matters as soon as you write matrix products by hand.
What does a negative dot product mean? The vectors point in broadly opposite directions — more than 90° apart.
Why normalise before comparing? Otherwise long vectors score highly against everything, and you are measuring magnitude rather than similarity.
Is the dot product the same as cosine similarity? Only for unit vectors. In general, cosine similarity is the dot product divided by both lengths.
How is this related to matrix multiplication? Every entry in a matrix product is a dot product between a row of the first matrix and a column of the second.
Recap in one screen
A vector is an ordered list of numbers, and geometrically an arrow with length and direction.
The dot product multiplies matching entries and sums them — one number out.
Geometrically it is |a||b|cosθ: positive means aligned, zero means perpendicular, negative means opposed.
Normalising to length 1 turns the dot product into pure similarity, which is cosine similarity.
A neuron, a similarity search and every entry of a matrix product are all dot products.
Check yourself
0 of 3
Answer without scrolling back up.
Two vectors point in exactly opposite directions. Their dot product is:
The dot product carries the cosine of the angle between them. At 180 degrees the cosine is -1, so the product is negative. It is zero only when they are perpendicular.
What does a dot product of zero tell you?
Perpendicular vectors give zero, and so does any vector dotted with the zero vector. Both cases are worth remembering - the second is a common source of silent bugs.
Where does the dot product show up in a neural network?
w·x is literally a dot product. A layer of neurons is a stack of them, which is why a layer is implemented as a matrix multiply.
Cheat sheet
Vectors and the Dot Product
A vector is a list of numbers that also has a geometric meaning: a direction and a length. The dot product combines two vectors into a single number that measures how much they point the same way.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.