Distance Metrics

Quick Context

A surprising number of algorithms do nothing but compare distances. KNN finds the nearest points, k-means assigns each point to the nearest centre, hierarchical clustering merges the closest pair, and a vector database retrieves the closest embeddings.

All of them take "distance" as an input, not an output. Change the metric and the same data yields different neighbours, different clusters and different results — which makes it a modelling choice worth understanding rather than a default worth accepting.

The four you will actually meet

Euclidean   d = √( Σ (pᵢ − qᵢ)² )

Straight-line distance. The one everybody means by default, and the L2 norm of the difference.

Manhattan   d = Σ |pᵢ − qᵢ|

Sum of the axis-aligned steps — the distance a taxi drives on a street grid. The L1 norm of the difference.

Chebyshev   d = max |pᵢ − qᵢ|

The single largest coordinate gap, ignoring all the others. The L∞ norm, and how a king moves on a chessboard.

Cosine   d = 1 − (p · q) / (‖p‖ ‖q‖)

The odd one out. It measures the angle between the two vectors as seen from the origin, and takes no account of how long either is.

Interactive Exploration Guide

1. Read the classic triangle. Leave P at (1, 1) and Q at (4, 5). Euclidean reads exactly 5 — the 3-4-5 triangle — while Manhattan reads 7 and Chebyshev reads 4. Three answers, same two points.
2. Find where they agree. Set Point Q y to 1 so the points share a row. Now Euclidean, Manhattan and Chebyshev all read the same number, because only one coordinate differs and there is nothing left to disagree about.Run this
3. Watch the rings. With Show Equal-Distance Rings on, notice their shapes: a circle for Euclidean, a diamond for Manhattan, a square for Chebyshev. Every point on a ring is exactly the same distance from P according to that metric, which is what "same distance" really means.
4. Make cosine ignore you. Drag the Stretch Q Outward slider from 0.2 to 3. All three geometric distances change enormously; cosine distance does not move at all, because Q is sliding along a fixed direction and the angle never changes.
5. Then change the angle. Now drag Point Q y. Cosine finally responds — it is the only thing it responds to.
6. Check the ordering. Drag the points anywhere. Chebyshev is never larger than Euclidean, and Euclidean is never larger than Manhattan. That ordering holds for every pair of points, in every dimension.

When cosine is the right answer

Cosine dominates text and embedding work, and the reason is specific. In a bag-of-words representation, a long document has larger counts everywhere than a short one on the same subject. Euclidean distance reads that as "far apart"; cosine reads the direction of the vector — the mix of words rather than the volume — and correctly calls them similar.

The same logic applies to embeddings, where magnitude often encodes something like frequency or confidence rather than meaning. That is why vector databases index by cosine similarity, and why embedding comparisons are almost always angular.

One useful fact: on vectors normalised to unit length, ranking by cosine and ranking by Euclidean distance give exactly the same order. Many systems normalise once up front and then use whichever is faster.

The curse of dimensionality

In high dimensions, distance stops being informative. As dimensions are added, the gap between the nearest and furthest points in a dataset shrinks relative to their absolute distance, so everything becomes roughly equidistant from everything else and "nearest neighbour" loses meaning.

Manhattan degrades more slowly than Euclidean here, which is why it is sometimes preferred on high-dimensional data. The more common fixes are to reduce the dimension first, or to use a metric learned for the task rather than an off-the-shelf one.

What usually goes wrong

• Not scaling the features. The single most common error. A feature measured in thousands contributes thousands of units to every distance, so your metric silently becomes "difference in that one feature". Standardise before measuring.
• Using Euclidean on text counts. It confuses document length with topic. Use cosine.
• Treating cosine as a proper metric. Cosine distance does not satisfy the triangle inequality, so algorithms that rely on that property for correctness or for pruning cannot use it directly.
• Mixing categorical and numeric features in one Euclidean distance. The gap between two one-hot categories is not comparable to a gap in a continuous variable. Use a mixed metric such as Gower.

Key Takeaway

Euclidean, Manhattan and Chebyshev are the L2, L1 and L∞ norms of the difference between two points, always ordered Chebyshev ≤ Euclidean ≤ Manhattan and agreeing only when a single coordinate differs; their equal-distance rings are a circle, a diamond and a square, which is what makes them disagree. Cosine is a different kind of measure entirely — it compares direction from the origin and ignores magnitude, which is why stretching a vector leaves it unchanged and why it is the standard for text and embeddings. Whichever you choose, scale the features first, because an unscaled metric measures units rather than similarity.