Distance Metrics

The four you will actually meet

Euclidean   d = √( Σ (pᵢ − qᵢ)² )

Straight-line distance. The one everybody means by default, and the L2 norm of the difference.

Manhattan   d = Σ |pᵢ − qᵢ|

Sum of the axis-aligned steps — the distance a taxi drives on a street grid. The L1 norm of the difference.

Chebyshev   d = max |pᵢ − qᵢ|

The single largest coordinate gap, ignoring all the others. The L∞ norm, and how a king moves on a chessboard.

Cosine   d = 1 − (p · q) / (‖p‖ ‖q‖)

The odd one out. It measures the angle between the two vectors as seen from the origin, and takes no account of how long either is.

Several ways to measure "far apart"

Distance is the foundation of clustering, nearest-neighbour methods, anomaly detection and most retrieval systems — and there is more than one sensible definition.

For the points (0, 0) and (3, 4):

• Euclidean: √(9 + 16) = 5
• Manhattan: 3 + 4 = 7
• Chebyshev: max(3, 4) = 4

All three are legitimate answers to "how far apart are these", and which is correct depends entirely on what movement is possible in your problem.

Choosing between them

Euclidean is the default and the right choice for continuous, well-scaled features in a modest number of dimensions. It is what k-means minimises and what most distance-based models assume.

Manhattan is more robust to outliers, because it does not square the differences — one wildly different feature does not dominate the total. It also holds up somewhat better in high dimensions, and is the natural choice when features are genuinely independent axes that cannot be traded off against each other.

Cosine ignores magnitude entirely and compares direction. It is the standard for text, embeddings and any sparse high-dimensional representation, because a long document should not be far from a short document about the same subject.

Hamming and Jaccard apply where the data is categorical or set-valued — comparing binary fingerprints, tag sets, or which products two customers bought.

The Minkowski distance unifies the first three with a parameter p: p = 1 is Manhattan, p = 2 is Euclidean, and p → ∞ is Chebyshev. Scikit-learn exposes exactly this as the p argument.

Scaling is not optional

Every distance except cosine adds up per-feature differences, so a feature with a large numeric range dominates the total.

Salary (20,000–200,000) against age (18–80): two people differing by 35 years and £500 have a squared distance of 1,225 + 250,000. The age difference contributes 0.5% of the total. Age has effectively been deleted from the model.

Standardise, or normalise to a range, before computing any distance. This is the most common reason KNN, k-means and DBSCAN "do not work", and it is a one-line fix.

The related trap is correlated features. Three near-duplicate columns count that information three times in the distance. Mahalanobis distance handles this properly by using the inverse covariance matrix — measuring distance in units of the data's own spread — which is why it is the standard choice for multivariate outlier detection.

Exploration guide

1. Read the classic triangle. Leave P at (1, 1) and Q at (4, 5). Euclidean reads exactly 5 — the 3-4-5 triangle — while Manhattan reads 7 and Chebyshev reads 4. Three answers, same two points.
2. Find where they agree. Set Point Q y to 1 so the points share a row. Now Euclidean, Manhattan and Chebyshev all read the same number, because only one coordinate differs and there is nothing left to disagree about.
3. Watch the rings. With Show Equal-Distance Rings on, notice their shapes: a circle for Euclidean, a diamond for Manhattan, a square for Chebyshev. Every point on a ring is exactly the same distance from P according to that metric, which is what "same distance" really means.
4. Make cosine ignore you. Drag the Stretch Q Outward slider from 0.2 to 3. All three geometric distances change enormously; cosine distance does not move at all, because Q is sliding along a fixed direction and the angle never changes.
5. Then change the angle. Now drag Point Q y. Cosine finally responds — it is the only thing it responds to.
6. Check the ordering. Drag the points anywhere. Chebyshev is never larger than Euclidean, and Euclidean is never larger than Manhattan. That ordering holds for every pair of points, in every dimension.

When cosine is the right answer

Cosine dominates text and embedding work, and the reason is specific. In a bag-of-words representation, a long document has larger counts everywhere than a short one on the same subject. Euclidean distance reads that as "far apart"; cosine reads the direction of the vector — the mix of words rather than the volume — and correctly calls them similar.

The same logic applies to embeddings, where magnitude often encodes something like frequency or confidence rather than meaning. That is why vector databases index by cosine similarity, and why embedding comparisons are almost always angular.

One useful fact: on vectors normalised to unit length, ranking by cosine and ranking by Euclidean distance give exactly the same order. Many systems normalise once up front and then use whichever is faster.

The curse of dimensionality

Distances behave strangely as dimensions increase, and it affects every distance-based method.

In high-dimensional space, the distance to the nearest point and the distance to the furthest point become proportionally similar. With 1,000 features, "nearest neighbour" can be barely nearer than average, and any method built on that ranking degrades towards guessing.

The intuition: adding a dimension adds another positive term to every distance, so all distances grow — and they grow at similar rates, compressing the relative differences.

Three practical responses:

1. Reduce first. PCA to 20–50 components before KNN or k-means, which usually improves both speed and accuracy.
2. Select features. Every irrelevant column adds pure noise to every distance.
3. Switch metric. Cosine and Manhattan hold up better than Euclidean in very high dimensions.

This is also why embeddings work despite having hundreds of dimensions: they are trained so that meaningful directions carry the variation, rather than being raw, mostly-irrelevant measurements.

Failure modes

• Not scaling the features. The single most common error. A feature measured in thousands contributes thousands of units to every distance, so your metric silently becomes "difference in that one feature". Standardise before measuring.
• Using Euclidean on text counts. It confuses document length with topic. Use cosine.
• Treating cosine as a proper metric. Cosine distance does not satisfy the triangle inequality, so algorithms that rely on that property for correctness or for pruning cannot use it directly.
• Mixing categorical and numeric features in one Euclidean distance. The gap between two one-hot categories is not comparable to a gap in a continuous variable. Use a mixed metric such as Gower.

Four metrics, three different winners

The same query point against the same three candidates. Which one counts as "nearest" is decided by the metric, not by the data.

import numpy as np

query = np.array([1.0, 1.0, 1.0, 1.0])
cands = {
    "A  spread over all dims": np.array([1.5, 1.4, 1.4, 1.3]),
    "B  one big difference":   np.array([1.0, 1.0, 1.0, 2.4]),
    "C  same direction, 4x":   np.array([4.0, 4.0, 4.0, 4.0]),
}

def cosine(u, v):
    return 1 - u.dot(v) / (np.linalg.norm(u) * np.linalg.norm(v))

metrics = {
    "euclidean": lambda c: np.linalg.norm(c - query),
    "manhattan": lambda c: np.abs(c - query).sum(),
    "chebyshev": lambda c: np.abs(c - query).max(),
    "cosine":    lambda c: cosine(query, c),
}

print("%-24s %10s %10s %10s %10s" % ("candidate", *metrics))
for name, c in cands.items():
    print("%-24s %10.4f %10.4f %10.4f %10.4f"
          % (name, *[f(c) for f in metrics.values()]))
print()
print("nearest neighbour, according to each:")
for mname, f in metrics.items():
    winner = min(cands, key=lambda k: f(cands[k]))
    print("  %-10s -> %s" % (mname, winner))
print()
print("manhattan forgives one big error and punishes many small ones;")
print("euclidean does the reverse; cosine ignores length entirely, so it")
print("calls a point four times further away a perfect match.")
print()
print("in high dimensions everything drifts equally far away:")
rng = np.random.default_rng(0)
for dim in (2, 10, 100, 1000):
    pts = rng.random((500, dim))
    d = np.linalg.norm(pts - pts[0], axis=1)[1:]
    print("  %4dd: nearest %.3f, farthest %.3f, ratio %.3f"
          % (dim, d.min(), d.max(), d.max() / d.min()))

Worth remembering

Euclidean, Manhattan and Chebyshev are the L2, L1 and L∞ norms of the difference between two points, always ordered Chebyshev ≤ Euclidean ≤ Manhattan and agreeing only when a single coordinate differs; their equal-distance rings are a circle, a diamond and a square, which is what makes them disagree. Cosine is a different kind of measure entirely — it compares direction from the origin and ignores magnitude, which is why stretching a vector leaves it unchanged and why it is the standard for text and embeddings. Whichever you choose, scale the features first, because an unscaled metric measures units rather than similarity.

What makes something a metric

A proper distance metric satisfies four conditions: it is never negative, it is zero only between identical points, it is symmetric, and it obeys the triangle inequality — going via a third point is never shorter.

That last condition is not pedantry: it is what allows spatial indexes such as KD-trees and ball trees to prune the search space and answer nearest-neighbour queries quickly.

Cosine similarity is not a metric (it is a similarity, not a distance), and cosine distance (1 − similarity) does not satisfy the triangle inequality either — which is why libraries often normalise vectors and use Euclidean distance instead, since on unit vectors the two produce the same ranking.

Questions people ask

Which metric should I use by default? Euclidean on scaled continuous features; cosine for text and embeddings; Hamming or Jaccard for categorical and set data.

Does the metric change the clusters? Substantially. Running k-means with cosine distance on the same data can produce entirely different groups — and note that standard k-means assumes Euclidean, so switching metric strictly means switching algorithm.

Can I mix numeric and categorical features? Gower distance handles mixed types by combining per-feature measures. One-hot encoding plus Euclidean is the common approximation and treats every category as equally distant.

Why is cosine used for embeddings? Because the length of an embedding often reflects frequency or confidence rather than meaning, and the direction is what carries the semantics.

Is squared Euclidean distance ever used? Often — it avoids the square root, is cheaper, and gives the same ranking. k-means minimises exactly this.

How do I speed up nearest-neighbour search? KD-trees below about 20 dimensions, ball trees a little beyond, and approximate methods (HNSW, IVF) for anything large or high-dimensional.

Recap in one screen

• Distance has several definitions; Euclidean, Manhattan, cosine, Hamming and Jaccard each fit different data.
• Scale your features first — unscaled columns hijack every distance calculation.
• Cosine ignores magnitude and compares direction, which is what text and embeddings need.
• Correlated features are double-counted; Mahalanobis distance corrects for that.
• In very high dimensions all distances converge — reduce dimensionality before relying on them.