Distance Metrics
Drag two points and get four different answers to "how far apart are they". Which one you pick changes what your model thinks is similar.
Two Points
same direction, different length
every point the same distance from P
Four Ways to Measure
—Green is the straight line, orange the grid route, blue the largest single step.
Distances
Angle vs Length
Cosine measures the angle from the origin and ignores length entirely, so stretching Q does not move it.
Distance Metrics
A distance rule defines which points count as close; scaling and geometry can change the answer.
Scale before comparing
If one feature is measured in thousands and another in fractions, Euclidean distance may be dominated by the first feature. Standardising features can help, but it changes the meaning of distance and should be based on the task and training data. See standard deviation and vectors and the dot product.
A metric must obey more than a distance formula
A metric is nonnegative, is zero only between identical points, is symmetric, and satisfies the triangle inequality. Norm distances ‖x−y‖ₚ meet these conditions for p≥1. For the points (0,0), (3,4), and (3,0), Euclidean distance from the first to the second is 5; going through the third costs 3+4=7, consistent with the triangle inequality. L1 distance counts coordinate movement, while L∞ is the largest coordinate gap. Their rankings can differ, so “nearest” depends on the geometry you choose.
Cosine similarity compares angle and ignores magnitude. The often-used cosine distance 1−cosine similarity is not a metric in general because it can violate the triangle inequality; angular distance is a different construction. This matters for algorithms that rely on metric properties. Standardising features also changes geometry: it prevents a large-unit feature from automatically dominating, but it should be estimated from training data and chosen for the task. A distance is a modelling decision, not a neutral property of the data.
Try this above
Move the points along one axis and compare the metric readouts. Then change one coordinate by a much larger amount to see which rule is most sensitive to that gap.
A nearest-neighbour model makes the metric choice visible: the selected neighbour can change when using L1 rather than L2 distance. If features have different scales, standardise them using statistics computed only on the training set, then apply the same transformation to validation and future examples. Otherwise information from evaluation data leaks into preprocessing. Standardisation is not always appropriate—for example, when physical units encode meaningful costs—so document the metric and scaling choices as part of the model.
Sources and next steps
- SciPy distance functions documents common vector distances and their conventions.
- Continue with vector norms to see how distances inherit the geometry of a norm.
Overview
Common ways to measure distance
For points x and y, L2 distance is straight-line distance, L1 sums coordinate differences, and L∞ takes the largest coordinate difference:
For x = (0, 0) and y = (3, 4), L2 gives 5, L1 gives 7, and L∞ gives 4. Each metric answers a different geometric question. L2 is natural for isotropic geometry; L1 is useful when coordinate-wise movement or sparse differences matter; L∞ measures the worst coordinate gap.
Cosine distance compares direction rather than magnitude, so it is useful for normalised text or image embeddings. It is not a metric in every common definition: 1 − cosine similarity can fail the triangle inequality. Check the assumptions of an algorithm that requires a true metric.