Vector Norms
Drag the vector and watch three different answers to "how big is it". The shape of each answer is what makes Lasso produce zeros and Ridge not.
Vector
p = 1 diamond · p = 2 circle · large p square
grow the ball until it first touches a line of equally good solutions
The Vector and Its Unit Balls
normsDrag anywhere to move the vector.
Three Answers
L∞ ≤ L2 ≤ L1 always. They only agree when the vector lies on an axis.
Sparsity
Turn on the sparsity demo to see why L1 lands on the axes and L2 does not.
Vector Norms
A norm measures vector size; the choice of norm changes what “small” means in geometry and optimisation.
Why the choice matters
L2 penalises large components smoothly and is common in weight decay. L1 can encourage exact zeros in optimisation, producing sparse coefficients. L∞ limits the largest single component. These are tendencies, not guarantees: the resulting solution also depends on the objective, constraints, and data. For regularisation details, see convexity and optimisation.
Every norm is nonnegative, is zero only for the zero vector, scales as ‖cx‖ = |c|‖x‖, and obeys the triangle inequality. Squared L2, ‖x‖₂², is useful in losses but is not itself a norm because it fails the triangle inequality.
A norm is a length rule with consistency properties
A norm must be nonnegative, equal zero only for the zero vector, scale by |c| when the vector is multiplied by c, and obey the triangle inequality ‖x+y‖≤‖x‖+‖y‖. The last rule says that going through an intermediate point cannot create a shorter route than the direct distance. These conditions distinguish a norm from an arbitrary formula that happens to return a number.
For x=(3,−4), L1=7 is the number of axis-aligned units, L2=5 is straight-line length, and L∞=4 is the largest coordinate change. In optimisation, adding an L1 penalty may make some fitted coefficients exactly zero, while L2 usually shrinks coefficients continuously; neither effect is guaranteed without considering the rest of the objective. Squared L2 is convenient because it is differentiable, but it fails the scaling rule: ‖2x‖₂²=4‖x‖₂² rather than 2‖x‖₂². That is why it is a useful penalty but not a norm.
Try this above
Compare the displayed unit shapes, then change the vector components. Notice that the same vector has a different length under L1, L2, and L∞.
Norms connect naturally to distances: the distance from x to y induced by a norm is ‖x−y‖. In two dimensions, the L2 unit circle contains points whose Euclidean length is one; the L1 unit diamond instead limits the sum of absolute coordinates. Thus the direction of a vector may be unchanged while its reported size differs. The choice is consequential in a regularised model: L1 and L2 penalties encode different preferences, so compare them using validation and the problem's interpretation rather than assuming one is universally superior.
The triangle inequality gives a useful bound in numerical analysis: ‖x+y‖≤‖x‖+‖y‖, so small perturbations cannot change a vector by more than the perturbation's norm. For a matrix, induced norms measure its maximum stretching under a chosen vector norm. Norms therefore appear in error bounds and convergence guarantees as well as geometry. Always identify which norm is used because a statement in L2 need not have the same numerical value in L1 or L∞.
Sources and next steps
- MIT OpenCourseWare: Linear Algebra provides the underlying vector geometry.
- Compare these lengths with distance metrics, which apply norms to differences between points.
Overview
Three common norms
For x = (x₁, …, xₙ), the L1 norm sums absolute components, the L2 norm is Euclidean length, and the L∞ norm is the largest absolute component:
For x = (3, −4), these are 7, 5, and 4. The L2 unit ball is a circle in two dimensions; the L1 ball is a diamond; the L∞ ball is a square. Those shapes explain why different norms favour different solutions.