How long, and how far

One number for a whole list

You have a vector. Sometimes you want a single number saying how big it is. That number is called a norm, and there is more than one sensible answer.

Take v = [3, 4].

The L2 norm, also called the Euclidean norm, is the one from school geometry: square each element, add them, take the square root.

||v||2 = sqrt(3^2 + 4^2) = sqrt(9 + 16) = sqrt(25) = 5

The L1 norm, also called Manhattan distance, adds absolute values:

||v||1 = |3| + |4| = 7

The L-infinity norm takes the largest absolute value:

||v||inf = max(|3|, |4|) = 4

Three answers, all correct, all measuring something slightly different. The names come from a family: the Lp norm raises each element to the power p, sums, and takes the p-th root. L2 is p = 2; L1 is p = 1; L-infinity is the limit.

Distance is the norm of a difference

Distance between two vectors is just the norm of what separates them. For a = [1, 2, 3] and b = [4, 6, 3]:

a - b = [-3, -4, 0]
L2 distance = sqrt(9 + 16 + 0) = 5
L1 distance = 3 + 4 + 0 = 7

The L1 name comes from walking a street grid: you cannot cut the diagonal, so you go three blocks then four blocks, seven blocks total. L2 is the crow's route, five blocks.

Why the choice changes your answer

The difference is entirely about outliers. Squaring makes a large element enormously more important than a small one. Compare [10, 0, 0, 0] with [5, 5, 5, 5]:

• L1: both are 10. They look identical.
• L2: sqrt(100) = 10 against sqrt(100) = 10. Also identical, by coincidence of the numbers chosen.

Now compare [10, 0, 0, 0] with [3, 3, 3, 3]:

• L1: 10 against 12. The spread-out one is bigger.
• L2: 10 against 6. The spiky one is bigger.

That reversal is the whole point. L2 says "one big deviation is worse than four small ones". L1 says "add up the damage, however it is distributed". This is why L2 loss (mean squared error) is pulled around by a single wild data point while L1 loss (mean absolute error) mostly ignores it, and why L1 regularisation drives weights exactly to zero while L2 only shrinks them.

Unit vectors: keeping the direction, dropping the size

Divide a vector by its own L2 norm and you get a vector of length exactly 1, pointing the same way.

v = [3, 4],  ||v||2 = 5
v_hat = [3/5, 4/5] = [0.6, 0.8]
check: sqrt(0.36 + 0.64) = sqrt(1) = 1

This is called normalising, and it is everywhere in machine learning. Embedding search libraries normalise every vector on insert, so that a dot product is a cosine and no division is needed at query time. Layer normalisation inside a transformer does a related thing to each token's activation vector on every layer.

The reason is the one from the dot-product lesson: raw dot products grow with length, so a long vector scores high against everything. Normalising strips length out and leaves only direction, which is where the meaning lives.

What normalising costs you

Length is not always noise. In a bag-of-words representation, the length of a document vector carries the document's size, and sometimes a long document genuinely is a better match. In an embedding from a modern model, vector norms have been shown to correlate loosely with token frequency and with how confident the model is — normalise, and you discard that.

There is also a practical trap. If you normalise your database vectors but forget to normalise the query, your scores are neither dot products nor cosines, and the ranking is quietly wrong in a way that still returns plausible-looking results. This is one of the most common bugs in a hand-built search system, and it never raises an error.

The rule to keep

Ask what you want the number to mean. If a single large deviation should dominate, square it: use L2. If you want a total that treats every unit of error alike, use L1. If you only care about the worst element — a latency budget, a safety margin — use L-infinity. And when you compare directions rather than sizes, normalise first, on both sides.