Cosine or distance, and why it often does not matter

The identity that settles the argument

Teams argue about whether to use cosine similarity or Euclidean distance in a vector search. Most of the argument disappears once you write down one line of algebra.

For any two vectors a and b:

||a - b||^2  =  ||a||^2 + ||b||^2 - 2 (a . b)

Expand the squared distance term by term and you get exactly this; it is the law of cosines wearing different clothes.

Now suppose both vectors have been normalised to length 1. Then ||a||^2 = 1 and ||b||^2 = 1, and the dot product of two unit vectors is the cosine of the angle between them. So:

||a - b||^2 = 2 - 2 cos(theta)

Distance squared is a straight, decreasing function of cosine. Bigger cosine, smaller distance, always, with no exceptions. So if every vector in your index is normalised, sorting by cosine descending and sorting by Euclidean distance ascending give the same order. Not a similar order. The same one.

That is worth knowing before you spend an afternoon benchmarking one against the other.

Worked, with numbers

Take a = [0.6, 0.8] and b = [0, 1], both unit length.

a . b = 0.6*0 + 0.8*1 = 0.8          so cos(theta) = 0.8, theta = 36.9 degrees
a - b = [0.6, -0.2]
||a - b||^2 = 0.36 + 0.04 = 0.40
check: 2 - 2(0.8) = 2 - 1.6 = 0.40   matches

Try a third vector c = [1, 0]: a . c = 0.6, and ||a − c||² = 0.16 + 0.64 = 0.80 = 2 − 2(0.6). Cosine says b beats c; distance says b is closer. They agree, and the identity guarantees they always will.

When they genuinely differ

The identity depends on the lengths being equal to one. Drop that and the two measures pull apart, because ||a||^2 and ||b||^2 re-enter the equation.

Consider two documents about the same subject, one a 200-word note and one a 4,000-word article, represented as raw word-count vectors. Their directions are similar — same vocabulary, similar proportions — so the cosine is high. Their Euclidean distance is large, because one vector's entries are roughly twenty times the other's. Cosine calls them near-duplicates. Euclidean calls them far apart.

Which is right depends on the question. For "are these about the same thing", cosine. For "is this the same size of thing", the raw numbers matter and you should not have thrown length away.

Three practical cases where length carries meaning:

• Counts and frequencies. A word appearing 40 times is different evidence from it appearing twice, even at the same proportion.
• Physical measurements. Height, weight, price. Two flats with the same shape of features but doubled everything are not the same flat.
• Model activations you are inspecting. The magnitude of an activation is often the interesting part; normalising it away deletes what you were looking at.

The similarity numbers people quote

Cosine runs from −1 (opposite) through 0 (perpendicular) to 1 (identical direction). People report thresholds — "we accept matches above 0.8" — and those thresholds do not transfer between models. One embedding model may put unrelated sentence pairs around 0.1; another routinely puts them at 0.6 because its embeddings are squeezed into a narrow cone of the space, a property called anisotropy. A 0.7 in one model can mean less than a 0.4 in another.

The only defensible way to set a threshold is empirical: take a few hundred pairs you have labelled as matching or not matching, compute the scores with your actual model, and read the threshold off the distribution. Copying a number from a blog post about a different model is guessing.

What both measures share, and cannot fix

Neither cosine nor Euclidean distance knows anything about meaning. They measure agreement between two lists of numbers produced by a model, and they inherit everything that model got wrong. If the embedding model was trained mostly on English web text, two Hindi sentences that mean opposite things may sit close together simply because the model represents both weakly. The geometry will report a confident 0.91 and be entirely wrong, because the metric is doing its job perfectly on numbers that were bad before it saw them.

The rule to keep

Normalise on insert and on query, then use dot product — it is the cheapest of the three to compute, it equals cosine once vectors are unit length, and it ranks identically to Euclidean distance. Depart from that only when you have a stated reason why length matters, and write the reason down next to the code.