Why your 3-D intuition is wrong in 768 dimensions

The picture you are drawing is misleading you

Every explanation of embeddings shows a 2-D or 3-D plot: dots, clusters, arrows. It is the only way to draw the thing. It is also wrong in ways that will cost you, because spaces with hundreds of dimensions behave nothing like the space you can see.

Here is the single most useful fact. Take two random vectors in d dimensions, each with entries drawn independently from a normal distribution, and compute the cosine of the angle between them. The expected value is 0 — they are perpendicular on average. That much is unsurprising. What surprises people is the spread: the standard deviation of that cosine is approximately 1/sqrt(d).

Run the numbers:

• In 3 dimensions: 1/sqrt(3) = 0.577. Random vectors routinely land at cosine 0.5 or −0.5. Everything looks related to everything.
• In 100 dimensions: 1/sqrt(100) = 0.10. A cosine of 0.3 is now three standard deviations out.
• In 768 dimensions: 1/sqrt(768) = 0.036. A cosine of 0.11 is already three standard deviations from random.

So in a 768-dimensional space, two vectors that a 3-D intuition would call "barely related at 0.15" are in fact a four-sigma event. Almost everything is almost perpendicular to almost everything else. There is an enormous amount of room.

You can verify this in ten lines of free NumPy:

import numpy as np
for d in (3, 100, 768):
    a = np.random.randn(20000, d)
    a /= np.linalg.norm(a, axis=1, keepdims=True)
    b = np.random.randn(20000, d)
    b /= np.linalg.norm(b, axis=1, keepdims=True)
    cos = (a * b).sum(axis=1)
    print(d, round(cos.std(), 3), round(1/np.sqrt(d), 3))

The two printed numbers agree to two decimal places. This is not a rule of thumb; it is what the space is.

Distances stop discriminating

The second strange fact is called distance concentration. Sample many points at random in d dimensions and look at the nearest and the farthest from a query point. As d grows, the ratio

(farthest distance - nearest distance) / nearest distance

tends towards zero. In high enough dimensions everything is roughly the same distance away, and "nearest neighbour" stops being a meaningful description. Formally this holds for independently distributed coordinates, and it is why brute-force nearest-neighbour search on genuinely random high-dimensional data is close to useless.

A third, related fact: in high dimensions, nearly all of a ball's volume sits in a thin shell just under its surface. The fraction of the volume of a d-dimensional unit ball lying within radius 0.9 is 0.9^d. For d = 3 that is 0.73; for d = 100 it is 0.000027. The interior is empty. Your mental image of a cluster as a filled blob is wrong; it is a shell.

So why does vector search work at all?

If distances concentrate, why does embedding search return sensible results every day?

Because real embeddings are not random. A trained model maps its inputs onto a much lower-dimensional structure inside the big space — a curved surface, usually called a manifold. The stated dimension is 768; the effective dimension, the number of directions along which the data actually varies, is typically far lower, often in the tens. Distance concentration is a statement about points spread through the whole space, and your points are not spread through the whole space.

This is exactly why approximate-nearest-neighbour indexes such as HNSW or IVF work. They exploit the fact that the data has local structure. On genuinely uniform random data they degrade towards brute force, which is one of the reasons a synthetic benchmark with random vectors tells you very little about how your index will behave on real ones.

What this changes in practice

Thresholds must be measured, never assumed. "Similarity above 0.8 means a match" is a claim about one model's output distribution. Compute the distribution of scores for your model on a few hundred labelled pairs and read the cut-off from it.

Beware anisotropy. Many trained embedding models push all their vectors into a narrow cone, so the average cosine between two unrelated sentences might be 0.6 rather than 0. The variance around that average is what carries the signal. If your scores all cluster between 0.55 and 0.75, that is normal, and subtracting the mean before comparing often sharpens results noticeably.

More dimensions are not free. Doubling dimensions doubles storage and query cost, and beyond the data's effective dimension it adds noise rather than signal. Measured retrieval quality often plateaus well before the model's full width, which is the reasoning behind models that let you truncate the vector to 256 or 512 dimensions with a small, measurable quality cost.

The rule to keep

1/sqrt(d) is the number to remember. It tells you how far from zero a cosine has to be before it means anything, and it explains why your intuition, trained in three dimensions, will always overestimate how related two things are.