What is embedding dimensionality and how does it affect performance?
NLPLevel: IntermediateEmbeddings Fundamentals
What is embedding dimensionality and how does it affect performance?
Hints
- Which information should a lower-dimensional representation preserve?
- Check centering, scaling and the difference between projection and feature selection.
Answer
Higher dimensionality = more expressive, better quality — at the cost of storage and compute:
| Dimensions | Storage per vector | ANN search speed | Quality |
|---|---|---|---|
| 384 | 1.5 KB | Fast | Good for many tasks |
| 768 | 3 KB | Moderate | Strong general quality |
| 1536 | 6 KB | Slower | High quality (text-embedding-3-small) |
| 3072 | 12 KB | Slow | Best quality (text-embedding-3-large) |
For 10M vectors at 1536 dims: 60 GB storage. Many systems use dimension reduction (PCA, Matryoshka Representation Learning) to truncate to smaller dimensions without proportional quality loss.
Matryoshka embeddings (OpenAI text-embedding-3): trained so that the first N dimensions form a valid embedding of quality proportional to N. You can truncate from 3072 to 256 and still get usable quality.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0