What is the difference between IVF and HNSW indexing?
NLPLevel: IntermediateIndexing & Search Algorithms
What is the difference between IVF and HNSW indexing?
Hints
- What does closeness mean in this representation and metric?
- Check normalization, exact versus approximate search and alignment with the training objective.
Answer
| HNSW | IVF (Inverted File Index) | |
|---|---|---|
| Structure | Navigable graph | Voronoi cell partitioning |
| Query speed | Very fast | Fast (with IVF-PQ, very fast) |
| Memory | High | Lower (especially with PQ compression) |
| Recall | Very high | Good with enough probes |
| Insertions | Incremental | Batch (requires rebuild) |
| Best for | General use, real-time insertions | Billions of vectors, memory-constrained |
IVF-PQ (Product Quantization): compresses vectors by encoding each sub-vector as an index into a learned codebook — reduces 1536×4 bytes → 96 bytes per vector. Enables billions of vectors in RAM.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0