Why is converting a generative model into an embedding model not enough to ensure good retrieval?
LLM & AILevel: IntermediateEmbedding model task suitability
Why is converting a generative model into an embedding model not enough to ensure good retrieval?
Hints
- What objective shaped the representation space?
- Check pooling and retrieval evaluation, not just vector size.
Answer
Extracting and pooling hidden states provides vectors, but a language-modeling objective does not directly optimize retrieval similarity. A dedicated embedding checkpoint usually has relevant training and a defined pooling, normalization and input-format contract. Evaluate Recall@k or ranking metrics on the target corpus; compare models with the same retrieval pipeline and do not select solely by parameter count.
Source and licence: vLLM contributors — FAQ · Apache-2.0 · answer: Quantoros editorial