Your RAG system fails for domain-specific jargon the embedding model doesn't understand.
LLM & AILevel: IntermediateTroubleshooting Scenarios
Your RAG system fails for domain-specific jargon the embedding model doesn't understand.
Hints
- Break the query into candidate retrieval, context selection and generation.
- How would you locate the stage that lost a fact without using only one end-to-end metric?
Answer
- Keyword search augmentation: add BM25 alongside vector search — BM25 matches exact terms even if embeddings don't capture meaning
- Domain-specific embedding model: fine-tune an embedding model on domain corpus; or use one pre-trained on similar domains (e.g.,
BioLinkBERTfor biomedical) - Synonym expansion: build a domain glossary; expand queries with known synonyms before retrieval
- Chunk size reduction: smaller chunks improve precision for specialized terminology retrieval
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0