RAG vs fine-tuning — when do you choose each?
LLM & AILevel: IntermediateCore Concepts
RAG vs fine-tuning — when do you choose each?
Hints
- Break the query into candidate retrieval, context selection and generation.
- How would you locate the stage that lost a fact without using only one end-to-end metric?
Answer
| Factor | Choose RAG | Choose Fine-Tuning |
|---|---|---|
| Knowledge type | Factual, frequently changing | Style, format, behavioral patterns |
| Update frequency | Daily/weekly knowledge updates | Behavior set once |
| Auditability | Need to cite sources | Output quality is the goal |
| Training data | Documents/text corpus available | High-quality Q&A pairs needed |
| Cost | Lower upfront | Higher (GPU training required) |
| Hallucination risk | Lower (grounded in docs) | Higher for knowledge injection |
| Latency | Adds retrieval step (~100-500ms) | No retrieval overhead |
In practice: start with RAG. Add fine-tuning if RAG can't achieve required output style or format. RAG + fine-tuning together achieves maximum performance (model adapts to domain, RAG provides fresh knowledge).
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0