RAG vs Fine-tuning vs In-context Learning — when to use which?
ML System DesignLevel: IntermediateSystem Design Framework
RAG vs Fine-tuning vs In-context Learning — when to use which?
Hints
- Break the query into candidate retrieval, context selection and generation.
- How would you locate the stage that lost a fact without using only one end-to-end metric?
Answer
| In-context Learning | RAG | Fine-tuning | |
|---|---|---|---|
| Knowledge source | Model weights + context | External documents | Updated weights |
| Knowledge freshness | Static (training cutoff) | Real-time | Static (re-train to update) |
| Personalization | Limited (long prompt) | Dynamic retrieval | Static per version |
| Latency | Fast | Moderate (retrieval overhead) | Fastest (no retrieval) |
| Cost | High (large prompts) | Moderate | High upfront, low at inference |
| When | Few examples, quick prototyping | Large/dynamic knowledge base | Consistent format or style |
Decision rule: start with prompting → add RAG when knowledge is large or dynamic → add fine-tuning when consistent behavior or format is the primary requirement.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0