Skip to content
Quantoros

RAG vs Fine-tuning vs In-context Learning — when to use which?

ML System DesignLevel: IntermediateSystem Design Framework

RAG vs Fine-tuning vs In-context Learning — when to use which?

Hints
  • Break the query into candidate retrieval, context selection and generation.
  • How would you locate the stage that lost a fact without using only one end-to-end metric?

Answer

In-context LearningRAGFine-tuning
Knowledge sourceModel weights + contextExternal documentsUpdated weights
Knowledge freshnessStatic (training cutoff)Real-timeStatic (re-train to update)
PersonalizationLimited (long prompt)Dynamic retrievalStatic per version
LatencyFastModerate (retrieval overhead)Fastest (no retrieval)
CostHigh (large prompts)ModerateHigh upfront, low at inference
WhenFew examples, quick prototypingLarge/dynamic knowledge baseConsistent format or style

Decision rule: start with prompting → add RAG when knowledge is large or dynamic → add fine-tuning when consistent behavior or format is the primary requirement.

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions