What is catastrophic forgetting and how do you prevent it?
LLM & AILevel: IntermediateAlignment Training
What is catastrophic forgetting and how do you prevent it?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
Catastrophic forgetting: fine-tuning on a narrow domain degrades the model's general capabilities — a legal fine-tune becomes worse at math, coding, and general language understanding.
Prevention strategies:
- LoRA — most effective in practice; frozen base weights retain all general capabilities; only the adapter is task-specific
- Data mixing — include 5–20% general-domain data (e.g., from OpenHermes, ShareGPT) in your fine-tuning dataset
- Elastic Weight Consolidation (EWC) — regularize updates to parameters important for prior tasks; works but computationally expensive
- Low learning rate + fewer epochs — conservative training stays closer to the pre-trained distribution
- Evaluation — track general benchmarks (MMLU, HellaSwag) alongside task-specific metrics during training; stop before general performance degrades
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0