What does instruction tuning teach a model that plain next-token pretraining does not?
LLM & AILevel: IntermediateFine-tuning & Alignment
What does instruction tuning teach a model that plain next-token pretraining does not?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
Pretraining only teaches what word comes next, so a base model continues text. Ask it a question and it may write more questions. Instruction tuning fine-tunes it on many pairs of an instruction and a good answer, so it learns the general habit of doing what is asked.
Source and licence: Skillumen — Awesome AI Engineer Interview Questions · CC0-1.0