What is the "aha moment" in reasoning model training?
LLM & AILevel: IntermediateTraining Methodology
What is the "aha moment" in reasoning model training?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
During RLVR training, researchers observe an emergent phenomenon: models spontaneously develop self-reflection behaviors — including re-examining their work with phrases like "wait, let me reconsider" or "actually that's wrong."
This wasn't explicitly trained — it emerged from the RL signal of getting math problems right. The model learned that reconsidering intermediate steps improves its reward.
This suggests the thinking trace isn't just outputting a pre-formed solution — it's actively being used for computation. The model is genuinely "thinking" in a meaningful sense, not just pattern-matching.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0