What is test-time compute (TTC) and why is it important?
LLM & AILevel: FoundationWhat Are Reasoning Models
What is test-time compute (TTC) and why is it important?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
Test-time compute refers to the amount of computation applied during inference — beyond just the forward pass to generate tokens. The core insight: you can trade more computation at inference time for better quality outputs.
Traditional scaling law: more parameters + more training data = better model.
TTC insight (OpenAI o1, 2024): for tasks with verifiable correct answers, spending more tokens thinking at inference reliably improves accuracy — similar to how a human does better on a math exam with scratch paper vs. answering immediately.
This is a fundamental shift because:
- It decouples quality from model size (a smaller model that thinks longer can beat a larger model that doesn't)
- It gives users a quality/cost/latency dial at inference time
- It opens a new scaling axis orthogonal to training
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0