Skip to content
Quantoros

What is test-time compute (TTC) and why is it important?

LLM & AILevel: FoundationWhat Are Reasoning Models

What is test-time compute (TTC) and why is it important?

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

Test-time compute refers to the amount of computation applied during inference — beyond just the forward pass to generate tokens. The core insight: you can trade more computation at inference time for better quality outputs.

Traditional scaling law: more parameters + more training data = better model.

TTC insight (OpenAI o1, 2024): for tasks with verifiable correct answers, spending more tokens thinking at inference reliably improves accuracy — similar to how a human does better on a math exam with scratch paper vs. answering immediately.

This is a fundamental shift because:

  • It decouples quality from model size (a smaller model that thinks longer can beat a larger model that doesn't)
  • It gives users a quality/cost/latency dial at inference time
  • It opens a new scaling axis orthogonal to training

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions