Skip to content
Quantoros

What is temperature, top-p, and top-k?

LLM & AILevel: FoundationInference & Decoding

What is temperature, top-p, and top-k?

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

All three control token sampling randomness:

Temperature (τ): logits_scaled = logits / τ

  • τ → 0: deterministic (greedy)
  • τ = 1: standard sampling
  • τ > 1: more random/creative

Top-k: retain only top k tokens, renormalize, sample. Fixed cutoff.

Top-p (nucleus): retain smallest set of tokens whose cumulative probability ≥ p. Adaptive — naturally narrows when one token dominates.

Best practice: use temperature (0.7–1.0) + top-p (0.9–0.95) together. Top-p adapts better than top-k to distribution shape.

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions