Skip to content
Quantoros

Explain greedy decoding vs beam search vs sampling.

LLM & AILevel: IntermediateInference & Decoding

Explain greedy decoding vs beam search vs sampling.

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

StrategyHowBest For
GreedyAlways pick highest-probability tokenFastest, deterministic, often repetitive
Beam searchKeep top-k partial sequences, pick best complete sequenceMachine translation, summarization (exact outputs)
Temperature samplingSample from softmax distributionCreative generation, chat
Nucleus (top-p)Sample from top-p cumulative probabilityGeneral purpose — best quality/diversity balance

Beam search is deterministic and optimizes sequence-level probability but can be generic/repetitive. Sampling is stochastic but produces more diverse outputs.

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions