Explain greedy decoding vs beam search vs sampling.
LLM & AILevel: IntermediateInference & Decoding
Explain greedy decoding vs beam search vs sampling.
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
| Strategy | How | Best For |
|---|---|---|
| Greedy | Always pick highest-probability token | Fastest, deterministic, often repetitive |
| Beam search | Keep top-k partial sequences, pick best complete sequence | Machine translation, summarization (exact outputs) |
| Temperature sampling | Sample from softmax distribution | Creative generation, chat |
| Nucleus (top-p) | Sample from top-p cumulative probability | General purpose — best quality/diversity balance |
Beam search is deterministic and optimizes sequence-level probability but can be generic/repetitive. Sampling is stochastic but produces more diverse outputs.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0