What is autoregressive generation and what are its limitations?
LLM & AILevel: FoundationInference & Decoding
What is autoregressive generation and what are its limitations?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
Autoregressive generation produces text one token at a time, each conditioned on all previous tokens. This captures coherent sequential structure but has key limitations:
- Slow: inherently sequential — can't parallelize generation (only prefill)
- Error accumulation: each token conditions on all previous, so early mistakes propagate
- No revision: unlike humans, the model cannot go back and edit
Alternatives like diffusion language models aim to generate all tokens in parallel, but current quality lags autoregressive models for text.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0