Skip to content
Quantoros

What is autoregressive generation and what are its limitations?

LLM & AILevel: FoundationInference & Decoding

What is autoregressive generation and what are its limitations?

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

Autoregressive generation produces text one token at a time, each conditioned on all previous tokens. This captures coherent sequential structure but has key limitations:

  • Slow: inherently sequential — can't parallelize generation (only prefill)
  • Error accumulation: each token conditions on all previous, so early mistakes propagate
  • No revision: unlike humans, the model cannot go back and edit

Alternatives like diffusion language models aim to generate all tokens in parallel, but current quality lags autoregressive models for text.

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions