Skip to content
Quantoros

What is Paged Attention?

LLM & AILevel: IntermediateInference & Decoding

What is Paged Attention?

Hints
  • Track Q, K and V shapes and which positions can interact.
  • Check masking, weight normalization and training versus autoregressive inference.

Answer

PagedAttention (vLLM, 2023) manages KV cache memory like an OS virtual memory system:

  • Divides KV cache into fixed-size pages (blocks)
  • Maintains a page table mapping logical sequence positions to physical memory locations
  • Allows non-contiguous storage of KV cache
  • Enables sharing KV cache across requests with common prefixes

Result: near-zero memory waste from fragmentation (traditional KV cache allocation wastes up to 60%), enabling 2–4× higher batch sizes and throughput.

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions