What is Paged Attention?
LLM & AILevel: IntermediateInference & Decoding
What is Paged Attention?
Hints
- Track Q, K and V shapes and which positions can interact.
- Check masking, weight normalization and training versus autoregressive inference.
Answer
PagedAttention (vLLM, 2023) manages KV cache memory like an OS virtual memory system:
- Divides KV cache into fixed-size pages (blocks)
- Maintains a page table mapping logical sequence positions to physical memory locations
- Allows non-contiguous storage of KV cache
- Enables sharing KV cache across requests with common prefixes
Result: near-zero memory waste from fragmentation (traditional KV cache allocation wastes up to 60%), enabling 2–4× higher batch sizes and throughput.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0