What is Grouped-Query Attention (GQA)?
LLM & AILevel: IntermediateAttention Mechanisms
What is Grouped-Query Attention (GQA)?
Hints
- Track Q, K and V shapes and which positions can interact.
- Check masking, weight normalization and training versus autoregressive inference.
Answer
GQA reduces KV cache memory by sharing K and V heads across groups of Q heads:
MHA: 32 Q heads, 32 K heads, 32 V heads → large KV cache
MQA: 32 Q heads, 1 K head, 1 V head → tiny KV cache, quality drop
GQA: 32 Q heads, 8 K heads, 8 V heads → moderate cache, near-MHA quality
Most modern open models (LLaMA 2/3, Mistral, Qwen) use GQA. The tradeoff is a moderate reduction in model quality for a large reduction in inference memory.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0