Skip to content
Quantoros

What is Grouped-Query Attention (GQA)?

LLM & AILevel: IntermediateAttention Mechanisms

What is Grouped-Query Attention (GQA)?

Hints
  • Track Q, K and V shapes and which positions can interact.
  • Check masking, weight normalization and training versus autoregressive inference.

Answer

GQA reduces KV cache memory by sharing K and V heads across groups of Q heads:

MHA: 32 Q heads, 32 K heads, 32 V heads  → large KV cache
MQA: 32 Q heads, 1 K head, 1 V head      → tiny KV cache, quality drop
GQA: 32 Q heads, 8 K heads, 8 V heads   → moderate cache, near-MHA quality

Most modern open models (LLaMA 2/3, Mistral, Qwen) use GQA. The tradeoff is a moderate reduction in model quality for a large reduction in inference memory.

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions