What is ROUGE and how does it differ from BLEU?
LLM & AILevel: FoundationAutomated Metrics
What is ROUGE and how does it differ from BLEU?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
ROUGE (Recall-Oriented Understudy for Gisting Evaluation): measures n-gram overlap from a recall perspective.
- ROUGE-N: recall of n-grams from reference that appear in the model output
- ROUGE-L: longest common subsequence — captures sentence-level structure
- ROUGE-1: unigram overlap (word-level)
- ROUGE-2: bigram overlap (phrase-level)
| Metric | Orientation | Best for |
|---|---|---|
| BLEU | Precision | Translation quality |
| ROUGE | Recall | Summarization (coverage) |
| F1 | Balanced | When both coverage and precision matter |
Limitation: same as BLEU — no semantic understanding; paraphrases fail.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0