What is BLEU and when should (not) you use it?
LLM & AILevel: FoundationAutomated Metrics
What is BLEU and when should (not) you use it?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
BLEU (Bilingual Evaluation Understudy): measures n-gram overlap between model output and reference text.
Where BP = brevity penalty, pₙ = n-gram precision.
When to use: machine translation (comparing translation to professional reference), structured text generation with single correct forms.
When NOT to use: open-ended generation, QA with multiple acceptable answers, chatbot responses. BLEU penalizes paraphrases that are semantically equivalent but lexically different — a perfect answer not using the same words scores near 0.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0