What is BERTScore?
LLM & AILevel: IntermediateAutomated Metrics
What is BERTScore?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
BERTScore (Zhang et al., 2019) computes token-level similarity using contextual embeddings rather than n-gram overlap:
- Embed each token in reference and candidate using BERT
- For each candidate token, find the most similar reference token (soft alignment)
- Compute precision, recall, F1 over these soft alignments
Advantages over BLEU/ROUGE:
- Captures semantic similarity — paraphrases score high
- Handles synonyms and word order flexibility
- Correlates better with human judgments than BLEU
Disadvantage: slower (requires BERT forward pass); more opaque; can still be gamed.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0