Skip to content
Quantoros

What specific capability does each of MMLU, GPQA, HumanEval, SWE-bench, and AIME actually measure?

LLM & AILevel: AdvancedEvaluation

What specific capability does each of MMLU, GPQA, HumanEval, SWE-bench, and AIME actually measure?

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

This answer is part of mentoring

Advanced questions come with a full breakdown and a mock interview with a working quant researcher. The first call is free.

Source and licence: Skillumen — Awesome AI Engineer Interview Questions · CC0-1.0

Related questions