What is RAGAS and what metrics does it provide?
LLM & AILevel: IntermediateRAG Evaluation
What is RAGAS and what metrics does it provide?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
RAGAS (Retrieval-Augmented Generation Assessment) is a framework for evaluating RAG pipelines end-to-end.
Core metrics:
| Metric | Measures | How |
|---|---|---|
| Faithfulness | Is the answer grounded in the context? | LLM checks if each claim in the answer can be traced to retrieved context |
| Answer Relevancy | Does the answer address the question? | LLM generates questions from the answer; compare to original |
| Context Precision | Are retrieved chunks relevant? | Fraction of retrieved chunks that are relevant to the question |
| Context Recall | Are all relevant chunks retrieved? | Fraction of ground-truth answer components that appear in retrieved context |
Composite score: RAGAS score = harmonic mean of all four metrics.
Usage:
from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_precision
results = evaluate(
dataset,
metrics=[faithfulness, answer_relevancy, context_precision]
)
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0