Skip to content
Quantoros

Why is a bad LLM output called a symptom rather than a diagnosis, and what's the overall approach?

LLM & AILevel: IntermediateEvaluation

Why is a bad LLM output called a symptom rather than a diagnosis, and what's the overall approach?

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

The same visible failure, a bad answer, can come from any layer of the pipeline: the prompt, the model, retrieval, the schema and parser, or a tool call. The screen shows you the end of the pipeline, never the origin. So the job is to turn one vague symptom into a specific, provable layer by reading the tell, whether that is malformed JSON, a 4xx, a made-up fact, or an ignored rule. Then you fix the cheapest layer first and confirm on a fixed eval before moving on.

Source and licence: Skillumen — Awesome AI Engineer Interview Questions · CC0-1.0

Related questions