Why is a bad LLM output called a symptom rather than a diagnosis, and what's the overall approach?
LLM & AILevel: IntermediateEvaluation
Why is a bad LLM output called a symptom rather than a diagnosis, and what's the overall approach?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
The same visible failure, a bad answer, can come from any layer of the pipeline: the prompt, the model, retrieval, the schema and parser, or a tool call. The screen shows you the end of the pipeline, never the origin. So the job is to turn one vague symptom into a specific, provable layer by reading the tell, whether that is malformed JSON, a 4xx, a made-up fact, or an ignored rule. Then you fix the cheapest layer first and confirm on a fixed eval before moving on.
Source and licence: Skillumen — Awesome AI Engineer Interview Questions · CC0-1.0