Your LLM keeps ignoring your instructions. How do you make it follow structured output formats?
LLM & AILevel: IntermediateTroubleshooting Scenarios
Your LLM keeps ignoring your instructions. How do you make it follow structured output formats?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
- Explicit format instruction — describe the exact schema in the system prompt with an example
- Few-shot examples — include 2–3 example input/output pairs showing the format
- JSON mode / structured outputs — use provider features (OpenAI's
response_format, function calling) that constrain output to valid JSON - Output parsing with retry — if output doesn't match schema, retry with error feedback
- Fine-tuning — last resort: fine-tune on examples of the desired format
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0