How do you design prompts for consistent structured output (JSON, XML)?
LLM & AILevel: IntermediateAdvanced Prompting
How do you design prompts for consistent structured output (JSON, XML)?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
In order of reliability:
- Native JSON mode — provider enforces valid JSON at decode level (OpenAI
response_format, Anthropic tool use). Most reliable. - JSON schema in prompt — describe the schema with a concrete example in the system prompt. Use few-shot examples.
- Grammar-constrained decoding — local models can use
llama.cppgrammars oroutlineslibrary to constrain sampling to valid schema tokens. 100% valid output. - Parse + retry loop — try to parse, if it fails, send error back to model with "fix this JSON:" instruction. Usually succeeds in 1–2 retries.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0