Skip to content
Quantoros

How do you design prompts for consistent structured output (JSON, XML)?

LLM & AILevel: IntermediateAdvanced Prompting

How do you design prompts for consistent structured output (JSON, XML)?

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

In order of reliability:

  1. Native JSON mode — provider enforces valid JSON at decode level (OpenAI response_format, Anthropic tool use). Most reliable.
  2. JSON schema in prompt — describe the schema with a concrete example in the system prompt. Use few-shot examples.
  3. Grammar-constrained decoding — local models can use llama.cpp grammars or outlines library to constrain sampling to valid schema tokens. 100% valid output.
  4. Parse + retry loop — try to parse, if it fails, send error back to model with "fix this JSON:" instruction. Usually succeeds in 1–2 retries.

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions