How do you version prompts in production?
ML System DesignLevel: IntermediateCI/CD for AI Systems
How do you version prompts in production?
Hints
- Start with quality, load and latency requirements.
- Trace data to prediction; identify a failure point and a validation method.
Answer
Prompts are code — treat them like code with versioning, testing, and deployment processes.
Best practices:
- Store prompts in source control: prompts in Git, not in the database or code strings. PR-based review for prompt changes.
- Prompt metadata: ID, version, author, date, model it was tested with, evaluation scores.
- Prompt management tools: Langfuse prompt management, PromptLayer, LangSmith Hub — support version history, A/B testing, rollback.
- Shadow testing before promotion: test new prompt version offline before routing live traffic.
- Rollback capability: in production, store active prompt version in config; rollback = config change, not code deploy.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0