What is the LLMOps lifecycle?
ML System DesignLevel: IntermediateLLMOps vs MLOps
What is the LLMOps lifecycle?
Hints
- Start with quality, load and latency requirements.
- Trace data to prediction; identify a failure point and a validation method.
Answer
1. Prototype
└─ Prompt engineering, choose base model, evaluate on small sample
2. Develop
└─ Build retrieval pipeline, tool integrations, guardrails, evals
3. Evaluate
└─ Offline evals (benchmarks, human eval), red-teaming, regression suite
4. Deploy
└─ Canary deploy, A/B test new prompt/model versions, monitoring
5. Monitor
└─ Track quality metrics, latency, cost, error rates, user feedback
6. Iterate
└─ Analyze failure cases, update prompts/RAG/fine-tune, re-evaluate
The key difference from MLOps: iteration cycles are faster (prompts can be changed in minutes), but "ground truth" is harder to define and requires continuous human eval.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0