What are guardrails in LLM applications?
ML System DesignLevel: FoundationGuardrails & Safety Controls
What are guardrails in LLM applications?
Hints
- Clarify the threat: an unsupported fact, an untrusted instruction or an unsafe action?
- Separate input checks, authority limits and output validation; consider false positives.
Answer
Guardrails are validation layers that control LLM inputs and outputs to enforce safety, quality, and compliance constraints.
Input guardrails (before sending to LLM):
- Prompt injection detection
- PII detection and masking
- Topic filtering (block off-topic queries for specialized apps)
- Jailbreak detection
Output guardrails (after LLM response, before showing user):
- Factual grounding check (does response contradict retrieved sources?)
- Toxicity / harmful content filter
- PII leakage detection
- Format validation (did the model follow the requested schema?)
- Brand safety (no competitor mentions, no false claims)
Tools: Guardrails AI (open source), NVIDIA NeMo Guardrails, Lakera Guard, Llama Guard.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0