Your LLM is revealing internal documents it shouldn't. How do you fix the access control failure?
LLM & AILevel: IntermediateTroubleshooting Scenarios
Your LLM is revealing internal documents it shouldn't. How do you fix the access control failure?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
- Immediate: disable or add manual review before any document is cited in a response
- Root cause: check vector DB queries — are access control filters being applied to every retrieval query?
- Common failure modes:
- ACL filter not applied (code regression or missing filter parameter)
- Cached results from a different user's session being served
- Metadata corruption — document stored with wrong ACL
- Fix: enforce ACL at retrieval layer, not just display layer. Every vector DB query must include
filter={"allowed_users": {"contains": user.id}} - Defense in depth: add output scanning — check if response contains any text matching high-sensitivity documents that the user shouldn't access
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0