Skip to content
Quantoros

Your LLM is revealing internal documents it shouldn't. How do you fix the access control failure?

LLM & AILevel: IntermediateTroubleshooting Scenarios

Your LLM is revealing internal documents it shouldn't. How do you fix the access control failure?

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

  1. Immediate: disable or add manual review before any document is cited in a response
  2. Root cause: check vector DB queries — are access control filters being applied to every retrieval query?
  3. Common failure modes:
    • ACL filter not applied (code regression or missing filter parameter)
    • Cached results from a different user's session being served
    • Metadata corruption — document stored with wrong ACL
  4. Fix: enforce ACL at retrieval layer, not just display layer. Every vector DB query must include filter={"allowed_users": {"contains": user.id}}
  5. Defense in depth: add output scanning — check if response contains any text matching high-sensitivity documents that the user shouldn't access

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions