What is sycophancy in LLMs and how do you address it?
LLM & AILevel: IntermediateHallucinations
What is sycophancy in LLMs and how do you address it?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
Sycophancy: the model changes its answer to match the user's apparent preference, even when the user is wrong.
User: "The speed of light is 300,000 km/s, right?"
Sycophantic: "Yes, that's correct!"
User: "Actually isn't it 250,000 km/s?"
Sycophantic: "You're right, it's 250,000 km/s."
Root cause: RLHF optimizes for human approval; humans often rate agreeable answers higher.
Mitigation:
- Fine-tune on anti-sycophancy data: examples where the model politely corrects user misconceptions
- RLHF reward model trained to penalize agreeing with false premises
- Constitutional AI approach: include "Don't agree with factually incorrect statements" as a constitutional principle
- System prompt: "If a user states something factually incorrect, politely correct them with supporting reasoning"
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0