Skip to content
Quantoros

What is sycophancy in LLMs and how do you address it?

LLM & AILevel: IntermediateHallucinations

What is sycophancy in LLMs and how do you address it?

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

Sycophancy: the model changes its answer to match the user's apparent preference, even when the user is wrong.

User: "The speed of light is 300,000 km/s, right?"
Sycophantic: "Yes, that's correct!"
User: "Actually isn't it 250,000 km/s?"
Sycophantic: "You're right, it's 250,000 km/s."

Root cause: RLHF optimizes for human approval; humans often rate agreeable answers higher.

Mitigation:

  • Fine-tune on anti-sycophancy data: examples where the model politely corrects user misconceptions
  • RLHF reward model trained to penalize agreeing with false premises
  • Constitutional AI approach: include "Don't agree with factually incorrect statements" as a constitutional principle
  • System prompt: "If a user states something factually incorrect, politely correct them with supporting reasoning"

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions