Skip to content
Quantoros

What are the three stages of RLHF, from preference data to the final model?

LLM & AILevel: FoundationFine-tuning & Alignment

What are the three stages of RLHF, from preference data to the final model?

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

Three stages. First, collect preferences: humans compare pairs of answers to the same prompt and pick the better one. Second, train a reward model: a network learns from those picks to output a single score for how much a human would prefer a reply. Third, optimize with RL: the model, now called the policy, writes answers, the reward model scores them, and PPO nudges the weights toward higher scores.

Source and licence: Skillumen — Awesome AI Engineer Interview Questions · CC0-1.0

Related questions