Skip to content
Quantoros

What is the LLM training pipeline?

LLM & AILevel: IntermediateTraining & Alignment

What is the LLM training pipeline?

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

  1. Pre-training — next-token prediction on trillions of web/book/code tokens. Builds general knowledge and language understanding.
  2. SFT (Supervised Fine-Tuning) — fine-tune on curated instruction-following examples. Teaches the model to follow instructions.
  3. Reward Modeling — train a reward model on human preference pairs (A vs B, which is better?)
  4. RLHF/DPO — optimize model outputs toward higher reward. Creates the helpful, harmless, honest behavior of deployed models.

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions