Skip to content
Quantoros

Your chain-of-thought prompting is not improving accuracy on reasoning tasks. What do you fix?

LLM & AILevel: IntermediateTroubleshooting Scenarios

Your chain-of-thought prompting is not improving accuracy on reasoning tasks. What do you fix?

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

  1. Verify the base model — CoT only helps models ≥7B parameters; below that, it can hurt by adding noise
  2. Add concrete examples — zero-shot CoT ("think step by step") is weaker than few-shot CoT with demonstrations
  3. Check format — ensure the model actually produces a reasoning trace; if it jumps to the answer, explicitly instruct step-by-step format
  4. Use self-consistency — single CoT sample is noisy; majority-vote over 5 samples
  5. Decompose the task — if the task requires knowledge the model doesn't have, CoT won't help; use RAG instead

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions