Your chain-of-thought prompting is not improving accuracy on reasoning tasks. What do you fix?
LLM & AILevel: IntermediateTroubleshooting Scenarios
Your chain-of-thought prompting is not improving accuracy on reasoning tasks. What do you fix?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
- Verify the base model — CoT only helps models ≥7B parameters; below that, it can hurt by adding noise
- Add concrete examples — zero-shot CoT ("think step by step") is weaker than few-shot CoT with demonstrations
- Check format — ensure the model actually produces a reasoning trace; if it jumps to the answer, explicitly instruct step-by-step format
- Use self-consistency — single CoT sample is noisy; majority-vote over 5 samples
- Decompose the task — if the task requires knowledge the model doesn't have, CoT won't help; use RAG instead
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0