Skip to content
Quantoros

Your few-shot examples give inconsistent results across similar inputs. How do you stabilize it?

LLM & AILevel: IntermediateTroubleshooting Scenarios

Your few-shot examples give inconsistent results across similar inputs. How do you stabilize it?

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

  1. Diversify examples — ensure few-shot examples cover the range of inputs you expect (edge cases, not just happy path)
  2. Check for confounding patterns — if all "positive" examples are short and all "negative" are long, the model may learn length as the signal
  3. Add contrastive examples — include pairs of similar-looking inputs with different correct outputs
  4. Use self-consistency — sample 3–5 times and take majority vote for critical decisions
  5. Evaluate per-category — decompose accuracy by input type; fix the categories that fail

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions