Your few-shot examples give inconsistent results across similar inputs. How do you stabilize it?
LLM & AILevel: IntermediateTroubleshooting Scenarios
Your few-shot examples give inconsistent results across similar inputs. How do you stabilize it?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
- Diversify examples — ensure few-shot examples cover the range of inputs you expect (edge cases, not just happy path)
- Check for confounding patterns — if all "positive" examples are short and all "negative" are long, the model may learn length as the signal
- Add contrastive examples — include pairs of similar-looking inputs with different correct outputs
- Use self-consistency — sample 3–5 times and take majority vote for critical decisions
- Evaluate per-category — decompose accuracy by input type; fix the categories that fail
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0