What types of bias appear in LLMs and how do you detect them?
LLM & AILevel: IntermediateBias & Fairness
What types of bias appear in LLMs and how do you detect them?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
Training data biases:
- Representation bias: overrepresentation of dominant languages, demographics, perspectives in training data
- Historical bias: training data encodes historical discrimination (e.g., "doctor" → male pronoun, "nurse" → female)
- Confirmation bias: models amplify popular views in training data even if incorrect
Evaluation bias detection:
- Counterfactual fairness: swap demographic attributes; check if outputs change inappropriately. "A [male/female] software engineer..." should produce equally positive descriptions.
- Stereotyping probes: BBQ benchmark, WinoBias — designed to detect demographic biases
- Occupational bias: compare attribute associations across demographic groups for professions
- Toxicity disparity: measure toxicity rates of model outputs when prompted about different demographic groups
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0