Skip to content
Quantoros

How do you handle imbalanced classes when computing evaluation metrics, statistically speaking?

StatisticsLevel: IntermediateStatistics and probability

How do you handle imbalanced classes when computing evaluation metrics, statistically speaking?

Hints
  • Write TP, FP, TN and FN for a small example.
  • Separate ranking quality, threshold selection and probability calibration. Which error costs more?

Answer

Accuracy is misleading on imbalanced data because a naive "always predict majority class" model scores high. Prefer precision, recall, F1, and PR-AUC (more informative than ROC-AUC under heavy imbalance), and consider class-weighted loss functions, resampling (SMOTE, undersampling), or stratified cross-validation to keep class ratios consistent across folds. Also consider whether the business cost of false positives vs. false negatives is symmetric — if not, optimize a cost-weighted metric directly rather than a generic one.

Source and licence: Ishan Dutta — Awesome Data Science Interview Q&A · MIT

Related questions