Skip to content
Quantoros

Why can a CatBoost ranking feature have high PredictionValuesChange yet barely improve the ranking metric?

Classic MLLevel: IntermediateCatBoost · Feature importance

Why can a CatBoost ranking feature have high PredictionValuesChange yet barely improve the ranking metric?

Hints
  • Ranking metrics care about ordering within a query.
  • A groupwise offset can change scores without changing order.

Answer

Prediction values can shift substantially between groups without improving within-group ordering. Use a metric-aligned importance analysis and held-out ablation, not raw score variation alone.

Source and licence: CatBoost · Feature importance · Apache-2.0 · answer: Quantoros editorial

Related questions