Why can a CatBoost ranking feature have high PredictionValuesChange yet barely improve the ranking metric?
Classic MLLevel: IntermediateCatBoost · Feature importance
Why can a CatBoost ranking feature have high PredictionValuesChange yet barely improve the ranking metric?
Hints
- Ranking metrics care about ordering within a query.
- A groupwise offset can change scores without changing order.
Answer
Prediction values can shift substantially between groups without improving within-group ordering. Use a metric-aligned importance analysis and held-out ablation, not raw score variation alone.
Source and licence: CatBoost · Feature importance · Apache-2.0 · answer: Quantoros editorial