Why should two CatBoost LossFunctionChange reports computed on different customer cohorts not be expected to match?
Classic MLLevel: IntermediateCatBoost · Feature importance
Why should two CatBoost LossFunctionChange reports computed on different customer cohorts not be expected to match?
Hints
- The calculation consumes data in addition to a fitted model.
- The same model can behave differently across populations.
Answer
The importance is calculated using a supplied dataset and is cohort-dependent. Different feature distributions and errors change the metric effect of approximate removal; compare like-for-like cohorts and report the reference population.
Source and licence: CatBoost · Feature importance · Apache-2.0 · answer: Quantoros editorial