Skip to content
Quantoros

Feature importance in gradient boosting trees — what are possible options?

Classic MLLevel: IntermediateGradient boosting

Feature importance in gradient boosting trees — what are possible options?

Hints
  • What does the next learner predict relative to the current ensemble?
  • Connect sequential model additions to loss optimization.

Answer

With CatBoost you can use implemented method get_feature_importance for getting SHAP values. https://arxiv.org/abs/1905.04610v1

It allows to understand how excluding features helps to provide better results. Higher value is better.

Also you can add random noise column to your data (with normal distribution), calculate feature importance and exclude all features below noise importance.

Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0

Related questions