Skip to content
Quantoros

How does XGBoost avoid enumerating all category subsets for scalar-leaf partition splits?

Classic MLLevel: IntermediateXGBoost · Categorical data

How does XGBoost avoid enumerating all category subsets for scalar-leaf partition splits?

Hints
  • The useful ordering is learned from the node.
  • Search boundaries in a sorted sequence.

Answer

It orders category gradient statistics by the induced leaf values and searches contiguous partitions in that order. This ordering depends on the training objective and node statistics, not alphabetical category names.

Source and licence: XGBoost · Categorical data · Apache-2.0 · answer: Quantoros editorial

Related questions