Skip to content
Quantoros

Why can leaf-estimation backtracking improve a CatBoost update without adding another tree?

Classic MLLevel: IntermediateCatBoost · Training parameters

Why can leaf-estimation backtracking improve a CatBoost update without adding another tree?

Hints
  • Think of step acceptance in numerical optimization.
  • Shrinking an update is not growing the ensemble.

Answer

It reduces a proposed gradient/Newton step until the selected acceptance condition is met. This modifies leaf-value optimization inside the existing tree, not the ensemble size or split structure.

Source and licence: CatBoost · Training parameters · Apache-2.0 · answer: Quantoros editorial

Related questions