Why can leaf-estimation backtracking improve a CatBoost update without adding another tree?
Classic MLLevel: IntermediateCatBoost · Training parameters
Why can leaf-estimation backtracking improve a CatBoost update without adding another tree?
Hints
- Think of step acceptance in numerical optimization.
- Shrinking an update is not growing the ensemble.
Answer
It reduces a proposed gradient/Newton step until the selected acceptance condition is met. This modifies leaf-value optimization inside the existing tree, not the ensemble size or split structure.
Source and licence: CatBoost · Training parameters · Apache-2.0 · answer: Quantoros editorial