Why can halving learningrate while keeping numiterations fixed make a fast LightGBM experiment misleading?
Classic MLLevel: IntermediateLightGBM · Tuning
Why can halving learning_rate while keeping num_iterations fixed make a fast LightGBM experiment misleading?
Hints
- Step size and number of steps interact.
- Distinguish model capacity reached from per-round compute.
Answer
Each round contributes less, so the same round budget may underfit. Compare learning curves with a suitable expanded round budget and validation-based stopping; lowering the rate alone does not make each round cheaper.
Source and licence: LightGBM · Tuning · MIT · answer: Quantoros editorial