Skip to content
Quantoros

Why can halving learningrate while keeping numiterations fixed make a fast LightGBM experiment misleading?

Classic MLLevel: IntermediateLightGBM · Tuning

Why can halving learning_rate while keeping num_iterations fixed make a fast LightGBM experiment misleading?

Hints
  • Step size and number of steps interact.
  • Distinguish model capacity reached from per-round compute.

Answer

Each round contributes less, so the same round budget may underfit. Compare learning curves with a suitable expanded round budget and validation-based stopping; lowering the rate alone does not make each round cheaper.

Source and licence: LightGBM · Tuning · MIT · answer: Quantoros editorial

Related questions