Why might rare categorical levels need smoothing even when LightGBM handles them natively?
Classic MLLevel: IntermediateLightGBM · Advanced topics
Why might rare categorical levels need smoothing even when LightGBM handles them natively?
Hints
- Native support is not unlimited statistical evidence.
- Low-count estimates can be unstable.
Answer
Native splitting does not eliminate noisy statistics from low-support categories. Smoothing and minimum group support can reduce fragile splits; evaluate rare and unseen-category performance, not only the aggregate score.
Source and licence: LightGBM · Advanced topics · MIT · answer: Quantoros editorial