Explain Gradient Boosting at a conceptual level.
Classic MLLevel: IntermediateClassical machine learning
Explain Gradient Boosting at a conceptual level.
Hints
- What does the next learner predict relative to the current ensemble?
- Connect sequential model additions to loss optimization.
Answer
Gradient Boosting builds an ensemble sequentially: each new weak learner (typically a shallow tree) is trained to predict the residual errors (negative gradient of the loss) of the current ensemble, and its predictions are added with a shrinkage factor (learning rate). Unlike bagging, which reduces variance via averaging independent models, boosting reduces bias by iteratively correcting mistakes — making it powerful but more prone to overfitting if not regularized (via learning rate, tree depth, early stopping, subsampling).
Source and licence: Ishan Dutta — Awesome Data Science Interview Q&A · MIT