How does a Random Forest differ from a single Decision Tree, and why does it usually generalize better?
Classic MLLevel: IntermediateClassical machine learning
How does a Random Forest differ from a single Decision Tree, and why does it usually generalize better?
Hints
- How are training samples formed and how do ensemble trees differ?
- Consider correlation between model errors, not only individual quality.
Answer
A Random Forest is an ensemble of decision trees trained on bootstrapped samples of the data, with each split considering only a random subset of features. This decorrelates the trees so their errors don't all point the same direction; averaging (regression) or majority voting (classification) across many decorrelated trees reduces variance substantially compared to one deep tree, without much increase in bias — this is the core idea of bagging.
Follow-up: How is Gradient Boosting different from Random Forest in terms of how it reduces error?
Source and licence: Ishan Dutta — Awesome Data Science Interview Q&A · MIT