Skip to content
Quantoros

How does a Random Forest differ from a single Decision Tree, and why does it usually generalize better?

Classic MLLevel: IntermediateClassical machine learning

How does a Random Forest differ from a single Decision Tree, and why does it usually generalize better?

Hints
  • How are training samples formed and how do ensemble trees differ?
  • Consider correlation between model errors, not only individual quality.

Answer

A Random Forest is an ensemble of decision trees trained on bootstrapped samples of the data, with each split considering only a random subset of features. This decorrelates the trees so their errors don't all point the same direction; averaging (regression) or majority voting (classification) across many decorrelated trees reduces variance substantially compared to one deep tree, without much increase in bias — this is the core idea of bagging.

Follow-up: How is Gradient Boosting different from Random Forest in terms of how it reduces error?

Source and licence: Ishan Dutta — Awesome Data Science Interview Q&A · MIT

Related questions