Why do we need randomization in random forest?
Classic MLLevel: IntermediateRandom forest
Why do we need randomization in random forest?
Hints
- How are training samples formed and how do ensemble trees differ?
- Consider correlation between model errors, not only individual quality.
Answer
Random forest in an extension of the bagging algorithm which takes random data samples from the training dataset (with replacement), trains several models and averages predictions. In addition to that, each time a split in a tree is considered, random forest takes a random sample of m features from full set of n features (without replacement) and uses this subset of features as candidates for the split (for example, m = sqrt(n)).
Training decision trees on random data samples from the training dataset reduces variance. Sampling features for each split in a decision tree decorrelates trees.
Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0