How do we select the depth of the trees in random forest?
Classic MLLevel: IntermediateRandom forest
How do we select the depth of the trees in random forest?
Hints
- How are training samples formed and how do ensemble trees differ?
- Consider correlation between model errors, not only individual quality.
Answer
The greater the depth, the greater amount of information is extracted from the tree, however, there is a limit to this, and the algorithm even if defensive against overfitting may learn complex features of noise present in data and as a result, may overfit on noise. Hence, there is no hard thumb rule in deciding the depth, but literature suggests a few tips on tuning the depth of the tree to prevent overfitting:
- limit the maximum depth of a tree
- limit the number of test nodes
- limit the minimum number of objects at a node required to split
- do not split a node when, at least, one of the resulting subsample sizes is below a given threshold
- stop developing a node if it does not sufficiently improve the fit.
Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0