Skip to content
Quantoros

How do we select the depth of the trees in random forest?

Classic MLLevel: IntermediateRandom forest

How do we select the depth of the trees in random forest?

Hints
  • How are training samples formed and how do ensemble trees differ?
  • Consider correlation between model errors, not only individual quality.

Answer

The greater the depth, the greater amount of information is extracted from the tree, however, there is a limit to this, and the algorithm even if defensive against overfitting may learn complex features of noise present in data and as a result, may overfit on noise. Hence, there is no hard thumb rule in deciding the depth, but literature suggests a few tips on tuning the depth of the tree to prevent overfitting:

  • limit the maximum depth of a tree
  • limit the number of test nodes
  • limit the minimum number of objects at a node required to split
  • do not split a node when, at least, one of the resulting subsample sizes is below a given threshold
  • stop developing a node if it does not sufficiently improve the fit.

Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0

Related questions