How do we train decision trees?
Classic MLLevel: IntermediateDecision trees
How do we train decision trees?
Hints
- How is one split selected and which objective does it improve?
- Check stopping rules, leaf size and the difference between fitting and prediction.
Answer
- Start at the root node.
- For each variable X, find the set S_1 that minimizes the sum of the node impurities in the two child nodes and choose the split {X*,S*} that gives the minimum over all X and S.
- If a stopping criterion is reached, exit. Otherwise, apply step 2 to each child node in turn.
Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0