What are the disadvantages of using classic decision tree algorithm for a large dataset?
Classic MLLevel: FoundationML concepts — Sebastian Raschka FAQ
What are the disadvantages of using classic decision tree algorithm for a large dataset?
Hints
- How is one split selected and which objective does it improve?
- Check stopping rules, leaf size and the difference between fitting and prediction.
Answer
A single unrestricted decision tree can grow large, overfit and change substantially after small changes to the training sample. Axis-aligned splits may require many nodes to approximate a smooth or oblique boundary. On a large dataset, searching candidate splits, moving sample indices and retaining nodes also consume time and memory.
There is no universal separate O(features²) or O(samples²) rule. Training cost depends on the split algorithm, sorting/caching strategy, number of candidate features and tree shape. Prediction follows one root-to-leaf path, so its cost is proportional to depth; an approximately balanced tree is much cheaper than a very deep chain.
Limit depth or leaf size, prune where appropriate, and compare against ensembles or histogram-based learners using validation and measured resource costs.
Source and licence: Sebastian Raschka — Python Machine Learning FAQ · MIT · answer: Quantoros editorial