Is it possible to parallelize training of a gradient boosting model? How to do it?
Classic MLLevel: IntermediateGradient boosting
Is it possible to parallelize training of a gradient boosting model? How to do it?
Hints
- What does the next learner predict relative to the current ensemble?
- Connect sequential model additions to loss optimization.
Answer
Yes, different frameworks provide different options to make training faster, using GPUs to speed up the process by making it highly parallelizable.For example, for XGBoost tree_method = 'gpu_hist' option makes training faster by use of GPUs.
Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0