Incremental model updates are fast, but end-to-end training is slow. What component should be timed before changing…
Classic MLLevel: IntermediatePreprocessing bottleneck
Incremental model updates are fast, but end-to-end training is slow. What component should be timed before changing the learner?
Hints
- Measure the complete pipeline by stage.
- Fast partial_fit does not imply fast feature extraction.
Answer
Time loading and vectorization separately: feature extraction may dominate total cost. Optimize the measured bottleneck and batch transfers rather than assuming optimization arithmetic is the slow stage.
Source and licence: scikit-learn 1.7.2 — scaling strategies · BSD-3-Clause · answer: Quantoros editorial