Why might inplacepredict reduce latency for small XGBoost requests without changing the model?
Classic MLLevel: IntermediateXGBoost · Prediction
Why might inplace_predict reduce latency for small XGBoost requests without changing the model?
Hints
- Separate input preparation from tree evaluation.
- Avoiding a wrapper allocation can matter for tiny batches.
Answer
It can avoid constructing a DMatrix for ordinary inference, reducing allocation and conversion overhead. Benchmark end-to-end latency with supported inputs and required output mode; the optimization does not guarantee faster tree traversal itself.
Source and licence: XGBoost · Prediction · Apache-2.0 · answer: Quantoros editorial