Why can converting sparse data to a dense matrix change XGBoost predictions?
Classic MLLevel: IntermediateSparse zeros versus missing values
Why can converting sparse data to a dense matrix change XGBoost predictions?
Hints
- What does the next learner predict relative to the current ensemble?
- Connect sequential model additions to loss optimization.
Answer
For tree boosters, absent entries in a sparse matrix are treated as missing; a stored numerical zero in a dense matrix is normally a valid value. Densifying a sparse matrix often fills absent entries with zeros, changing their semantics and potentially their tree paths. The linear booster handles these entries differently, as zeros. Preserve an explicit missing-value policy and test dense/sparse parity instead of assuming that the storage formats represent identical inputs to every booster.
Source and licence: XGBoost contributors — Official FAQ · Apache-2.0 · answer: Quantoros editorial