Skip to content
Quantoros

Why can converting sparse data to a dense matrix change XGBoost predictions?

Classic MLLevel: IntermediateSparse zeros versus missing values

Why can converting sparse data to a dense matrix change XGBoost predictions?

Hints
  • What does the next learner predict relative to the current ensemble?
  • Connect sequential model additions to loss optimization.

Answer

For tree boosters, absent entries in a sparse matrix are treated as missing; a stored numerical zero in a dense matrix is normally a valid value. Densifying a sparse matrix often fills absent entries with zeros, changing their semantics and potentially their tree paths. The linear booster handles these entries differently, as zeros. Preserve an explicit missing-value policy and test dense/sparse parity instead of assuming that the storage formats represent identical inputs to every booster.

Source and licence: XGBoost contributors — Official FAQ · Apache-2.0 · answer: Quantoros editorial

Related questions