Why is summing a multiclass SHAP tensor over both features and classes usually the wrong validation check?
Classic MLLevel: IntermediateSHAP · TreeExplainer
Why is summing a multiclass SHAP tensor over both features and classes usually the wrong validation check?
Hints
- Each output has its own decomposition.
- Tensor axes encode different meanings.
Answer
Additivity holds separately for each output/class and its baseline. Preserve the output axis and sum only feature contributions for that output; adding unrelated class outputs destroys the intended equality.
Source and licence: SHAP · TreeExplainer · MIT · answer: Quantoros editorial