Why does a zero-probability category contribute zero to discrete entropy rather than NaN?
MathematicsLevel: IntermediateInformation measures
Why does a zero-probability category contribute zero to discrete entropy rather than NaN?
Hints
- Evaluate a limit.
- Direct floating-point arithmetic can differ.
Answer
The convention follows lim(p→0+) p log p=0. Implementations should handle this limit explicitly rather than directly multiplying zero by negative infinity.
Source and licence: SciPy — entropy and relative entropy · BSD-3-Clause · answer: Quantoros editorial