Skip to content
Quantoros

Why does a zero-probability category contribute zero to discrete entropy rather than NaN?

MathematicsLevel: IntermediateInformation measures

Why does a zero-probability category contribute zero to discrete entropy rather than NaN?

Hints
  • Evaluate a limit.
  • Direct floating-point arithmetic can differ.

Answer

The convention follows lim(p→0+) p log p=0. Implementations should handle this limit explicitly rather than directly multiplying zero by negative infinity.

Source and licence: SciPy — entropy and relative entropy · BSD-3-Clause · answer: Quantoros editorial

Related questions