Skip to content
Quantoros

What is the maximum entropy of four categories in bits, and when is it attained?

MathematicsLevel: FoundationInformation measures

What is the maximum entropy of four categories in bits, and when is it attained?

Hints
  • Use log2(number of categories).
  • Maximum uncertainty is uniform.

Answer

Two bits, at uniform probabilities 1/4. Concentrating probability on fewer categories lowers entropy.

Source and licence: SciPy — entropy and relative entropy · BSD-3-Clause · answer: Quantoros editorial

Related questions