What is the maximum entropy of four categories in bits, and when is it attained?
MathematicsLevel: FoundationInformation measures
What is the maximum entropy of four categories in bits, and when is it attained?
Hints
- Use log2(number of categories).
- Maximum uncertainty is uniform.
Answer
Two bits, at uniform probabilities 1/4. Concentrating probability on fewer categories lowers entropy.
Source and licence: SciPy — entropy and relative entropy · BSD-3-Clause · answer: Quantoros editorial