Skip to content
Quantoros

Why can reversing the arguments of KL produce a different result?

MathematicsLevel: IntermediateInformation measures

Why can reversing the arguments of KL produce a different result?

Hints
  • Identify the weighting distribution.
  • Distances need symmetry.

Answer

The expectation is weighted by the first distribution and penalizes missing support in the second. KL is asymmetric, so it is not a metric.

Source and licence: SciPy — entropy and relative entropy · BSD-3-Clause · answer: Quantoros editorial

Related questions