Skip to content
Quantoros

Why can the maximum logit be subtracted before softmax without changing probabilities?

MathematicsLevel: IntermediateDerivative and probability numerics

Why can the maximum logit be subtracted before softmax without changing probabilities?

Hints
  • Factor exp(−c) everywhere.
  • Only relative logits matter.

Answer

The common exponential factor cancels between numerator and denominator. Subtracting the maximum avoids large exponentials while preserving relative logits.

Source and licence: SciPy — logsumexp and softmax numerics · BSD-3-Clause · answer: Quantoros editorial

Related questions