Why can the maximum logit be subtracted before softmax without changing probabilities?
MathematicsLevel: IntermediateDerivative and probability numerics
Why can the maximum logit be subtracted before softmax without changing probabilities?
Hints
- Factor exp(−c) everywhere.
- Only relative logits matter.
Answer
The common exponential factor cancels between numerator and denominator. Subtracting the maximum avoids large exponentials while preserving relative logits.
Source and licence: SciPy — logsumexp and softmax numerics · BSD-3-Clause · answer: Quantoros editorial