Skip to content
Quantoros

One parameter is around 10⁻⁶ and another around 10⁶. Why can rescaling help optimization?

MathematicsLevel: IntermediateMultivariate optimizer reasoning

One parameter is around 10⁻⁶ and another around 10⁶. Why can rescaling help optimization?

Hints
  • Step sizes interact with units.
  • Transform constraints too.

Answer

Comparable parameter scales improve numerical differencing and step selection. Transform the objective and constraints consistently, then map the solution back.

Source and licence: SciPy — scalar and multivariate minimization · BSD-3-Clause · answer: Quantoros editorial

Related questions