One parameter is around 10⁻⁶ and another around 10⁶. Why can rescaling help optimization?
MathematicsLevel: IntermediateMultivariate optimizer reasoning
One parameter is around 10⁻⁶ and another around 10⁶. Why can rescaling help optimization?
Hints
- Step sizes interact with units.
- Transform constraints too.
Answer
Comparable parameter scales improve numerical differencing and step selection. Transform the objective and constraints consistently, then map the solution back.
Source and licence: SciPy — scalar and multivariate minimization · BSD-3-Clause · answer: Quantoros editorial