Skip to content
Quantoros

An analytic gradient disagrees with finite differences. What should be checked before blaming the optimizer?

MathematicsLevel: IntermediateMultivariate optimizer reasoning

An analytic gradient disagrees with finite differences. What should be checked before blaming the optimizer?

Hints
  • Both methods must differentiate the same objective.
  • Vary step size.

Answer

Check sign, shape, parameter ordering, objective normalization and deterministic evaluation. Then vary finite-difference step to distinguish formula mistakes from numerical noise.

Source and licence: SciPy — scalar and multivariate minimization · BSD-3-Clause · answer: Quantoros editorial

Related questions