Skip to content
Quantoros

Why can stochastic objective evaluations ruin an otherwise sensible finite-difference gradient check?

MathematicsLevel: IntermediateDerivative and probability numerics

Why can stochastic objective evaluations ruin an otherwise sensible finite-difference gradient check?

Hints
  • Noise is amplified by 1/h.
  • Compare like-for-like evaluations.

Answer

Evaluation noise is divided by the small step and can dominate the signal. Freeze random draws or use controlled common randomness during checking.

Source and licence: SciPy — numerical differentiation · BSD-3-Clause · answer: Quantoros editorial

Related questions