How can one cheaply check a huge analytic gradient without testing every coordinate?
MathematicsLevel: IntermediateDerivative and probability numerics
How can one cheaply check a huge analytic gradient without testing every coordinate?
Hints
- Differentiate along a line.
- Use multiple directions.
Answer
Compare gᵀv with a finite difference along several random directions v. This detects many errors cheaply but does not prove every coordinate correct.
Source and licence: SciPy — numerical differentiation · BSD-3-Clause · answer: Quantoros editorial