What shape should the gradient of a scalar objective on n parameters have?
MathematicsLevel: IntermediateMultivariate optimizer reasoning
What shape should the gradient of a scalar objective on n parameters have?
Hints
- One derivative per parameter.
- Separate first and second derivatives.
Answer
A length-n vector. A full n×n matrix is a Hessian, while a residual-vector Jacobian has a different role.
Source and licence: SciPy — scalar and multivariate minimization · BSD-3-Clause · answer: Quantoros editorial