Why use limited-memory BFGS when a full quasi-Newton approximation is too large?
MathematicsLevel: IntermediateMultivariate optimizer reasoning
Why use limited-memory BFGS when a full quasi-Newton approximation is too large?
Hints
- Full curvature storage scales quadratically.
- History vectors are cheaper.
Answer
It retains a limited history of update vectors instead of a dense inverse-Hessian approximation. Memory falls at the cost of less accumulated curvature information.
Source and licence: SciPy — scalar and multivariate minimization · BSD-3-Clause · answer: Quantoros editorial