A class of diagonal preconditioners for limited memory BFGS method.
A major weakness of the limited memory BFGS (LBFGS) method is that it may converge very slowly on ill-conditioned problems when the identity matrix is used for initialization. Very often, the LBFGS method can adopt a preconditioner on the identity matrix to speed up the convergence. For this purpo...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English English |
| Published: |
Taylor & Francis
2013
|
| Online Access: | http://psasir.upm.edu.my/id/eprint/29993/ http://psasir.upm.edu.my/id/eprint/29993/1/A%20class%20of%20diagonal%20preconditioners%20for%20limited%20memory%20BFGS%20method.pdf |
| Summary: | A major weakness of the limited memory BFGS (LBFGS) method is that it may converge very slowly on
ill-conditioned problems when the identity matrix is used for initialization. Very often, the LBFGS method
can adopt a preconditioner on the identity matrix to speed up the convergence. For this purpose, we propose
a class of diagonal preconditioners to boost the performance of the LBFGS method. In this context, we find
that it is appropriate to use a diagonal preconditioner, in the form of a diagonal matrix plus a positive multiple
of the identity matrix, so as to fit information of local Hessian as well as to induce positive definiteness for
the diagonal preconditioner at a whole. The property of hereditary positive definiteness is maintained by
a careful choice of the positive scalar on the scaled identity matrix while the local curvature information
is carried implicitly on the other diagonal matrix through the variational techniques, commonly employed
in the derivation of quasi-Newton updates. Several preconditioning formulae are then derived and tested
on a large set of standard test problems to access the impact of different choices of such preconditioners
on the minimization performance. |
|---|