A class of diagonal preconditioners for limited memory BFGS method.
A major weakness of the limited memory BFGS (LBFGS) method is that it may converge very slowly on ill-conditioned problems when the identity matrix is used for initialization. Very often, the LBFGS method can adopt a preconditioner on the identity matrix to speed up the convergence. For this purpo...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English English |
| Published: |
Taylor & Francis
2013
|
| Online Access: | http://psasir.upm.edu.my/id/eprint/29993/ http://psasir.upm.edu.my/id/eprint/29993/1/A%20class%20of%20diagonal%20preconditioners%20for%20limited%20memory%20BFGS%20method.pdf |
| _version_ | 1848846552917868544 |
|---|---|
| author | Leong, Wah June Chen, Chuei Yee |
| author_facet | Leong, Wah June Chen, Chuei Yee |
| author_sort | Leong, Wah June |
| building | UPM Institutional Repository |
| collection | Online Access |
| description | A major weakness of the limited memory BFGS (LBFGS) method is that it may converge very slowly on
ill-conditioned problems when the identity matrix is used for initialization. Very often, the LBFGS method
can adopt a preconditioner on the identity matrix to speed up the convergence. For this purpose, we propose
a class of diagonal preconditioners to boost the performance of the LBFGS method. In this context, we find
that it is appropriate to use a diagonal preconditioner, in the form of a diagonal matrix plus a positive multiple
of the identity matrix, so as to fit information of local Hessian as well as to induce positive definiteness for
the diagonal preconditioner at a whole. The property of hereditary positive definiteness is maintained by
a careful choice of the positive scalar on the scaled identity matrix while the local curvature information
is carried implicitly on the other diagonal matrix through the variational techniques, commonly employed
in the derivation of quasi-Newton updates. Several preconditioning formulae are then derived and tested
on a large set of standard test problems to access the impact of different choices of such preconditioners
on the minimization performance. |
| first_indexed | 2025-11-15T09:04:32Z |
| format | Article |
| id | upm-29993 |
| institution | Universiti Putra Malaysia |
| institution_category | Local University |
| language | English English |
| last_indexed | 2025-11-15T09:04:32Z |
| publishDate | 2013 |
| publisher | Taylor & Francis |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | upm-299932015-10-01T23:38:32Z http://psasir.upm.edu.my/id/eprint/29993/ A class of diagonal preconditioners for limited memory BFGS method. Leong, Wah June Chen, Chuei Yee A major weakness of the limited memory BFGS (LBFGS) method is that it may converge very slowly on ill-conditioned problems when the identity matrix is used for initialization. Very often, the LBFGS method can adopt a preconditioner on the identity matrix to speed up the convergence. For this purpose, we propose a class of diagonal preconditioners to boost the performance of the LBFGS method. In this context, we find that it is appropriate to use a diagonal preconditioner, in the form of a diagonal matrix plus a positive multiple of the identity matrix, so as to fit information of local Hessian as well as to induce positive definiteness for the diagonal preconditioner at a whole. The property of hereditary positive definiteness is maintained by a careful choice of the positive scalar on the scaled identity matrix while the local curvature information is carried implicitly on the other diagonal matrix through the variational techniques, commonly employed in the derivation of quasi-Newton updates. Several preconditioning formulae are then derived and tested on a large set of standard test problems to access the impact of different choices of such preconditioners on the minimization performance. Taylor & Francis 2013 Article PeerReviewed application/pdf en http://psasir.upm.edu.my/id/eprint/29993/1/A%20class%20of%20diagonal%20preconditioners%20for%20limited%20memory%20BFGS%20method.pdf Leong, Wah June and Chen, Chuei Yee (2013) A class of diagonal preconditioners for limited memory BFGS method. Optimization Methods and Software, 28 (2). pp. 379-392. ISSN 1055-6788; ESSN: 1029-4937 10.1080/10556788.2011.653356 English |
| spellingShingle | Leong, Wah June Chen, Chuei Yee A class of diagonal preconditioners for limited memory BFGS method. |
| title | A class of diagonal preconditioners for limited memory BFGS method. |
| title_full | A class of diagonal preconditioners for limited memory BFGS method. |
| title_fullStr | A class of diagonal preconditioners for limited memory BFGS method. |
| title_full_unstemmed | A class of diagonal preconditioners for limited memory BFGS method. |
| title_short | A class of diagonal preconditioners for limited memory BFGS method. |
| title_sort | class of diagonal preconditioners for limited memory bfgs method. |
| url | http://psasir.upm.edu.my/id/eprint/29993/ http://psasir.upm.edu.my/id/eprint/29993/ http://psasir.upm.edu.my/id/eprint/29993/1/A%20class%20of%20diagonal%20preconditioners%20for%20limited%20memory%20BFGS%20method.pdf |