Modified quasi-Newton type methods using gradient flow system for solving unconstrained optimization
In this thesis, we are mainly concerned with finding the numerical solution of nonlinear unconstrained optimization problems via gradient ow system. First, we give some brief mathematical background and then we consider a famous class of optimization methods called the quasi-Newton methods. Specific...
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
2016
|
| Subjects: | |
| Online Access: | http://psasir.upm.edu.my/id/eprint/85082/ http://psasir.upm.edu.my/id/eprint/85082/1/FS%202016%2092%20IR.pdf |
| _version_ | 1848859987673088000 |
|---|---|
| author | Yap, Chui Ying |
| author_facet | Yap, Chui Ying |
| author_sort | Yap, Chui Ying |
| building | UPM Institutional Repository |
| collection | Online Access |
| description | In this thesis, we are mainly concerned with finding the numerical solution of nonlinear unconstrained optimization problems via gradient ow system. First, we give some brief mathematical background and then we consider a famous class of optimization methods called the quasi-Newton methods. Specifically, we focus on a class of quasi-Newton method named Broyden-Fletcher-Goldfarb-Shanno (BFGS) method. We investigate the possible use of control theory, particularly theory on gradient ow system to derive some modified line search and trust region methods for optimization. The implementation of these methods in line search algorithm in their original forms would generate a Newton-type matrix which require inversion of a non-sparse matrix or equivalently solving a linear system in every iteration. Thus, an approximation of the proposed methods via BFGS update is constructed. Numerical experiments are carried out to illustrate the numerical performance and efficiency of the proposed methods by comparing the number of iterations, the number of function evaluations and also the CPU time in second. Our computational results show that the proposed methods are comparable with the existing standard methods. Other than that, we also analyse the global convergence properties of the modified methods. It is shown that the modified methods converge globally and the rate of convergence is superlinear convergence. We also implement the Newton-type methods on trust region framework by using unit step length to adjust the radius of the region to obtain desired reduction in the objective function. We make an approximation to the proposed Newton-type matrix by using BFGS updating scheme and then apply this modified Newtontype matrix to generate new quadratic approximation subproblem. Numerical results are established to demonstrate the efficiency of our modified methods. Our proposed methods outperform the standard trust region method in term of lower number of function evaluations and much reduction in computational time. It is proved under appropriate assumptions that the modified trust region methods are globally convergent. |
| first_indexed | 2025-11-15T12:38:04Z |
| format | Thesis |
| id | upm-85082 |
| institution | Universiti Putra Malaysia |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-15T12:38:04Z |
| publishDate | 2016 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | upm-850822021-09-13T04:10:32Z http://psasir.upm.edu.my/id/eprint/85082/ Modified quasi-Newton type methods using gradient flow system for solving unconstrained optimization Yap, Chui Ying In this thesis, we are mainly concerned with finding the numerical solution of nonlinear unconstrained optimization problems via gradient ow system. First, we give some brief mathematical background and then we consider a famous class of optimization methods called the quasi-Newton methods. Specifically, we focus on a class of quasi-Newton method named Broyden-Fletcher-Goldfarb-Shanno (BFGS) method. We investigate the possible use of control theory, particularly theory on gradient ow system to derive some modified line search and trust region methods for optimization. The implementation of these methods in line search algorithm in their original forms would generate a Newton-type matrix which require inversion of a non-sparse matrix or equivalently solving a linear system in every iteration. Thus, an approximation of the proposed methods via BFGS update is constructed. Numerical experiments are carried out to illustrate the numerical performance and efficiency of the proposed methods by comparing the number of iterations, the number of function evaluations and also the CPU time in second. Our computational results show that the proposed methods are comparable with the existing standard methods. Other than that, we also analyse the global convergence properties of the modified methods. It is shown that the modified methods converge globally and the rate of convergence is superlinear convergence. We also implement the Newton-type methods on trust region framework by using unit step length to adjust the radius of the region to obtain desired reduction in the objective function. We make an approximation to the proposed Newton-type matrix by using BFGS updating scheme and then apply this modified Newtontype matrix to generate new quadratic approximation subproblem. Numerical results are established to demonstrate the efficiency of our modified methods. Our proposed methods outperform the standard trust region method in term of lower number of function evaluations and much reduction in computational time. It is proved under appropriate assumptions that the modified trust region methods are globally convergent. 2016-06 Thesis NonPeerReviewed text en http://psasir.upm.edu.my/id/eprint/85082/1/FS%202016%2092%20IR.pdf Yap, Chui Ying (2016) Modified quasi-Newton type methods using gradient flow system for solving unconstrained optimization. Masters thesis, Universiti Putra Malaysia. Conjugate gradient methods Mathematical models |
| spellingShingle | Conjugate gradient methods Mathematical models Yap, Chui Ying Modified quasi-Newton type methods using gradient flow system for solving unconstrained optimization |
| title | Modified quasi-Newton type methods using gradient flow system for solving unconstrained optimization |
| title_full | Modified quasi-Newton type methods using gradient flow system for solving unconstrained optimization |
| title_fullStr | Modified quasi-Newton type methods using gradient flow system for solving unconstrained optimization |
| title_full_unstemmed | Modified quasi-Newton type methods using gradient flow system for solving unconstrained optimization |
| title_short | Modified quasi-Newton type methods using gradient flow system for solving unconstrained optimization |
| title_sort | modified quasi-newton type methods using gradient flow system for solving unconstrained optimization |
| topic | Conjugate gradient methods Mathematical models |
| url | http://psasir.upm.edu.my/id/eprint/85082/ http://psasir.upm.edu.my/id/eprint/85082/1/FS%202016%2092%20IR.pdf |