New search direction formula of steepest descent method for large scale unconstrained optimization - Siti Farhana binti Husin
One of the simplest optimization methods for solving unconstrained optimization problems is the steepest descent (SD) method. The main ad antage of the SD algorithm with exact line search is that it satisfies global convergence properties under suitable assumptions. This method requires only the first derivative to be solved for the search direction which leads to low computational cost and storage requirements. However, the method has a slow convergence rate based on the higher number of iterations and central processing unit (CPU) time. Therefore, a new SD method that possesses lower iterations and CPU time is needed. This research concerns on the development of the SD method for solving large-scale nonlinear unconstrained optimization problems by suggesting new search direction in SD algorithm. This study focuses on the modification f search direction for SD methods by adding new parameters to the clas ical SD direction with the first suggestion is in two-term direction. Secondly, this work has also developed a three-term search direction for the SD method with two different parameters. The proposed method is specifically designed for solving large scale optimization problems under exact line search procedures. A new formulation of search direction for SD method combined with the conjugate gradient coefficients has been suggested. Twenty-six test problems are tested under different initial points ranging from two to five thousand variables. Numerical results for all of these methods are compared with existing SD methods based on the number of iterations and CPU time in which each method is evaluated over the same set of test problems and are interpreted by using the performance profile. The applicability of the introduced methods is shown by applying in least square method to solve some chosen nonlinear ordinary differential equations and to be implemented on data fitting through regression analysis. A set of data regarding relationship between fin length and total length of silky shark species has been chosen to construct a linear regression model. Theoretical proofs showed that all fthe proposed search directions fulfil sufficient descent conditions and the global c nvergence properties. Numerical results using performance profile indicate that all of the e methods give superior performance compared to the Classical SD (SDC), Zubai'ah, Mustafa, Rivaie and Ismail (ZMRI) and Rashidah, Rivaie, Mustafa (RRM) methods as they are able to lessen the nu : of iterations and CPU time. Results also show that all of these new methods are applicable 11 aily life problem and could produce useful regression equations. In conclusion, the numerical results for all the proposed methods are able to minimize the number of iterations and CPU time. Besides, the methods also have capabilities to be implemented in the least squar method and regression analysis for solving the nonlinear ordinary differential equations and real-life problems.