Stochastic diagonal approximate greatest descent in neural networks
© 2017 IEEE. Optimization is important in neural networks to iteratively update weights for pattern classification. Existing optimization techniques suffer from suboptimal local minima and slow convergence rate. In this paper, stochastic diagonal Approximate Greatest Descent (SDAGD) algorithm is pro...
| Main Authors: | , , |
|---|---|
| Format: | Conference Paper |
| Published: |
2017
|
| Online Access: | http://hdl.handle.net/20.500.11937/57712 |
| _version_ | 1848760077172867072 |
|---|---|
| author | Tan, H. Lim, Hann Harno, H. |
| author_facet | Tan, H. Lim, Hann Harno, H. |
| author_sort | Tan, H. |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | © 2017 IEEE. Optimization is important in neural networks to iteratively update weights for pattern classification. Existing optimization techniques suffer from suboptimal local minima and slow convergence rate. In this paper, stochastic diagonal Approximate Greatest Descent (SDAGD) algorithm is proposed to optimize neural network weights using multi-stage backpropagation manner. SDAGD is derived from the operation of a multi-stage decision control system. It uses the concept of control system consisting of: (1) when the local search region does not contain a minimum point, the iteration shall be defined at the boundary of the local search region, (2) when the local region contains a minimum point, the Newton method is used to search for the optimum solution. The implementation of SDAGD on Multilayer perceptron (MLP) is investigated with the goal of improving the learning ability and structural simplicity. Simulation results showed that two layer MLP with SDAGD achieved a misclassification rate of 4.7% on MNIST dataset. |
| first_indexed | 2025-11-14T10:10:02Z |
| format | Conference Paper |
| id | curtin-20.500.11937-57712 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T10:10:02Z |
| publishDate | 2017 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-577122017-11-20T08:58:25Z Stochastic diagonal approximate greatest descent in neural networks Tan, H. Lim, Hann Harno, H. © 2017 IEEE. Optimization is important in neural networks to iteratively update weights for pattern classification. Existing optimization techniques suffer from suboptimal local minima and slow convergence rate. In this paper, stochastic diagonal Approximate Greatest Descent (SDAGD) algorithm is proposed to optimize neural network weights using multi-stage backpropagation manner. SDAGD is derived from the operation of a multi-stage decision control system. It uses the concept of control system consisting of: (1) when the local search region does not contain a minimum point, the iteration shall be defined at the boundary of the local search region, (2) when the local region contains a minimum point, the Newton method is used to search for the optimum solution. The implementation of SDAGD on Multilayer perceptron (MLP) is investigated with the goal of improving the learning ability and structural simplicity. Simulation results showed that two layer MLP with SDAGD achieved a misclassification rate of 4.7% on MNIST dataset. 2017 Conference Paper http://hdl.handle.net/20.500.11937/57712 10.1109/IJCNN.2017.7966081 restricted |
| spellingShingle | Tan, H. Lim, Hann Harno, H. Stochastic diagonal approximate greatest descent in neural networks |
| title | Stochastic diagonal approximate greatest descent in neural networks |
| title_full | Stochastic diagonal approximate greatest descent in neural networks |
| title_fullStr | Stochastic diagonal approximate greatest descent in neural networks |
| title_full_unstemmed | Stochastic diagonal approximate greatest descent in neural networks |
| title_short | Stochastic diagonal approximate greatest descent in neural networks |
| title_sort | stochastic diagonal approximate greatest descent in neural networks |
| url | http://hdl.handle.net/20.500.11937/57712 |