Stochastic diagonal approximate greatest descent in convolutional neural networks
© 2017 IEEE. Deep structured of Convolutional Neural Networks (CNN) has recently gained intense attention in development due to its good performance in object recognition. One of the crucial components in CNN is the learning mechanism of weight parameters through backpropagation. In this paper, stoc...
| Main Authors: | , , |
|---|---|
| Format: | Conference Paper |
| Published: |
2017
|
| Online Access: | http://hdl.handle.net/20.500.11937/65863 |
| _version_ | 1848761220368171008 |
|---|---|
| author | Tan, H. Lim, Hann Harno, H. |
| author_facet | Tan, H. Lim, Hann Harno, H. |
| author_sort | Tan, H. |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | © 2017 IEEE. Deep structured of Convolutional Neural Networks (CNN) has recently gained intense attention in development due to its good performance in object recognition. One of the crucial components in CNN is the learning mechanism of weight parameters through backpropagation. In this paper, stochastic diagonal Approximate Greatest Descent (SDAGD) is proposed to train weight parameters in CNN. SDAGD adopts the concept of multistage control system and diagonal Hessian approximation for weight optimization. It can be defined into two-phase optimization. In phase 1, when an initial guessing point is far from the solution, SDAGD constructs local search regions to determine the step length of next iteration at the boundary of search region. Subsequently, when the solution is at the final search region, SDAGD will shift to phase 2 by approximating Newton method to obtain a fast weight convergence. The calculation of Hessian in diagonal approximation results in less computational cost as compared to full Hessian calculation. The experiment showed that SDAGD learning algorithm could achieve misclassification rate of 8.85% on MNIST dataset. |
| first_indexed | 2025-11-14T10:28:12Z |
| format | Conference Paper |
| id | curtin-20.500.11937-65863 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T10:28:12Z |
| publishDate | 2017 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-658632018-02-19T08:06:52Z Stochastic diagonal approximate greatest descent in convolutional neural networks Tan, H. Lim, Hann Harno, H. © 2017 IEEE. Deep structured of Convolutional Neural Networks (CNN) has recently gained intense attention in development due to its good performance in object recognition. One of the crucial components in CNN is the learning mechanism of weight parameters through backpropagation. In this paper, stochastic diagonal Approximate Greatest Descent (SDAGD) is proposed to train weight parameters in CNN. SDAGD adopts the concept of multistage control system and diagonal Hessian approximation for weight optimization. It can be defined into two-phase optimization. In phase 1, when an initial guessing point is far from the solution, SDAGD constructs local search regions to determine the step length of next iteration at the boundary of search region. Subsequently, when the solution is at the final search region, SDAGD will shift to phase 2 by approximating Newton method to obtain a fast weight convergence. The calculation of Hessian in diagonal approximation results in less computational cost as compared to full Hessian calculation. The experiment showed that SDAGD learning algorithm could achieve misclassification rate of 8.85% on MNIST dataset. 2017 Conference Paper http://hdl.handle.net/20.500.11937/65863 10.1109/ICSIPA.2017.8120653 restricted |
| spellingShingle | Tan, H. Lim, Hann Harno, H. Stochastic diagonal approximate greatest descent in convolutional neural networks |
| title | Stochastic diagonal approximate greatest descent in convolutional neural networks |
| title_full | Stochastic diagonal approximate greatest descent in convolutional neural networks |
| title_fullStr | Stochastic diagonal approximate greatest descent in convolutional neural networks |
| title_full_unstemmed | Stochastic diagonal approximate greatest descent in convolutional neural networks |
| title_short | Stochastic diagonal approximate greatest descent in convolutional neural networks |
| title_sort | stochastic diagonal approximate greatest descent in convolutional neural networks |
| url | http://hdl.handle.net/20.500.11937/65863 |