Adaptive Second-order Derivative Approximate Greatest Descent Optimization for Deep Learning Neural Networks
Backpropagation using Stochastic Diagonal Approximate Greatest Descent (SDAGD) is a novel adaptive second-order derivative optimization method in updating weights of deep learning neural networks. SDAGD applies two-phase switching strategy to seek for solution at far using long-term optimal trajecto...
| Main Author: | |
|---|---|
| Format: | Thesis |
| Published: |
Curtin University
2019
|
| Online Access: | http://hdl.handle.net/20.500.11937/77991 |
| _version_ | 1848763926570860544 |
|---|---|
| author | Tan, Hong Hui |
| author_facet | Tan, Hong Hui |
| author_sort | Tan, Hong Hui |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | Backpropagation using Stochastic Diagonal Approximate Greatest Descent (SDAGD) is a novel adaptive second-order derivative optimization method in updating weights of deep learning neural networks. SDAGD applies two-phase switching strategy to seek for solution at far using long-term optimal trajectory and automatically switch to Newton method when nearer to optimal solution. SDAGD has the advantages of steepest training roll-off rate, adaptive adjustment of step-length and the ability to deal with vanishing gradient issues in deep architecture. |
| first_indexed | 2025-11-14T11:11:13Z |
| format | Thesis |
| id | curtin-20.500.11937-77991 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T11:11:13Z |
| publishDate | 2019 |
| publisher | Curtin University |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-779912020-03-04T00:59:57Z Adaptive Second-order Derivative Approximate Greatest Descent Optimization for Deep Learning Neural Networks Tan, Hong Hui Backpropagation using Stochastic Diagonal Approximate Greatest Descent (SDAGD) is a novel adaptive second-order derivative optimization method in updating weights of deep learning neural networks. SDAGD applies two-phase switching strategy to seek for solution at far using long-term optimal trajectory and automatically switch to Newton method when nearer to optimal solution. SDAGD has the advantages of steepest training roll-off rate, adaptive adjustment of step-length and the ability to deal with vanishing gradient issues in deep architecture. 2019 Thesis http://hdl.handle.net/20.500.11937/77991 Curtin University fulltext |
| spellingShingle | Tan, Hong Hui Adaptive Second-order Derivative Approximate Greatest Descent Optimization for Deep Learning Neural Networks |
| title | Adaptive Second-order Derivative Approximate Greatest Descent Optimization for Deep Learning Neural Networks |
| title_full | Adaptive Second-order Derivative Approximate Greatest Descent Optimization for Deep Learning Neural Networks |
| title_fullStr | Adaptive Second-order Derivative Approximate Greatest Descent Optimization for Deep Learning Neural Networks |
| title_full_unstemmed | Adaptive Second-order Derivative Approximate Greatest Descent Optimization for Deep Learning Neural Networks |
| title_short | Adaptive Second-order Derivative Approximate Greatest Descent Optimization for Deep Learning Neural Networks |
| title_sort | adaptive second-order derivative approximate greatest descent optimization for deep learning neural networks |
| url | http://hdl.handle.net/20.500.11937/77991 |