Radial effect in stochastic diagonal approximate greatest descent
© 2017 IEEE. Stochastic Diagonal Approximate Greatest Descent (SDAGD) is proposed to manage the optimization in two stages, (a) apply a radial boundary to estimate step length when the weights are far from solution, (b) apply Newton method when the weights are within the solution level set. This is...
| Main Authors: | , , |
|---|---|
| Format: | Conference Paper |
| Published: |
2017
|
| Online Access: | http://hdl.handle.net/20.500.11937/65903 |
| _version_ | 1848761229114343424 |
|---|---|
| author | Tan, H. Lim, Hann Harno, H. |
| author_facet | Tan, H. Lim, Hann Harno, H. |
| author_sort | Tan, H. |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | © 2017 IEEE. Stochastic Diagonal Approximate Greatest Descent (SDAGD) is proposed to manage the optimization in two stages, (a) apply a radial boundary to estimate step length when the weights are far from solution, (b) apply Newton method when the weights are within the solution level set. This is inspired by a multi-stage decision control system where different strategies is used at different conditions. In numerical optimization context, larger steps should be taken at the beginning of optimization and gradually reduced when it is near to the minimum point. Nevertheless, the intuition of determining the radial boundary when the optimized parameters are far from the solution is yet to be investigated for high dimensional data. Radial step length in SDAGD manipulates the relative step length for iteration construction. SDAGD is implemented in a two layer Multilayer Perceptron to evaluate the effects of R on artificial neural networks. It is concluded that the greater the value of R, the higher the learning rate of SDAGD algorithm when the value of R is constrained in between 100 to 10,000. |
| first_indexed | 2025-11-14T10:28:21Z |
| format | Conference Paper |
| id | curtin-20.500.11937-65903 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T10:28:21Z |
| publishDate | 2017 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-659032018-02-19T08:06:06Z Radial effect in stochastic diagonal approximate greatest descent Tan, H. Lim, Hann Harno, H. © 2017 IEEE. Stochastic Diagonal Approximate Greatest Descent (SDAGD) is proposed to manage the optimization in two stages, (a) apply a radial boundary to estimate step length when the weights are far from solution, (b) apply Newton method when the weights are within the solution level set. This is inspired by a multi-stage decision control system where different strategies is used at different conditions. In numerical optimization context, larger steps should be taken at the beginning of optimization and gradually reduced when it is near to the minimum point. Nevertheless, the intuition of determining the radial boundary when the optimized parameters are far from the solution is yet to be investigated for high dimensional data. Radial step length in SDAGD manipulates the relative step length for iteration construction. SDAGD is implemented in a two layer Multilayer Perceptron to evaluate the effects of R on artificial neural networks. It is concluded that the greater the value of R, the higher the learning rate of SDAGD algorithm when the value of R is constrained in between 100 to 10,000. 2017 Conference Paper http://hdl.handle.net/20.500.11937/65903 10.1109/ICSIPA.2017.8120611 restricted |
| spellingShingle | Tan, H. Lim, Hann Harno, H. Radial effect in stochastic diagonal approximate greatest descent |
| title | Radial effect in stochastic diagonal approximate greatest descent |
| title_full | Radial effect in stochastic diagonal approximate greatest descent |
| title_fullStr | Radial effect in stochastic diagonal approximate greatest descent |
| title_full_unstemmed | Radial effect in stochastic diagonal approximate greatest descent |
| title_short | Radial effect in stochastic diagonal approximate greatest descent |
| title_sort | radial effect in stochastic diagonal approximate greatest descent |
| url | http://hdl.handle.net/20.500.11937/65903 |