Improved stochastic gradient descent algorithm with mean-gradient adaptive stepsize for solving large-scale optimization problems

QR Code

Improved stochastic gradient descent algorithm with mean-gradient adaptive stepsize for solving large-scale optimization problems

Stochastic gradient descent (SGD) is one of the most common algorithms used in solving large unconstrained optimization problems. It utilizes the concept of classical gradient descent method with modification on the gradient selection. SGD uses random or batch data sets to compute gradient in solvin...

Full description

Bibliographic Details
Main Authors:	Zulkifli, Munierah, Abd Rahmin, Nor Aliza, Wah, June Leong
Format:	Article
Language:	English
Published:	Persatuan Sains Matematik Malaysia 2023
Online Access:	http://psasir.upm.edu.my/id/eprint/110372/ http://psasir.upm.edu.my/id/eprint/110372/1/document%20%284%29.pdf

Similar Items

A new stepsize for gradient method via weak secant equation
by: Leong, Wah June
Published: (2007)

Sufficient descent three term conjugate gradient method via symmetric rank-one update for large-scale optimization
by: Moyi, Aliyu Usman, et al.
Published: (2016)

An improvement of stochastic gradient descent approach for mean-variance portfolio optimization problem
by: S. W. Su, Stephanie, et al.
Published: (2021)

Conjugate gradient methods with sufficient descent condition for large-scale unconstrained optimization
by: Ling, Mei Mei
Published: (2015)

An improved multi-step gradient-type method for large scale optimization
by: Farid, Mahboubeh, et al.
Published: (2011)

Gradient method with multiple damping for large-scale unconstrained optimization
by: Sim, Hong Seng, et al.
Published: (2019)

A Mini-Batch Proximal Stochastic Recursive Gradient Algorithm with Diagonal Barzilai–Borwein Stepsize
by: Yu, T.T., et al.
Published: (2022)

A new gradient method via quasi-Cauchy relation which guarantees descent
by: Abu Hassan, Malik, et al.
Published: (2009)

A new diagonal gradient-type method for large scale unconstrained optimization
by: Farid, Mahboubeh, et al.
Published: (2013)

Scaled diagonal gradient-type method with extra update for large-scale unconstrained optimization
by: Farid, Mahboubeh, et al.
Published: (2013)

A new two-step gradient-type method for large-scale unconstrained optimization
by: Farid, Mahboubeh, et al.
Published: (2010)

Scaled memoryless BFGS preconditioned steepest descent method for very large-scale unconstrained optimization
by: Leong, Wah June, et al.
Published: (2009)

The sufficient descent condition of nonlinear conjugate gradient method
by: Basri, Sri Mazzura, et al.
Published: (2018)

A Minibatch Proximal Stochastic Recursive Gradient Algorithm Using a Trust-Region-Like Scheme and Barzilai-Borwein Stepsizes
by: Yu, T., et al.
Published: (2021)

Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization
by: Hong, Seng Sim, et al.
Published: (2018)

Nonmonotone spectral gradient method based on memoryless symmetric rank-one update for large-scale unconstrained optimization
by: Hong, Seng Sim, et al.
Published: (2021)

A scaled three-term conjugate gradient method for unconstrained optimization
by: Arzuka, Ibrahim, et al.
Published: (2016)

Hybridization of gradient descent algorithms with dynamic tunneling methods for global optimization
by: RoyChowdhury, Pinaki, et al.
Published: (2000)

Conjugate gradient and steepest descent approach on quasi-Newton search direction

Multi-spectral gradient method via variational technique under log-determinant norm for large-scale optimization
by: Hong, Seng Sim, et al.
Published: (2017)

The proof of sufficient descent condition for a new type of conjugate gradient methods

A new type of descent conjugate gradient method with exact line search

A conjugate gradient method with descent properties under strong Wolfe line search

New class of hybrid conjugate gradient coefficients with guaranteed descent and efficient line search

2020_New Modification of Conjugate Gradient Method and Its Hybridization to Solve Large-Scale Unconstrained Optimization Problems

On the long-time integration of stochastic gradient systems
by: Leimkuhler, B., et al.
Published: (2014)

Diagonal preconditioned conjugate gradient algorithm for unconstrained optimization
by: Ng, Choong Boon, et al.
Published: (2014)

A new gradient method via least change secant update
by: Leong, Wah June, et al.
Published: (2011)

A new edition of conjugate gradient methods for large-scale unconstrained optimization

Mental Stress Classification Among Higher Education Students In Malaysia From Electroencephalogram (Eeg) Using Convolutional Neural Network With Modified Stochastic Gradient Descent.
by: Rashid, Nur Ramizah Ramino
Published: (2024)

A scalar modification of three-term PRP-DL conjugate gradient method for solving large-scaled unconstrained optimization problems
by: Marjugi, Siti Mahani, et al.
Published: (2024)

A gradient descent boosting spectrum modeling method based on back interval partial least squares
by: Ren, D., et al.
Published: (2015)

2015_New Family of Conjugate Gradient Methods with Sufficient Descent Condition and Global Convergence For Unconstrained Optimizations

A three-term conjugate gradient method with nonmonotone line search for unconstrained optimization
by: Moyi, Aliyu Usman, et al.
Published: (2016)

Solving a large scale nonlinear unconstrained optimization with exact line search direction by using new coefficient of conjugate gradient methods

A monotone gradient method via weak secant equation for unconstrained optimization
by: Leong, Wah June, et al.
Published: (2010)

Barzilai-Borwein gradient method for solving fuzzy nonlinear
by: Mohammed Sulaiman, Ibrahim, et al.
Published: (2018)

Improved Hessian approximation with modified quasi-Cauchy relation for a gradient-type method
by: Leong, Wah June, et al.
Published: (2010)

A matrix-free quasi-Newton method for solving large-scale nonlinear systems.
by: Leong, Wah June, et al.
Published: (2011)

Variable order variable stepsize algorithm for solving nonlinear Duffing oscillator
by: Rasedee, Ahmad Fadly Nurullah, et al.
Published: (2017)