Improved stochastic gradient descent algorithm with mean-gradient adaptive stepsize for solving large-scale optimization problems

QR Code

Improved stochastic gradient descent algorithm with mean-gradient adaptive stepsize for solving large-scale optimization problems

Stochastic gradient descent (SGD) is one of the most common algorithms used in solving large unconstrained optimization problems. It utilizes the concept of classical gradient descent method with modification on the gradient selection. SGD uses random or batch data sets to compute gradient in solvin...

Full description

Bibliographic Details
Main Authors:	Zulkifli, Munierah, Abd Rahmin, Nor Aliza, Wah, June Leong
Format:	Article
Language:	English
Published:	Persatuan Sains Matematik Malaysia 2023
Online Access:	http://psasir.upm.edu.my/id/eprint/110372/ http://psasir.upm.edu.my/id/eprint/110372/1/document%20%284%29.pdf

Similar Items

A new stepsize for gradient method via weak secant equation
by: Leong, Wah June
Published: (2007)

Sufficient descent three term conjugate gradient method via symmetric rank-one update for large-scale optimization
by: Moyi, Aliyu Usman, et al.
Published: (2016)

An improvement of stochastic gradient descent approach for mean-variance portfolio optimization problem
by: S. W. Su, Stephanie, et al.
Published: (2021)

Conjugate gradient methods with sufficient descent condition for large-scale unconstrained optimization
by: Ling, Mei Mei
Published: (2015)

An improved multi-step gradient-type method for large scale optimization
by: Farid, Mahboubeh, et al.
Published: (2011)

Gradient method with multiple damping for large-scale unconstrained optimization
by: Sim, Hong Seng, et al.
Published: (2019)

A new gradient method via quasi-Cauchy relation which guarantees descent
by: Abu Hassan, Malik, et al.
Published: (2009)

A Mini-Batch Proximal Stochastic Recursive Gradient Algorithm with Diagonal Barzilai–Borwein Stepsize
by: Yu, T.T., et al.
Published: (2022)

A new diagonal gradient-type method for large scale unconstrained optimization
by: Farid, Mahboubeh, et al.
Published: (2013)

Scaled diagonal gradient-type method with extra update for large-scale unconstrained optimization
by: Farid, Mahboubeh, et al.
Published: (2013)

A new two-step gradient-type method for large-scale unconstrained optimization
by: Farid, Mahboubeh, et al.
Published: (2010)

The sufficient descent condition of nonlinear conjugate gradient method
by: Basri, Sri Mazzura, et al.
Published: (2018)

Scaled memoryless BFGS preconditioned steepest descent method for very large-scale unconstrained optimization
by: Leong, Wah June, et al.
Published: (2009)

A Minibatch Proximal Stochastic Recursive Gradient Algorithm Using a Trust-Region-Like Scheme and Barzilai-Borwein Stepsizes
by: Yu, T., et al.
Published: (2021)

Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization
by: Hong, Seng Sim, et al.
Published: (2018)

Hybridization of gradient descent algorithms with dynamic tunneling methods for global optimization
by: RoyChowdhury, Pinaki, et al.
Published: (2000)

Nonmonotone spectral gradient method based on memoryless symmetric rank-one update for large-scale unconstrained optimization
by: Hong, Seng Sim, et al.
Published: (2021)

A scaled three-term conjugate gradient method for unconstrained optimization
by: Arzuka, Ibrahim, et al.
Published: (2016)

Multi-spectral gradient method via variational technique under log-determinant norm for large-scale optimization
by: Hong, Seng Sim, et al.
Published: (2017)

On the long-time integration of stochastic gradient systems
by: Leimkuhler, B., et al.
Published: (2014)

Diagonal preconditioned conjugate gradient algorithm for unconstrained optimization
by: Ng, Choong Boon, et al.
Published: (2014)

A new gradient method via least change secant update
by: Leong, Wah June, et al.
Published: (2011)

Mental Stress Classification Among Higher Education Students In Malaysia From Electroencephalogram (Eeg) Using Convolutional Neural Network With Modified Stochastic Gradient Descent.
by: Rashid, Nur Ramizah Ramino
Published: (2024)

A gradient descent boosting spectrum modeling method based on back interval partial least squares
by: Ren, D., et al.
Published: (2015)

A scalar modification of three-term PRP-DL conjugate gradient method for solving large-scaled unconstrained optimization problems
by: Marjugi, Siti Mahani, et al.
Published: (2024)

A three-term conjugate gradient method with nonmonotone line search for unconstrained optimization
by: Moyi, Aliyu Usman, et al.
Published: (2016)

A monotone gradient method via weak secant equation for unconstrained optimization
by: Leong, Wah June, et al.
Published: (2010)

Barzilai-Borwein gradient method for solving fuzzy nonlinear
by: Mohammed Sulaiman, Ibrahim, et al.
Published: (2018)

Improved Hessian approximation with modified quasi-Cauchy relation for a gradient-type method
by: Leong, Wah June, et al.
Published: (2010)

Convergence of the Steepest Descent Method for Minimizing Convex Functions
by: Abu Hassan, Malik, et al.
Published: (2002)

A matrix-free quasi-Newton method for solving large-scale nonlinear systems.
by: Leong, Wah June, et al.
Published: (2011)

Variable order variable stepsize algorithm for solving nonlinear Duffing oscillator
by: Rasedee, Ahmad Fadly Nurullah, et al.
Published: (2017)

Modified Quasi-Newton Methods For Large-Scale Unconstrained Optimization
by: Leong, Wah June
Published: (2003)

Application of conjugate gradient approach for nonlinear optimal control problem with model-reality differences
by: Leong, Wah June, et al.
Published: (2018)

Application of conjugate gradient approach for nonlinear optimal control problem with model-reality difference
by: Sie, Long Kek, et al.
Published: (2018)

Adaptation to implied tilt: extensive spatial extrapolation of orientation gradients
by: Roach, Neil W., et al.
Published: (2013)

A Modified Spectral Gradient Method for Solving Non Linear System
by: Koay, Yeong Lin
Published: (2020)

Radial effect in stochastic diagonal approximate greatest descent
by: Tan, H., et al.
Published: (2017)

Stochastic diagonal approximate greatest descent in neural networks
by: Tan, H., et al.
Published: (2017)

Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization
by: Yu, T., et al.
Published: (2022)