Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization
We study the problem of minimizing the sum of two functions. The first function is the average of a large number of nonconvex component functions and the second function is a convex (possibly nonsmooth) function that admits a simple proximal mapping. With a diagonal Barzilai-Borwein stepsize for upd...
| Main Authors: | , , , |
|---|---|
| Format: | Journal Article |
| Published: |
2022
|
| Online Access: | http://hdl.handle.net/20.500.11937/91423 |
| _version_ | 1848765516494143488 |
|---|---|
| author | Yu, T. Liu, X.W. Dai, Y.H. Sun, Jie |
| author_facet | Yu, T. Liu, X.W. Dai, Y.H. Sun, Jie |
| author_sort | Yu, T. |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | We study the problem of minimizing the sum of two functions. The first function is the average of a large number of nonconvex component functions and the second function is a convex (possibly nonsmooth) function that admits a simple proximal mapping. With a diagonal Barzilai-Borwein stepsize for updating the metric, we propose a variable metric proximal stochastic variance reduced gradient method in the mini-batch setting, named VM-SVRG. It is proved that VM-SVRG converges sublinearly to a stationary point in expectation. We further suggest a variant of VM-SVRG to achieve linear convergence rate in expectation for nonconvex problems satisfying the proximal Polyak-Lojasiewicz inequality. The complexity of VM-SVRG is lower than that of the proximal gradient method and proximal stochastic gradient method, and is the same as the proximal stochastic variance reduced gradient method. Numerical experiments are conducted on standard data sets. Comparisons with other advanced proximal stochastic gradient methods show the efficiency of the proposed method. |
| first_indexed | 2025-11-14T11:36:30Z |
| format | Journal Article |
| id | curtin-20.500.11937-91423 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T11:36:30Z |
| publishDate | 2022 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-914232023-05-04T04:50:38Z Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization Yu, T. Liu, X.W. Dai, Y.H. Sun, Jie We study the problem of minimizing the sum of two functions. The first function is the average of a large number of nonconvex component functions and the second function is a convex (possibly nonsmooth) function that admits a simple proximal mapping. With a diagonal Barzilai-Borwein stepsize for updating the metric, we propose a variable metric proximal stochastic variance reduced gradient method in the mini-batch setting, named VM-SVRG. It is proved that VM-SVRG converges sublinearly to a stationary point in expectation. We further suggest a variant of VM-SVRG to achieve linear convergence rate in expectation for nonconvex problems satisfying the proximal Polyak-Lojasiewicz inequality. The complexity of VM-SVRG is lower than that of the proximal gradient method and proximal stochastic gradient method, and is the same as the proximal stochastic variance reduced gradient method. Numerical experiments are conducted on standard data sets. Comparisons with other advanced proximal stochastic gradient methods show the efficiency of the proposed method. 2022 Journal Article http://hdl.handle.net/20.500.11937/91423 10.3934/jimo.2021084 fulltext |
| spellingShingle | Yu, T. Liu, X.W. Dai, Y.H. Sun, Jie Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization |
| title | Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization |
| title_full | Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization |
| title_fullStr | Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization |
| title_full_unstemmed | Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization |
| title_short | Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization |
| title_sort | variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization |
| url | http://hdl.handle.net/20.500.11937/91423 |