On group-wise ℓp regularization: Theory and efficient algorithms
Following advances in compressed sensing and high-dimensional statistics, many pattern recognition methods have been developed with ℓ1 regularization, which promotes sparse solutions. In this work, we instead advocate the use of ℓp (2 ≥ p > 1) regularization in a group setting which provides a be...
| Main Author: | |
|---|---|
| Format: | Journal Article |
| Published: |
Elsevier
2015
|
| Online Access: | http://hdl.handle.net/20.500.11937/58894 |
| _version_ | 1848760369926897664 |
|---|---|
| author | Pham, DucSon |
| author_facet | Pham, DucSon |
| author_sort | Pham, DucSon |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | Following advances in compressed sensing and high-dimensional statistics, many pattern recognition methods have been developed with ℓ1 regularization, which promotes sparse solutions. In this work, we instead advocate the use of ℓp (2 ≥ p > 1) regularization in a group setting which provides a better trade-off between sparsity and algorithmic stability. We focus on the simplest case with squared loss, which is known as group bridge regression. On the theoretical side, we prove that group bridge regression is uniformly stable and thus generalizes, which is an important property of a learning method. On the computational side, we make group bridge regression more practically attractive by deriving provably convergent and computationally efficient optimization algorithms. We show that there are at least several values of p over (1,2) at which the iterative update is analytical, thus it is even suitable for large-scale settings. We demonstrate the clear advantage of group bridge regression with the proposed algorithms over other competitive alternatives on several datasets. As ℓp-regularization allows one to achieve flexibility in sparseness/denseness of the solution, we hope that the algorithms will be useful for future applications of this regularization. |
| first_indexed | 2025-11-14T10:14:41Z |
| format | Journal Article |
| id | curtin-20.500.11937-58894 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T10:14:41Z |
| publishDate | 2015 |
| publisher | Elsevier |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-588942018-02-26T08:16:32Z On group-wise ℓp regularization: Theory and efficient algorithms Pham, DucSon Following advances in compressed sensing and high-dimensional statistics, many pattern recognition methods have been developed with ℓ1 regularization, which promotes sparse solutions. In this work, we instead advocate the use of ℓp (2 ≥ p > 1) regularization in a group setting which provides a better trade-off between sparsity and algorithmic stability. We focus on the simplest case with squared loss, which is known as group bridge regression. On the theoretical side, we prove that group bridge regression is uniformly stable and thus generalizes, which is an important property of a learning method. On the computational side, we make group bridge regression more practically attractive by deriving provably convergent and computationally efficient optimization algorithms. We show that there are at least several values of p over (1,2) at which the iterative update is analytical, thus it is even suitable for large-scale settings. We demonstrate the clear advantage of group bridge regression with the proposed algorithms over other competitive alternatives on several datasets. As ℓp-regularization allows one to achieve flexibility in sparseness/denseness of the solution, we hope that the algorithms will be useful for future applications of this regularization. 2015 Journal Article http://hdl.handle.net/20.500.11937/58894 10.1016/j.patcog.2015.05.009 Elsevier fulltext |
| spellingShingle | Pham, DucSon On group-wise ℓp regularization: Theory and efficient algorithms |
| title | On group-wise ℓp regularization: Theory and efficient algorithms |
| title_full | On group-wise ℓp regularization: Theory and efficient algorithms |
| title_fullStr | On group-wise ℓp regularization: Theory and efficient algorithms |
| title_full_unstemmed | On group-wise ℓp regularization: Theory and efficient algorithms |
| title_short | On group-wise ℓp regularization: Theory and efficient algorithms |
| title_sort | on group-wise ℓp regularization: theory and efficient algorithms |
| url | http://hdl.handle.net/20.500.11937/58894 |