Quasi-Newton method for sparse matrix factorization with frobenius norm regularization - June, Leong Wah; Ab Rahmin, Nor Aliza; Peng, Lim Fong; Firdaus Faris, Muhammad Yusril (Persatuan Sains Matematik Malaysia, 2024)
This paper considers quasi-Newton method for sparse matrix factorization (SMF) that incorporates Frobenius norm regularization to control overfitting and enhance generalization in data-driven applications. Sparse matrix factorization seeks to decompose a matrix into 2 matrices while promoting sparsity in one (or both) of the resulting factors, which is particularly useful in applications such as recommendation systems, signal processing, and dimensionality reduction. The Frobenius norm regularization is employed to penalize large parameter values, ensuring sparser factorization. The proposed quasi-Newton method, leveraging approximate second-order information, efficiently optimizes the objective function with significantly reduced computational overhead compared to full Newton methods. Experimental results on an example demonstrate the efficacy of the method in achieving high-quality sparse factorizations under different regularization parameter.