Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data
The main aim of this paper is to propose a novel method (RMD-MRCD-PCA) of identification of High Leverage Points (HLPs) in high-dimensional sparse data. It is to address the weakness of the Robust Mahalanobis Distance (RMD) method which is based on the Minimum Regularized Covariance Determinant (RMD...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Published: |
Taylor and Francis
2022
|
| Online Access: | http://psasir.upm.edu.my/id/eprint/102186/ |
| _version_ | 1848863737706971136 |
|---|---|
| author | Siti Zahariah Midi, Habshah |
| author_facet | Siti Zahariah Midi, Habshah |
| author_sort | Siti Zahariah |
| building | UPM Institutional Repository |
| collection | Online Access |
| description | The main aim of this paper is to propose a novel method (RMD-MRCD-PCA) of identification of High Leverage Points (HLPs) in high-dimensional sparse data. It is to address the weakness of the Robust Mahalanobis Distance (RMD) method which is based on the Minimum Regularized Covariance Determinant (RMD-MRCD), which indicates a decrease in its performance as the number of independent variables (p) increases. The RMD-MRCD-PCA is developed by incorporating the Principal Component Analysis (PCA) in the MRCD algorithm whereby this robust approach shrinks the covariance matrix to make it invertible and thus, can be employed to compute the RMD for high dimensional data. A simulation study and two real data sets are used to illustrate the merit of our proposed method compared to the RMD-MRCD and Robust PCA (ROBPCA) methods. Findings show that the performance of the RMD-MRCD is similar to the performance of the RMD-MRCD-PCA for p close to 200. However, its performance tends to decrease when the number of p is more than 200 and worsens at p equals 700 and larger. On the other hand, the ROBPCA is not effective for less than 20% contamination as it suffers from serious swamping problems. |
| first_indexed | 2025-11-15T13:37:41Z |
| format | Article |
| id | upm-102186 |
| institution | Universiti Putra Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-15T13:37:41Z |
| publishDate | 2022 |
| publisher | Taylor and Francis |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | upm-1021862023-07-10T00:10:13Z http://psasir.upm.edu.my/id/eprint/102186/ Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data Siti Zahariah Midi, Habshah The main aim of this paper is to propose a novel method (RMD-MRCD-PCA) of identification of High Leverage Points (HLPs) in high-dimensional sparse data. It is to address the weakness of the Robust Mahalanobis Distance (RMD) method which is based on the Minimum Regularized Covariance Determinant (RMD-MRCD), which indicates a decrease in its performance as the number of independent variables (p) increases. The RMD-MRCD-PCA is developed by incorporating the Principal Component Analysis (PCA) in the MRCD algorithm whereby this robust approach shrinks the covariance matrix to make it invertible and thus, can be employed to compute the RMD for high dimensional data. A simulation study and two real data sets are used to illustrate the merit of our proposed method compared to the RMD-MRCD and Robust PCA (ROBPCA) methods. Findings show that the performance of the RMD-MRCD is similar to the performance of the RMD-MRCD-PCA for p close to 200. However, its performance tends to decrease when the number of p is more than 200 and worsens at p equals 700 and larger. On the other hand, the ROBPCA is not effective for less than 20% contamination as it suffers from serious swamping problems. Taylor and Francis 2022-07-08 Article PeerReviewed Siti Zahariah and Midi, Habshah (2022) Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data. Journal of Applied Statistics. pp. 1-19. ISSN 0266-4763 https://www.tandfonline.com/doi/abs/10.1080/02664763.2022.2093842?journalCode=cjas20 10.1080/02664763.2022.2093842 |
| spellingShingle | Siti Zahariah Midi, Habshah Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data |
| title | Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data |
| title_full | Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data |
| title_fullStr | Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data |
| title_full_unstemmed | Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data |
| title_short | Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data |
| title_sort | minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data |
| url | http://psasir.upm.edu.my/id/eprint/102186/ http://psasir.upm.edu.my/id/eprint/102186/ http://psasir.upm.edu.my/id/eprint/102186/ |