A comparative study on some methods for handling multicollinearity problems
In regression, the objective is to explain the variation in one or more response variables, by associating this variation with proportional variation in one or more explanatory variables. A frequent obstacle is that several of the explanatory variables will vary in rather similar ways. As a result,...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Faculty of Science, Universiti Teknologi Malaysia
2006
|
| Subjects: | |
| Online Access: | http://eprints.utm.my/3662/ http://eprints.utm.my/3662/2/NorlizaAdnan2007_comparativestudyMulti.pdf |
| _version_ | 1848890619842265088 |
|---|---|
| author | Adnan, Norliza Ahmad, Maizah Hura Adnan, Robiah |
| author_facet | Adnan, Norliza Ahmad, Maizah Hura Adnan, Robiah |
| author_sort | Adnan, Norliza |
| building | UTeM Institutional Repository |
| collection | Online Access |
| description | In regression, the objective is to explain the variation in one or more response variables, by associating this variation with proportional variation in one or more explanatory variables. A frequent obstacle is that several of the explanatory variables will vary in rather similar ways. As a result, their collective power of explanation is considerably less than the sum of their individual powers. This phenomenon called multicollinearity, is a common problem in regression analysis. Handling multicollinearity problem in regression analysis is important because least squares estimations assume that predictor variables are not correlated with each other. The performances of ridge regression (RR), principal component regression (PCR) and partial least squares regression (PLSR) in handling multicollinearity problem in simulated data sets are compared to help and give future researchers a comprehensive view about the best procedure to handle multicollinearity problems. PCR is a combination of principal component analysis (PCA) and ordinary least squares regression (OLS) while PLSR is an approach similar to PCR because a component that can be used to reduce the number of variables need to be constructed. RR on the other hand is the modified least square method that allows a biased but more precise estimator. The algorithm is described and for the purpose of comparing the three methods, simulated data sets where the number of cases were less than the number of observations used. The goal was to develop a linear equation that relates all the predictor variables to a response variable. For comparison purposes, mean square errors (MSE) were calculated. A Monte Carlo simulation study was used to evaluate the effectiveness of these three procedures. The analysis including all simulations and calculations were done using statistical package S-Plus 2000 software.
|
| first_indexed | 2025-11-15T20:44:57Z |
| format | Article |
| id | utm-3662 |
| institution | Universiti Teknologi Malaysia |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-15T20:44:57Z |
| publishDate | 2006 |
| publisher | Faculty of Science, Universiti Teknologi Malaysia |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | utm-36622010-06-01T03:11:15Z http://eprints.utm.my/3662/ A comparative study on some methods for handling multicollinearity problems Adnan, Norliza Ahmad, Maizah Hura Adnan, Robiah QA Mathematics In regression, the objective is to explain the variation in one or more response variables, by associating this variation with proportional variation in one or more explanatory variables. A frequent obstacle is that several of the explanatory variables will vary in rather similar ways. As a result, their collective power of explanation is considerably less than the sum of their individual powers. This phenomenon called multicollinearity, is a common problem in regression analysis. Handling multicollinearity problem in regression analysis is important because least squares estimations assume that predictor variables are not correlated with each other. The performances of ridge regression (RR), principal component regression (PCR) and partial least squares regression (PLSR) in handling multicollinearity problem in simulated data sets are compared to help and give future researchers a comprehensive view about the best procedure to handle multicollinearity problems. PCR is a combination of principal component analysis (PCA) and ordinary least squares regression (OLS) while PLSR is an approach similar to PCR because a component that can be used to reduce the number of variables need to be constructed. RR on the other hand is the modified least square method that allows a biased but more precise estimator. The algorithm is described and for the purpose of comparing the three methods, simulated data sets where the number of cases were less than the number of observations used. The goal was to develop a linear equation that relates all the predictor variables to a response variable. For comparison purposes, mean square errors (MSE) were calculated. A Monte Carlo simulation study was used to evaluate the effectiveness of these three procedures. The analysis including all simulations and calculations were done using statistical package S-Plus 2000 software. Faculty of Science, Universiti Teknologi Malaysia 2006-12 Article PeerReviewed application/pdf en http://eprints.utm.my/3662/2/NorlizaAdnan2007_comparativestudyMulti.pdf Adnan, Norliza and Ahmad, Maizah Hura and Adnan, Robiah (2006) A comparative study on some methods for handling multicollinearity problems. Matematika, 22 (2). pp. 109-119. http://161.139.72.2/oldfs/images/stories/matematika/20062222.pdf |
| spellingShingle | QA Mathematics Adnan, Norliza Ahmad, Maizah Hura Adnan, Robiah A comparative study on some methods for handling multicollinearity problems |
| title | A comparative study on some methods for handling multicollinearity problems |
| title_full | A comparative study on some methods for handling multicollinearity problems |
| title_fullStr | A comparative study on some methods for handling multicollinearity problems |
| title_full_unstemmed | A comparative study on some methods for handling multicollinearity problems |
| title_short | A comparative study on some methods for handling multicollinearity problems |
| title_sort | comparative study on some methods for handling multicollinearity problems |
| topic | QA Mathematics |
| url | http://eprints.utm.my/3662/ http://eprints.utm.my/3662/ http://eprints.utm.my/3662/2/NorlizaAdnan2007_comparativestudyMulti.pdf |