OAERP: a better measure than accuracy in discriminating a better solution for stochastic classification training
The use of accuracy metric for stochastic classification training could lead the solution selecting towards the sub-optimal solution due to its less distinctive value and also unable to perform optimally when confronted with imbalanced class problem. In this study, a new evaluation metric that combi...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Asian Network for Scientific Information
2011
|
| Online Access: | http://psasir.upm.edu.my/id/eprint/22494/ http://psasir.upm.edu.my/id/eprint/22494/1/OAERP%20A%20better%20measure%20than%20accuracy%20in%20discriminating%20a%20better%20solution%20for%20stochastic%20classification%20training.pdf |
| _version_ | 1848844499656114176 |
|---|---|
| author | Hossin, Mohammad Sulaiman, Md. Nasir Mustapha, Aida Mustapha, Norwati O. K. Rahmat, Rahmita Wirza |
| author_facet | Hossin, Mohammad Sulaiman, Md. Nasir Mustapha, Aida Mustapha, Norwati O. K. Rahmat, Rahmita Wirza |
| author_sort | Hossin, Mohammad |
| building | UPM Institutional Repository |
| collection | Online Access |
| description | The use of accuracy metric for stochastic classification training could lead the solution selecting towards the sub-optimal solution due to its less distinctive value and also unable to perform optimally when confronted with imbalanced class problem. In this study, a new evaluation metric that combines accuracy metric with the extended precision and recall metrics to negate these detrimental effects was proposed. This new evaluation metric is known as Optimized Accuracy with Extended Recall-precision (OAERP). By using two examples, the results has shown that the OAERP metric has produced more distinctive and discriminating values as compared to accuracy metric. This paper also empirically demonstrates that Monte Carlo Sampling (MCS) algorithm that is trained by OAERP metric was able to obtain better predictive results than the one trained by the accuracy metric alone, using nine medical data sets. In addition, the OAERP metric also performed effectively when dealing with imbalanced class problems. Moreover, the t-test analysis also shows a clear advantage of the MCS model trained by the OAERP metric against its previous metric over five out of nine medical data sets. From the abovementioned results, it is clearly indicates that the OAERP metric is more likely to choose a better solution during classification training and lead towards a better trained classification model. |
| first_indexed | 2025-11-15T08:31:54Z |
| format | Article |
| id | upm-22494 |
| institution | Universiti Putra Malaysia |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-15T08:31:54Z |
| publishDate | 2011 |
| publisher | Asian Network for Scientific Information |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | upm-224942015-11-27T06:39:17Z http://psasir.upm.edu.my/id/eprint/22494/ OAERP: a better measure than accuracy in discriminating a better solution for stochastic classification training Hossin, Mohammad Sulaiman, Md. Nasir Mustapha, Aida Mustapha, Norwati O. K. Rahmat, Rahmita Wirza The use of accuracy metric for stochastic classification training could lead the solution selecting towards the sub-optimal solution due to its less distinctive value and also unable to perform optimally when confronted with imbalanced class problem. In this study, a new evaluation metric that combines accuracy metric with the extended precision and recall metrics to negate these detrimental effects was proposed. This new evaluation metric is known as Optimized Accuracy with Extended Recall-precision (OAERP). By using two examples, the results has shown that the OAERP metric has produced more distinctive and discriminating values as compared to accuracy metric. This paper also empirically demonstrates that Monte Carlo Sampling (MCS) algorithm that is trained by OAERP metric was able to obtain better predictive results than the one trained by the accuracy metric alone, using nine medical data sets. In addition, the OAERP metric also performed effectively when dealing with imbalanced class problems. Moreover, the t-test analysis also shows a clear advantage of the MCS model trained by the OAERP metric against its previous metric over five out of nine medical data sets. From the abovementioned results, it is clearly indicates that the OAERP metric is more likely to choose a better solution during classification training and lead towards a better trained classification model. Asian Network for Scientific Information 2011 Article PeerReviewed application/pdf en http://psasir.upm.edu.my/id/eprint/22494/1/OAERP%20A%20better%20measure%20than%20accuracy%20in%20discriminating%20a%20better%20solution%20for%20stochastic%20classification%20training.pdf Hossin, Mohammad and Sulaiman, Md. Nasir and Mustapha, Aida and Mustapha, Norwati and O. K. Rahmat, Rahmita Wirza (2011) OAERP: a better measure than accuracy in discriminating a better solution for stochastic classification training. Journal of Artificial Intelligence, 4 (3). pp. 187-196. ISSN 1994-5450; ESSN: 2077-2173 http://scialert.net/abstract/?doi=jai.2011.187.196 10.3923/jai.2011.187.196 |
| spellingShingle | Hossin, Mohammad Sulaiman, Md. Nasir Mustapha, Aida Mustapha, Norwati O. K. Rahmat, Rahmita Wirza OAERP: a better measure than accuracy in discriminating a better solution for stochastic classification training |
| title | OAERP: a better measure than accuracy in discriminating a better solution for stochastic classification training |
| title_full | OAERP: a better measure than accuracy in discriminating a better solution for stochastic classification training |
| title_fullStr | OAERP: a better measure than accuracy in discriminating a better solution for stochastic classification training |
| title_full_unstemmed | OAERP: a better measure than accuracy in discriminating a better solution for stochastic classification training |
| title_short | OAERP: a better measure than accuracy in discriminating a better solution for stochastic classification training |
| title_sort | oaerp: a better measure than accuracy in discriminating a better solution for stochastic classification training |
| url | http://psasir.upm.edu.my/id/eprint/22494/ http://psasir.upm.edu.my/id/eprint/22494/ http://psasir.upm.edu.my/id/eprint/22494/ http://psasir.upm.edu.my/id/eprint/22494/1/OAERP%20A%20better%20measure%20than%20accuracy%20in%20discriminating%20a%20better%20solution%20for%20stochastic%20classification%20training.pdf |