A new accuracy measure based on bounded relative error for time series forecasting
Many accuracy measures have been proposed in the past for time series forecasting comparisons. However, many of these measures suffer from one or more issues such as poor resistance to outliers and scale dependence. In this paper, while summarising commonly used accuracy measures, a special review i...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Published: |
Public Library of Science
2017
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/41565/ |
| _version_ | 1848796304261513216 |
|---|---|
| author | Chen, Chao Twycross, Jamie Garibaldi, Jonathan M. |
| author_facet | Chen, Chao Twycross, Jamie Garibaldi, Jonathan M. |
| author_sort | Chen, Chao |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | Many accuracy measures have been proposed in the past for time series forecasting comparisons. However, many of these measures suffer from one or more issues such as poor resistance to outliers and scale dependence. In this paper, while summarising commonly used accuracy measures, a special review is made on the symmetric mean absolute percentage error. Moreover, a new accuracy measure called the Unscaled Mean Bounded Relative Absolute Error (UMBRAE), which combines the best features of various alternative measures, is proposed to address the common issues of existing measures. A comparative evaluation on the proposed and related measures has been made with both synthetic and real-world data. The results indicate that the proposed measure, with user selectable benchmark, performs as well as or better than other measures on selected criteria. Though it has been commonly accepted that there is no single best accuracy measure, we suggest that UMBRAE could be a good choice to evaluate forecasting methods, especially for cases where measures based on geometric mean of relative errors, such as the geometric mean relative absolute error, are preferred. |
| first_indexed | 2025-11-14T19:45:51Z |
| format | Article |
| id | nottingham-41565 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| last_indexed | 2025-11-14T19:45:51Z |
| publishDate | 2017 |
| publisher | Public Library of Science |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-415652020-05-04T18:38:58Z https://eprints.nottingham.ac.uk/41565/ A new accuracy measure based on bounded relative error for time series forecasting Chen, Chao Twycross, Jamie Garibaldi, Jonathan M. Many accuracy measures have been proposed in the past for time series forecasting comparisons. However, many of these measures suffer from one or more issues such as poor resistance to outliers and scale dependence. In this paper, while summarising commonly used accuracy measures, a special review is made on the symmetric mean absolute percentage error. Moreover, a new accuracy measure called the Unscaled Mean Bounded Relative Absolute Error (UMBRAE), which combines the best features of various alternative measures, is proposed to address the common issues of existing measures. A comparative evaluation on the proposed and related measures has been made with both synthetic and real-world data. The results indicate that the proposed measure, with user selectable benchmark, performs as well as or better than other measures on selected criteria. Though it has been commonly accepted that there is no single best accuracy measure, we suggest that UMBRAE could be a good choice to evaluate forecasting methods, especially for cases where measures based on geometric mean of relative errors, such as the geometric mean relative absolute error, are preferred. Public Library of Science 2017-03-24 Article PeerReviewed Chen, Chao, Twycross, Jamie and Garibaldi, Jonathan M. (2017) A new accuracy measure based on bounded relative error for time series forecasting. PLoS ONE, 12 (3). e0174202/1-e0174202/23. ISSN 1932-6203 Forecasting performance; Accuracy measure; Relative measure; Bounded error; Unscaled mean bounded relative absolute error http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0174202 doi:10.1371/journal.pone.0174202 doi:10.1371/journal.pone.0174202 |
| spellingShingle | Forecasting performance; Accuracy measure; Relative measure; Bounded error; Unscaled mean bounded relative absolute error Chen, Chao Twycross, Jamie Garibaldi, Jonathan M. A new accuracy measure based on bounded relative error for time series forecasting |
| title | A new accuracy measure based on bounded relative error for time series forecasting |
| title_full | A new accuracy measure based on bounded relative error for time series forecasting |
| title_fullStr | A new accuracy measure based on bounded relative error for time series forecasting |
| title_full_unstemmed | A new accuracy measure based on bounded relative error for time series forecasting |
| title_short | A new accuracy measure based on bounded relative error for time series forecasting |
| title_sort | new accuracy measure based on bounded relative error for time series forecasting |
| topic | Forecasting performance; Accuracy measure; Relative measure; Bounded error; Unscaled mean bounded relative absolute error |
| url | https://eprints.nottingham.ac.uk/41565/ https://eprints.nottingham.ac.uk/41565/ https://eprints.nottingham.ac.uk/41565/ |