Forecast evaluation tests and negative long-run variance estimates in small samples
In this paper, we show that when computing standard Diebold-Mariano-type tests for equal forecast accuracy and forecast encompassing, the long-run variance can frequently be negative when dealing with multi-step-ahead predictions in small, but empirically relevant, sample sizes. We subsequently cons...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Published: |
Elsevier
2017
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/43017/ |
| _version_ | 1848796623875866624 |
|---|---|
| author | Harvey, David I. Leybourne, Stephen J. Whitehouse, Emily J. |
| author_facet | Harvey, David I. Leybourne, Stephen J. Whitehouse, Emily J. |
| author_sort | Harvey, David I. |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | In this paper, we show that when computing standard Diebold-Mariano-type tests for equal forecast accuracy and forecast encompassing, the long-run variance can frequently be negative when dealing with multi-step-ahead predictions in small, but empirically relevant, sample sizes. We subsequently consider a number of alternative approaches to dealing with this problem, including direct inference in the problem cases and use of long-run variance estimators that guarantee positivity. The finite sample size and power of the different approaches are evaluated using extensive Monte Carlo simulation exercises. Overall, for multi-step-ahead forecasts, we find that the recently proposed Coroneo and Iacone (2016) test, which is based on a weighted periodogram long-run variance estimator, offers the best finite sample size and power performance. |
| first_indexed | 2025-11-14T19:50:56Z |
| format | Article |
| id | nottingham-43017 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| last_indexed | 2025-11-14T19:50:56Z |
| publishDate | 2017 |
| publisher | Elsevier |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-430172020-05-04T19:15:40Z https://eprints.nottingham.ac.uk/43017/ Forecast evaluation tests and negative long-run variance estimates in small samples Harvey, David I. Leybourne, Stephen J. Whitehouse, Emily J. In this paper, we show that when computing standard Diebold-Mariano-type tests for equal forecast accuracy and forecast encompassing, the long-run variance can frequently be negative when dealing with multi-step-ahead predictions in small, but empirically relevant, sample sizes. We subsequently consider a number of alternative approaches to dealing with this problem, including direct inference in the problem cases and use of long-run variance estimators that guarantee positivity. The finite sample size and power of the different approaches are evaluated using extensive Monte Carlo simulation exercises. Overall, for multi-step-ahead forecasts, we find that the recently proposed Coroneo and Iacone (2016) test, which is based on a weighted periodogram long-run variance estimator, offers the best finite sample size and power performance. Elsevier 2017-11-01 Article PeerReviewed Harvey, David I., Leybourne, Stephen J. and Whitehouse, Emily J. (2017) Forecast evaluation tests and negative long-run variance estimates in small samples. International Journal of Forecasting, 33 (4). pp. 833-847. ISSN 0169-2070 Forecast evaluation; Long-run variance estimation; Simulation; Diebold-Mariano test; Forecasting http://www.sciencedirect.com/science/article/pii/S0169207017300559 doi:10.1016/j.ijforecast.2017.05.001 doi:10.1016/j.ijforecast.2017.05.001 |
| spellingShingle | Forecast evaluation; Long-run variance estimation; Simulation; Diebold-Mariano test; Forecasting Harvey, David I. Leybourne, Stephen J. Whitehouse, Emily J. Forecast evaluation tests and negative long-run variance estimates in small samples |
| title | Forecast evaluation tests and negative long-run variance estimates in small samples |
| title_full | Forecast evaluation tests and negative long-run variance estimates in small samples |
| title_fullStr | Forecast evaluation tests and negative long-run variance estimates in small samples |
| title_full_unstemmed | Forecast evaluation tests and negative long-run variance estimates in small samples |
| title_short | Forecast evaluation tests and negative long-run variance estimates in small samples |
| title_sort | forecast evaluation tests and negative long-run variance estimates in small samples |
| topic | Forecast evaluation; Long-run variance estimation; Simulation; Diebold-Mariano test; Forecasting |
| url | https://eprints.nottingham.ac.uk/43017/ https://eprints.nottingham.ac.uk/43017/ https://eprints.nottingham.ac.uk/43017/ |