Validation of spatial microsimulation models: a proposal to adopt the Bland-Altman method
Model validation is recognised as crucial to microsimulation modelling. However, modellers encounter difficulty in choosing the most meaningful methods to compare simulated and actual values. The aim of this paper is to introduce and demonstrate a method employed widely in healthcare calibration stu...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Published: |
International Microsimulation Association
2016
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/41830/ |
| _version_ | 1848796362496278528 |
|---|---|
| author | Timmins, Kate A. Edwards, Kimberley L. |
| author_facet | Timmins, Kate A. Edwards, Kimberley L. |
| author_sort | Timmins, Kate A. |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | Model validation is recognised as crucial to microsimulation modelling. However, modellers encounter difficulty in choosing the most meaningful methods to compare simulated and actual values. The aim of this paper is to introduce and demonstrate a method employed widely in healthcare calibration studies.
The ‘Bland-Altman plot’ consists of a plot of the difference between two methods against the mean (x-y versus x+y/2). A case study is presented to illustrate the method in practice for spatial microsimulation validation. The study features a deterministic combinatorial model (SimObesity), which modelled a synthetic population for England at the ward level using survey (ELSA) and Census 2011 data. Bland-Altman plots were generated, plotting simulated and census ward-level totals for each category of all constraint (benchmark) variables. Other validation metrics, such as R2, SEI, TAE and RMSE, are also presented for comparison.
The case study demonstrates how the Bland-Altman plots are interpreted. The simple visualisation of both individual- (ward-) level difference and total variation gives the method an advantage over existing tools used in model validation. There still remains the question of what constitutes a valid or well-fitting model. However, the Bland Altman method can usefully be added to the canon of calibration methods. |
| first_indexed | 2025-11-14T19:46:47Z |
| format | Article |
| id | nottingham-41830 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| last_indexed | 2025-11-14T19:46:47Z |
| publishDate | 2016 |
| publisher | International Microsimulation Association |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-418302020-05-04T17:52:18Z https://eprints.nottingham.ac.uk/41830/ Validation of spatial microsimulation models: a proposal to adopt the Bland-Altman method Timmins, Kate A. Edwards, Kimberley L. Model validation is recognised as crucial to microsimulation modelling. However, modellers encounter difficulty in choosing the most meaningful methods to compare simulated and actual values. The aim of this paper is to introduce and demonstrate a method employed widely in healthcare calibration studies. The ‘Bland-Altman plot’ consists of a plot of the difference between two methods against the mean (x-y versus x+y/2). A case study is presented to illustrate the method in practice for spatial microsimulation validation. The study features a deterministic combinatorial model (SimObesity), which modelled a synthetic population for England at the ward level using survey (ELSA) and Census 2011 data. Bland-Altman plots were generated, plotting simulated and census ward-level totals for each category of all constraint (benchmark) variables. Other validation metrics, such as R2, SEI, TAE and RMSE, are also presented for comparison. The case study demonstrates how the Bland-Altman plots are interpreted. The simple visualisation of both individual- (ward-) level difference and total variation gives the method an advantage over existing tools used in model validation. There still remains the question of what constitutes a valid or well-fitting model. However, the Bland Altman method can usefully be added to the canon of calibration methods. International Microsimulation Association 2016-05-12 Article PeerReviewed Timmins, Kate A. and Edwards, Kimberley L. (2016) Validation of spatial microsimulation models: a proposal to adopt the Bland-Altman method. International Journal of Microsimulation, 9 (2). pp. 106-122. ISSN 1747-5864 Validation Bland Altman Spatial microsimulation http://www.microsimulation.org/IJM/V9_2/IJM_9_2_2016_Timmins.pdf |
| spellingShingle | Validation Bland Altman Spatial microsimulation Timmins, Kate A. Edwards, Kimberley L. Validation of spatial microsimulation models: a proposal to adopt the Bland-Altman method |
| title | Validation of spatial microsimulation models: a proposal to adopt the Bland-Altman method |
| title_full | Validation of spatial microsimulation models: a proposal to adopt the Bland-Altman method |
| title_fullStr | Validation of spatial microsimulation models: a proposal to adopt the Bland-Altman method |
| title_full_unstemmed | Validation of spatial microsimulation models: a proposal to adopt the Bland-Altman method |
| title_short | Validation of spatial microsimulation models: a proposal to adopt the Bland-Altman method |
| title_sort | validation of spatial microsimulation models: a proposal to adopt the bland-altman method |
| topic | Validation Bland Altman Spatial microsimulation |
| url | https://eprints.nottingham.ac.uk/41830/ https://eprints.nottingham.ac.uk/41830/ |