An evaluation of statistical methods for determining agreement and reliability in medicine / Rafdzah Ahmad Zaki
Background and objective: Agreement and reliability are both important parameters in determining the quality of an instrument. The general aim of this study was to evaluate different statistical methods used to assess agreement and reliability of medical instruments that measure the same continuous...
| Main Author: | |
|---|---|
| Format: | Thesis |
| Published: |
2013
|
| Subjects: | |
| Online Access: | http://studentsrepo.um.edu.my/5543/ http://studentsrepo.um.edu.my/5543/1/ThesisFinal_AN_EVALUATION_OF_STATISTICAL_METHODS.pdf |
| _version_ | 1848772916177534976 |
|---|---|
| author | Rafdzah Ahmad Zakibud, Abdullah |
| author_facet | Rafdzah Ahmad Zakibud, Abdullah |
| author_sort | Rafdzah Ahmad Zakibud, Abdullah |
| building | UM Research Repository |
| collection | Online Access |
| description | Background and objective:
Agreement and reliability are both important parameters in determining the quality of an instrument. The general aim of this study was to evaluate different statistical methods used to assess agreement and reliability of medical instruments that measure the same continuous outcome. This study compares the most commonly used statistical methods, and also compares the proposed method in the analysis of agreement. Two separate systematic reviews were performed at the beginning of this study to identify the most popular method used to assess agreement and reliability of medical instruments.
Methods:
Two systematic reviews, one on agreement studies and another on reliability studies were carried out. A cross-sectional study was then conducted to collect data in two population settings; an institutional and community setting. Data were collected for blood glucose level, systolic blood pressure, diastolic blood pressure, body weight, and peak expiratory flow rate to assess methods used in agreement studies. Data were also collected for reliability studies. The variables for this were systolic blood pressure, diastolic blood pressure, heart rate, body temperature, peak expiratory flow rate and carbon monoxide level. Evaluations of agreement and reliability statistical methods were carried out on the original clinical data and simulated data. The agreement evaluation involved a comparison of the most commonly used methods (Bland-Altman Limits of Agreement and Intra-class Correlation Coefficient for agreement), comparison of slopes and y-intercepts analysis, and a new proposed agreement model. The reliability evaluation involved a comparison of the Bland-Altman Limits of Agreement, Intra-class Correlation Coefficient for consistency (ICCC), and Intra-class Correlation Coefficient for absolute agreement (ICCA).
iii
Results and Conclusion:
The systematic reviews identified some issues related to method comparison studies including the application of inappropriate statistical methods, and the importance of education in method comparison studies among medical professionals. The evaluations of different statistical methods provided different conclusions on agreement and reliability. Each method was found to have its own strengths and weaknesses and no single method was found to be perfect. Several recommendations were made including optimal sample size for Bland-Altman analysis, and a proposed flowchart to guide analysis of agreement and reliability in method comparison studies. |
| first_indexed | 2025-11-14T13:34:06Z |
| format | Thesis |
| id | um-5543 |
| institution | University Malaya |
| institution_category | Local University |
| last_indexed | 2025-11-14T13:34:06Z |
| publishDate | 2013 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | um-55432015-06-25T01:00:05Z An evaluation of statistical methods for determining agreement and reliability in medicine / Rafdzah Ahmad Zaki Rafdzah Ahmad Zakibud, Abdullah R Medicine (General) Background and objective: Agreement and reliability are both important parameters in determining the quality of an instrument. The general aim of this study was to evaluate different statistical methods used to assess agreement and reliability of medical instruments that measure the same continuous outcome. This study compares the most commonly used statistical methods, and also compares the proposed method in the analysis of agreement. Two separate systematic reviews were performed at the beginning of this study to identify the most popular method used to assess agreement and reliability of medical instruments. Methods: Two systematic reviews, one on agreement studies and another on reliability studies were carried out. A cross-sectional study was then conducted to collect data in two population settings; an institutional and community setting. Data were collected for blood glucose level, systolic blood pressure, diastolic blood pressure, body weight, and peak expiratory flow rate to assess methods used in agreement studies. Data were also collected for reliability studies. The variables for this were systolic blood pressure, diastolic blood pressure, heart rate, body temperature, peak expiratory flow rate and carbon monoxide level. Evaluations of agreement and reliability statistical methods were carried out on the original clinical data and simulated data. The agreement evaluation involved a comparison of the most commonly used methods (Bland-Altman Limits of Agreement and Intra-class Correlation Coefficient for agreement), comparison of slopes and y-intercepts analysis, and a new proposed agreement model. The reliability evaluation involved a comparison of the Bland-Altman Limits of Agreement, Intra-class Correlation Coefficient for consistency (ICCC), and Intra-class Correlation Coefficient for absolute agreement (ICCA). iii Results and Conclusion: The systematic reviews identified some issues related to method comparison studies including the application of inappropriate statistical methods, and the importance of education in method comparison studies among medical professionals. The evaluations of different statistical methods provided different conclusions on agreement and reliability. Each method was found to have its own strengths and weaknesses and no single method was found to be perfect. Several recommendations were made including optimal sample size for Bland-Altman analysis, and a proposed flowchart to guide analysis of agreement and reliability in method comparison studies. 2013 Thesis NonPeerReviewed application/pdf http://studentsrepo.um.edu.my/5543/1/ThesisFinal_AN_EVALUATION_OF_STATISTICAL_METHODS.pdf Rafdzah Ahmad Zakibud, Abdullah (2013) An evaluation of statistical methods for determining agreement and reliability in medicine / Rafdzah Ahmad Zaki. PhD thesis, University of Malaya. http://studentsrepo.um.edu.my/5543/ |
| spellingShingle | R Medicine (General) Rafdzah Ahmad Zakibud, Abdullah An evaluation of statistical methods for determining agreement and reliability in medicine / Rafdzah Ahmad Zaki |
| title | An evaluation of statistical methods for determining agreement and reliability in medicine / Rafdzah Ahmad Zaki |
| title_full | An evaluation of statistical methods for determining agreement and reliability in medicine / Rafdzah Ahmad Zaki |
| title_fullStr | An evaluation of statistical methods for determining agreement and reliability in medicine / Rafdzah Ahmad Zaki |
| title_full_unstemmed | An evaluation of statistical methods for determining agreement and reliability in medicine / Rafdzah Ahmad Zaki |
| title_short | An evaluation of statistical methods for determining agreement and reliability in medicine / Rafdzah Ahmad Zaki |
| title_sort | evaluation of statistical methods for determining agreement and reliability in medicine / rafdzah ahmad zaki |
| topic | R Medicine (General) |
| url | http://studentsrepo.um.edu.my/5543/ http://studentsrepo.um.edu.my/5543/1/ThesisFinal_AN_EVALUATION_OF_STATISTICAL_METHODS.pdf |