From Item Analysis to Assessment Analysis: Introducing New Formulae
Item analysis of individual multiple choice questions have been widely used for several decades. But formulae for analysis of manually-marked assessments are lacking. Evaluation and comparison of such assessments used in medical schools remain a guess work. In this study we have introduced new for...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
AMEE MedEdPublish
2017
|
| Subjects: | |
| Online Access: | http://ir.unimas.my/id/eprint/16234/ http://ir.unimas.my/id/eprint/16234/1/From%20Item%20Analysis%20to%20Assessment%20Analysis%20%28abstract%29.pdf |
| Summary: | Item analysis of individual multiple choice questions have been widely used for several decades. But formulae
for analysis of manually-marked assessments are lacking. Evaluation and comparison of such assessments used
in medical schools remain a guess work. In this study we have introduced new formulae aligned with item
analysis formulae, which can be used to analyse all assessment methods. While the existing formulae use a
binary (pass/fail) criterion, the new formulae have used actual scores. While the existing formulae use the
scores of a single assessment to rank the candidates to determine the high and low scorers for calculating the
discrimination index (DISi), the new formula used the grand total scores of the entire examination for it. It is
claimed that the new formulae, utilizing actual scores, would make the indexes more realistic. Eight
examinations, each comprising nine assessments, were used to validate this claim. The DISi and Difficulty
Index (DIFi) were calculated using the binary formulae and the new formulae. Comparisons were made using
parametric and non-parametric tests. The ensued positive correlation of indexes indicated that the new formulae
are feasible, realistic and easy to apply. More replicates are required to prove their validity and reliability. |
|---|