Does calibration reduce variability in the assessment of accounting learning outcomes?
© 2015 Taylor & Francis Reliable, consistent assessment process that produces comparable assessment grades between assessors and institutions is a core activity and an ongoing challenge with which universities have failed to come to terms. In this paper, we report results from an experiment that...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Journal Article |
| Published: |
Routledge
2015
|
| Online Access: | http://www.tandfonline.com/ http://hdl.handle.net/20.500.11937/25986 |
| _version_ | 1848751858526453760 |
|---|---|
| author | O Connell, B. De Lange, Paul Freeman, M. Hancock, P. Abraham, A. Howieson, B. Watty, K. |
| author_facet | O Connell, B. De Lange, Paul Freeman, M. Hancock, P. Abraham, A. Howieson, B. Watty, K. |
| author_sort | O Connell, B. |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | © 2015 Taylor & Francis Reliable, consistent assessment process that produces comparable assessment grades between assessors and institutions is a core activity and an ongoing challenge with which universities have failed to come to terms. In this paper, we report results from an experiment that tests the impact of an intervention designed to reduce grader variability and develop a shared understanding of national threshold learning standards by a cohort of reviewers. The intervention involved consensus moderation of samples of accounting students’ work, with a focus on three research questions. First, what is the quantifiable difference in grader variability on the assessment of learning outcomes in ‘application skills’ and ‘judgement’? Second, does participation in the workshops lead to reduced disparity in the assessment of the students’ learning outcomes in ‘application skills’ and ‘judgement’? Third, does participation in the workshops lead to greater confidence by reviewers in their ability to assess students’ skills in application skills and judgement? Our findings suggest consensus moderation does reduce variability across graders and also builds grader confidence. |
| first_indexed | 2025-11-14T07:59:24Z |
| format | Journal Article |
| id | curtin-20.500.11937-25986 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T07:59:24Z |
| publishDate | 2015 |
| publisher | Routledge |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-259862017-09-13T15:25:12Z Does calibration reduce variability in the assessment of accounting learning outcomes? O Connell, B. De Lange, Paul Freeman, M. Hancock, P. Abraham, A. Howieson, B. Watty, K. © 2015 Taylor & Francis Reliable, consistent assessment process that produces comparable assessment grades between assessors and institutions is a core activity and an ongoing challenge with which universities have failed to come to terms. In this paper, we report results from an experiment that tests the impact of an intervention designed to reduce grader variability and develop a shared understanding of national threshold learning standards by a cohort of reviewers. The intervention involved consensus moderation of samples of accounting students’ work, with a focus on three research questions. First, what is the quantifiable difference in grader variability on the assessment of learning outcomes in ‘application skills’ and ‘judgement’? Second, does participation in the workshops lead to reduced disparity in the assessment of the students’ learning outcomes in ‘application skills’ and ‘judgement’? Third, does participation in the workshops lead to greater confidence by reviewers in their ability to assess students’ skills in application skills and judgement? Our findings suggest consensus moderation does reduce variability across graders and also builds grader confidence. 2015 Journal Article http://hdl.handle.net/20.500.11937/25986 10.1080/02602938.2015.1008398 http://www.tandfonline.com/ Routledge restricted |
| spellingShingle | O Connell, B. De Lange, Paul Freeman, M. Hancock, P. Abraham, A. Howieson, B. Watty, K. Does calibration reduce variability in the assessment of accounting learning outcomes? |
| title | Does calibration reduce variability in the assessment of accounting learning outcomes? |
| title_full | Does calibration reduce variability in the assessment of accounting learning outcomes? |
| title_fullStr | Does calibration reduce variability in the assessment of accounting learning outcomes? |
| title_full_unstemmed | Does calibration reduce variability in the assessment of accounting learning outcomes? |
| title_short | Does calibration reduce variability in the assessment of accounting learning outcomes? |
| title_sort | does calibration reduce variability in the assessment of accounting learning outcomes? |
| url | http://www.tandfonline.com/ http://hdl.handle.net/20.500.11937/25986 |