Designing formative and summative evaluation to capture the quality of performance skills in occupational therapy education
Introduction: The assessment of students’ skill perfor-mance is an often high- stakes task that is frequently reliant on expert examiners’ judgements. However, judgements are subject to bias, examiners may ‘fail to fail’ underper-forming students in person, and expert judgements provide little assis...
| Main Authors: | , |
|---|---|
| Format: | Conference Paper |
| Language: | English |
| Published: |
Wiley-Blackwell
2021
|
| Online Access: | http://hdl.handle.net/20.500.11937/84391 |
| _version_ | 1848764645337202688 |
|---|---|
| author | Waters, Rebecca Brentnall, Jennie |
| author_facet | Waters, Rebecca Brentnall, Jennie |
| author_sort | Waters, Rebecca |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | Introduction: The assessment of students’ skill perfor-mance is an often high- stakes task that is frequently reliant on expert examiners’ judgements. However, judgements are subject to bias, examiners may ‘fail to fail’ underper-forming students in person, and expert judgements provide little assistance for students developing their own evalua-tive judgement.Objectives: With occupational therapy students’ interview skills as the focus, this study: (i) interrogated the design of tools for the formative and summative evaluation of stu-dents’ skill performance, and (ii) created a tool to support actionable formative feedback, robust summative assess-ment, and shared understanding of qualitative performance characteristics.Method: In a reflexive action research cycle, we re- designed an interview skills checklist into a qualitative rubric using prior research, empirical data, shared experience, and a re-corded examiner consultation and practice session. We implemented the tools formatively for self, peer and/or exam-iner feedback in simulation programs across two universities, and evaluated a summative rubric in otherwise equivalent viva examinations with successive cohorts at one institution.Results: A rubric richly describing levels of performance in one cohort (n = 249) vastly improved the measure-ment of the quality of performance in a subsequent cohort (n=235) compared with a skills checklist and examiner judgement plus the Objective Borderline Method. The tools demonstrated utility in supporting students’ self and peer evaluation.Conclusion: Taking a novel approach to rubric design involving markedly shifting the presentation of performance levels refocussed the task from one of recording a judgement to one of evaluating a performance against commonly agreed criteria. |
| first_indexed | 2025-11-14T11:22:39Z |
| format | Conference Paper |
| id | curtin-20.500.11937-84391 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-14T11:22:39Z |
| publishDate | 2021 |
| publisher | Wiley-Blackwell |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-843912021-07-13T03:29:38Z Designing formative and summative evaluation to capture the quality of performance skills in occupational therapy education Waters, Rebecca Brentnall, Jennie Introduction: The assessment of students’ skill perfor-mance is an often high- stakes task that is frequently reliant on expert examiners’ judgements. However, judgements are subject to bias, examiners may ‘fail to fail’ underper-forming students in person, and expert judgements provide little assistance for students developing their own evalua-tive judgement.Objectives: With occupational therapy students’ interview skills as the focus, this study: (i) interrogated the design of tools for the formative and summative evaluation of stu-dents’ skill performance, and (ii) created a tool to support actionable formative feedback, robust summative assess-ment, and shared understanding of qualitative performance characteristics.Method: In a reflexive action research cycle, we re- designed an interview skills checklist into a qualitative rubric using prior research, empirical data, shared experience, and a re-corded examiner consultation and practice session. We implemented the tools formatively for self, peer and/or exam-iner feedback in simulation programs across two universities, and evaluated a summative rubric in otherwise equivalent viva examinations with successive cohorts at one institution.Results: A rubric richly describing levels of performance in one cohort (n = 249) vastly improved the measure-ment of the quality of performance in a subsequent cohort (n=235) compared with a skills checklist and examiner judgement plus the Objective Borderline Method. The tools demonstrated utility in supporting students’ self and peer evaluation.Conclusion: Taking a novel approach to rubric design involving markedly shifting the presentation of performance levels refocussed the task from one of recording a judgement to one of evaluating a performance against commonly agreed criteria. 2021 Conference Paper http://hdl.handle.net/20.500.11937/84391 10.1111/1440-1630.12737 English Wiley-Blackwell restricted |
| spellingShingle | Waters, Rebecca Brentnall, Jennie Designing formative and summative evaluation to capture the quality of performance skills in occupational therapy education |
| title | Designing formative and summative evaluation to capture the quality of performance skills in occupational therapy education |
| title_full | Designing formative and summative evaluation to capture the quality of performance skills in occupational therapy education |
| title_fullStr | Designing formative and summative evaluation to capture the quality of performance skills in occupational therapy education |
| title_full_unstemmed | Designing formative and summative evaluation to capture the quality of performance skills in occupational therapy education |
| title_short | Designing formative and summative evaluation to capture the quality of performance skills in occupational therapy education |
| title_sort | designing formative and summative evaluation to capture the quality of performance skills in occupational therapy education |
| url | http://hdl.handle.net/20.500.11937/84391 |