Developing a research design for comparative evaluation of marking and feedback support systems
Marking and provision of formative feedback on student assessment items are essential but onerous and potentially error prone activities in teaching and learning. Marking and Feedback Support Systems (MFSS) aim to improve the efficiency and effectiveness of human (not automated) marking and provisio...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Conference Paper |
| Published: |
Murdoch University
2012
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/20.500.11937/11857 |
| _version_ | 1848747917402177536 |
|---|---|
| author | Venable, John Aitken, Ashley Chang, Vanessa Dreher, Heinz Issa, Tomayess Von Konsky, Brian Wood, Lincoln |
| author2 | Roger Atkinson |
| author_facet | Roger Atkinson Venable, John Aitken, Ashley Chang, Vanessa Dreher, Heinz Issa, Tomayess Von Konsky, Brian Wood, Lincoln |
| author_sort | Venable, John |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | Marking and provision of formative feedback on student assessment items are essential but onerous and potentially error prone activities in teaching and learning. Marking and Feedback Support Systems (MFSS) aim to improve the efficiency and effectiveness of human (not automated) marking and provision of feedback, resulting in reduced marking time, improved accuracy of marks, improved student satisfaction with feedback, and improved student learning. This paper highlights issues in rigorous evaluation of MFSS, including potential confounding variables as well as ethical issues relating to fairness of actual student assessments during evaluation. To address these issues the paper proposes an evaluation research approach, which combines artificial evaluation in the form of a controlled field experiment with naturalistic evaluation in the form of a field study, with the evaluation to be conducted through the live application of the MFSS being evaluated on a variety of units, assessment items, and marking schemes. The controlled field experiment approach requires the assessment item for each student to be marked once each using each MFSS together with a manual (non-MFSS) marking control method. It also requires markers to use all the MFSS as well as the manual method. Through such a design, the results of the comparative evaluation will facilitate design-based education research to further develop MFSS with the overall goal of more efficient and effective assessment and feedback systems and practices to enhance teaching and learning. |
| first_indexed | 2025-11-14T06:56:46Z |
| format | Conference Paper |
| id | curtin-20.500.11937-11857 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T06:56:46Z |
| publishDate | 2012 |
| publisher | Murdoch University |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-118572023-02-02T07:57:37Z Developing a research design for comparative evaluation of marking and feedback support systems Venable, John Aitken, Ashley Chang, Vanessa Dreher, Heinz Issa, Tomayess Von Konsky, Brian Wood, Lincoln Roger Atkinson Clare McBeath Research - Design Teaching Technology Evaluation Marking and Feedback Support System Marking and provision of formative feedback on student assessment items are essential but onerous and potentially error prone activities in teaching and learning. Marking and Feedback Support Systems (MFSS) aim to improve the efficiency and effectiveness of human (not automated) marking and provision of feedback, resulting in reduced marking time, improved accuracy of marks, improved student satisfaction with feedback, and improved student learning. This paper highlights issues in rigorous evaluation of MFSS, including potential confounding variables as well as ethical issues relating to fairness of actual student assessments during evaluation. To address these issues the paper proposes an evaluation research approach, which combines artificial evaluation in the form of a controlled field experiment with naturalistic evaluation in the form of a field study, with the evaluation to be conducted through the live application of the MFSS being evaluated on a variety of units, assessment items, and marking schemes. The controlled field experiment approach requires the assessment item for each student to be marked once each using each MFSS together with a manual (non-MFSS) marking control method. It also requires markers to use all the MFSS as well as the manual method. Through such a design, the results of the comparative evaluation will facilitate design-based education research to further develop MFSS with the overall goal of more efficient and effective assessment and feedback systems and practices to enhance teaching and learning. 2012 Conference Paper http://hdl.handle.net/20.500.11937/11857 Murdoch University fulltext |
| spellingShingle | Research - Design Teaching Technology Evaluation Marking and Feedback Support System Venable, John Aitken, Ashley Chang, Vanessa Dreher, Heinz Issa, Tomayess Von Konsky, Brian Wood, Lincoln Developing a research design for comparative evaluation of marking and feedback support systems |
| title | Developing a research design for comparative evaluation of marking and feedback support systems |
| title_full | Developing a research design for comparative evaluation of marking and feedback support systems |
| title_fullStr | Developing a research design for comparative evaluation of marking and feedback support systems |
| title_full_unstemmed | Developing a research design for comparative evaluation of marking and feedback support systems |
| title_short | Developing a research design for comparative evaluation of marking and feedback support systems |
| title_sort | developing a research design for comparative evaluation of marking and feedback support systems |
| topic | Research - Design Teaching Technology Evaluation Marking and Feedback Support System |
| url | http://hdl.handle.net/20.500.11937/11857 |