Judging behaviour and rater errors: an application of the many-facet rasch model.
Of the potential sources of construct irrelevant variance or unwanted variability in performance assessment, those associated with raters have been found to be extensive, difficult to control, and impossible to eliminate. And as rater-related errors are non-trivial and threaten the validity of test...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Penerbit UKM
2011
|
| Online Access: | http://journalarticle.ukm.my/2769/ http://journalarticle.ukm.my/2769/1/pp179_197.pdf |
| _version_ | 1848809970467864576 |
|---|---|
| author | Noor Lide Abu Kassim, |
| author_facet | Noor Lide Abu Kassim, |
| author_sort | Noor Lide Abu Kassim, |
| building | UKM Institutional Repository |
| collection | Online Access |
| description | Of the potential sources of construct irrelevant variance or unwanted variability in performance assessment, those associated with raters have been found to be extensive,
difficult to control, and impossible to eliminate. And as rater-related errors are non-trivial and threaten the validity of test results, it is necessary that these errors are accounted for and controlled in some way. This paper explains the different types of rater errors and illustrates how they can be identified using the Many-facet Rasch Model, as implemented by FACETS. It also demonstrates what these errors mean in terms of actual judging or
rating behaviour and elucidates how they may affect the accuracy of estimation of performance. Rater errors that are explicated in this paper are those related to rater
severity, restriction of range, central tendency, and internal consistency. As assessment and its procedures are central to student learning, matters related to valid and fair testing need to be taken seriously. It is hoped that with greater awareness of how we judge and a better understanding of how rater-related errors are introduced into the assessment process, we can be better raters and better teachers. |
| first_indexed | 2025-11-14T23:23:04Z |
| format | Article |
| id | oai:generic.eprints.org:2769 |
| institution | Universiti Kebangasaan Malaysia |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-14T23:23:04Z |
| publishDate | 2011 |
| publisher | Penerbit UKM |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | oai:generic.eprints.org:27692016-12-14T06:32:35Z http://journalarticle.ukm.my/2769/ Judging behaviour and rater errors: an application of the many-facet rasch model. Noor Lide Abu Kassim, Of the potential sources of construct irrelevant variance or unwanted variability in performance assessment, those associated with raters have been found to be extensive, difficult to control, and impossible to eliminate. And as rater-related errors are non-trivial and threaten the validity of test results, it is necessary that these errors are accounted for and controlled in some way. This paper explains the different types of rater errors and illustrates how they can be identified using the Many-facet Rasch Model, as implemented by FACETS. It also demonstrates what these errors mean in terms of actual judging or rating behaviour and elucidates how they may affect the accuracy of estimation of performance. Rater errors that are explicated in this paper are those related to rater severity, restriction of range, central tendency, and internal consistency. As assessment and its procedures are central to student learning, matters related to valid and fair testing need to be taken seriously. It is hoped that with greater awareness of how we judge and a better understanding of how rater-related errors are introduced into the assessment process, we can be better raters and better teachers. Penerbit UKM 2011-09 Article PeerReviewed application/pdf en http://journalarticle.ukm.my/2769/1/pp179_197.pdf Noor Lide Abu Kassim, (2011) Judging behaviour and rater errors: an application of the many-facet rasch model. GEMA: Online Journal of Language Studies, 11 (3). pp. 179-197. ISSN 1675-8021 (In Press) http://www.ukm.my/ppbl/Gema/gemahome.html. |
| spellingShingle | Noor Lide Abu Kassim, Judging behaviour and rater errors: an application of the many-facet rasch model. |
| title | Judging behaviour and rater errors: an application of the many-facet rasch model. |
| title_full | Judging behaviour and rater errors: an application of the many-facet rasch model. |
| title_fullStr | Judging behaviour and rater errors: an application of the many-facet rasch model. |
| title_full_unstemmed | Judging behaviour and rater errors: an application of the many-facet rasch model. |
| title_short | Judging behaviour and rater errors: an application of the many-facet rasch model. |
| title_sort | judging behaviour and rater errors: an application of the many-facet rasch model. |
| url | http://journalarticle.ukm.my/2769/ http://journalarticle.ukm.my/2769/ http://journalarticle.ukm.my/2769/1/pp179_197.pdf |