Specification and testing of hierarchical ordered response models with anchoring vignettes
Collection and analysis of self‐reported information on an ordered Likert scale is ubiquitous across the social sciences. Inference from such analyses is valid where the response scale employed means the same thing to all individuals. That is, if there is no differential item functioning (DIF) prese...
| Main Authors: | , , , |
|---|---|
| Format: | Journal Article |
| Published: |
Wiley-Blackwell
2020
|
| Online Access: | http://hdl.handle.net/20.500.11937/81792 |
| _version_ | 1848764419599761408 |
|---|---|
| author | Harris, Mark Greene, william Knott, R. Rice, N. |
| author_facet | Harris, Mark Greene, william Knott, R. Rice, N. |
| author_sort | Harris, Mark |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | Collection and analysis of self‐reported information on an ordered Likert scale is ubiquitous across the social sciences. Inference from such analyses is valid where the response scale employed means the same thing to all individuals. That is, if there is no differential item functioning (DIF) present in the data. A priori this is unlikely to hold across all individuals and cohorts in any sample of data. For this reason, anchoring vignettes have been proposed as a way to correct for DIF when individuals self‐assess their health (or well‐being, or satisfaction levels, or disability levels, etc.) on an ordered categorical scale. Using an example of self‐assessed pain, we illustrate the use of vignettes to adjust for DIF using the compound hierarchical ordered probit model (CHOPIT). The validity of this approach relies on the two underlying assumptions of response consistency (RC) and vignette equivalence (VE). Using a minor amendment to the specification of the standard CHOPIT model, we develop easy‐to‐implement score tests of the null hypothesis of RC and VE both separately and jointly. Monte Carlo simulations show that the tests have good size and power properties in finite samples. We illustrate the use of the tests by applying them to our empirical example. The tests should aid more robust analyses of self‐reported survey outcomes collected alongside anchoring vignettes. |
| first_indexed | 2025-11-14T11:19:03Z |
| format | Journal Article |
| id | curtin-20.500.11937-81792 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T11:19:03Z |
| publishDate | 2020 |
| publisher | Wiley-Blackwell |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-817922021-01-05T03:16:14Z Specification and testing of hierarchical ordered response models with anchoring vignettes Harris, Mark Greene, william Knott, R. Rice, N. Collection and analysis of self‐reported information on an ordered Likert scale is ubiquitous across the social sciences. Inference from such analyses is valid where the response scale employed means the same thing to all individuals. That is, if there is no differential item functioning (DIF) present in the data. A priori this is unlikely to hold across all individuals and cohorts in any sample of data. For this reason, anchoring vignettes have been proposed as a way to correct for DIF when individuals self‐assess their health (or well‐being, or satisfaction levels, or disability levels, etc.) on an ordered categorical scale. Using an example of self‐assessed pain, we illustrate the use of vignettes to adjust for DIF using the compound hierarchical ordered probit model (CHOPIT). The validity of this approach relies on the two underlying assumptions of response consistency (RC) and vignette equivalence (VE). Using a minor amendment to the specification of the standard CHOPIT model, we develop easy‐to‐implement score tests of the null hypothesis of RC and VE both separately and jointly. Monte Carlo simulations show that the tests have good size and power properties in finite samples. We illustrate the use of the tests by applying them to our empirical example. The tests should aid more robust analyses of self‐reported survey outcomes collected alongside anchoring vignettes. 2020 Journal Article http://hdl.handle.net/20.500.11937/81792 10.1111/rssa.12612 http://creativecommons.org/licenses/by/4.0/ Wiley-Blackwell fulltext |
| spellingShingle | Harris, Mark Greene, william Knott, R. Rice, N. Specification and testing of hierarchical ordered response models with anchoring vignettes |
| title | Specification and testing of hierarchical ordered response models with anchoring vignettes |
| title_full | Specification and testing of hierarchical ordered response models with anchoring vignettes |
| title_fullStr | Specification and testing of hierarchical ordered response models with anchoring vignettes |
| title_full_unstemmed | Specification and testing of hierarchical ordered response models with anchoring vignettes |
| title_short | Specification and testing of hierarchical ordered response models with anchoring vignettes |
| title_sort | specification and testing of hierarchical ordered response models with anchoring vignettes |
| url | http://hdl.handle.net/20.500.11937/81792 |