Think aloud: using cognitive interviewing to validate the PISA assessment of student self-efficacy in mathematics
Cognitive interviewing (CI) provides a method of systematically collecting validity evidence of response processes for questionnaire items. CI involves a range of techniques for prompting individuals to verbalise their responses to items. One such technique is concurrent verbalisation, as developed...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Published: |
Taylor & Francis
2016
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/38673/ |
| _version_ | 1848795665327456256 |
|---|---|
| author | Pepper, David Hodgen, Jeremy Lamesoo, Katri Kõiv, Pille Tolboom, Jos |
| author_facet | Pepper, David Hodgen, Jeremy Lamesoo, Katri Kõiv, Pille Tolboom, Jos |
| author_sort | Pepper, David |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | Cognitive interviewing (CI) provides a method of systematically collecting validity evidence of response processes for questionnaire items. CI involves a range of techniques for prompting individuals to verbalise their responses to items. One such technique is concurrent verbalisation, as developed in Think Aloud Protocol (TAP). This article investigates the value of the technique for validating questionnaire items administered to young people in international surveys. To date, the literature on TAP has focused on allaying concerns about reactivity – whether response processes are affected by thinking aloud. This article investigates another concern, namely the completeness of concurrent verbalisations – the extent to which respondents verbalise their response processes.
An independent, exploratory validation of the PISA assessment of student self-efficacy in mathematics by a small international team of researchers using CI with concurrent verbalisation in four education systems (England, Estonia, Hong Kong, and the Netherlands) provided the basis for this investigation. The researchers found that students generally thought aloud in response to each of the items, thereby providing validity evidence of responses processes varying within and between the education systems, but that practical steps could be taken to increase the completeness of concurrent verbalisations in future validations. |
| first_indexed | 2025-11-14T19:35:42Z |
| format | Article |
| id | nottingham-38673 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| last_indexed | 2025-11-14T19:35:42Z |
| publishDate | 2016 |
| publisher | Taylor & Francis |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-386732020-05-04T18:17:13Z https://eprints.nottingham.ac.uk/38673/ Think aloud: using cognitive interviewing to validate the PISA assessment of student self-efficacy in mathematics Pepper, David Hodgen, Jeremy Lamesoo, Katri Kõiv, Pille Tolboom, Jos Cognitive interviewing (CI) provides a method of systematically collecting validity evidence of response processes for questionnaire items. CI involves a range of techniques for prompting individuals to verbalise their responses to items. One such technique is concurrent verbalisation, as developed in Think Aloud Protocol (TAP). This article investigates the value of the technique for validating questionnaire items administered to young people in international surveys. To date, the literature on TAP has focused on allaying concerns about reactivity – whether response processes are affected by thinking aloud. This article investigates another concern, namely the completeness of concurrent verbalisations – the extent to which respondents verbalise their response processes. An independent, exploratory validation of the PISA assessment of student self-efficacy in mathematics by a small international team of researchers using CI with concurrent verbalisation in four education systems (England, Estonia, Hong Kong, and the Netherlands) provided the basis for this investigation. The researchers found that students generally thought aloud in response to each of the items, thereby providing validity evidence of responses processes varying within and between the education systems, but that practical steps could be taken to increase the completeness of concurrent verbalisations in future validations. Taylor & Francis 2016-10-13 Article PeerReviewed Pepper, David, Hodgen, Jeremy, Lamesoo, Katri, Kõiv, Pille and Tolboom, Jos (2016) Think aloud: using cognitive interviewing to validate the PISA assessment of student self-efficacy in mathematics. International Journal of Research & Method in Education . pp. 1-14. ISSN 1743-7288 Cognitive interviewing think aloud validation assessment PISA international self-efficacy mathematics http://www.tandfonline.com/doi/full/10.1080/1743727X.2016.1238891 doi:10.1080/1743727X.2016.1238891 doi:10.1080/1743727X.2016.1238891 |
| spellingShingle | Cognitive interviewing think aloud validation assessment PISA international self-efficacy mathematics Pepper, David Hodgen, Jeremy Lamesoo, Katri Kõiv, Pille Tolboom, Jos Think aloud: using cognitive interviewing to validate the PISA assessment of student self-efficacy in mathematics |
| title | Think aloud: using cognitive interviewing to validate the PISA assessment of student self-efficacy in mathematics |
| title_full | Think aloud: using cognitive interviewing to validate the PISA assessment of student self-efficacy in mathematics |
| title_fullStr | Think aloud: using cognitive interviewing to validate the PISA assessment of student self-efficacy in mathematics |
| title_full_unstemmed | Think aloud: using cognitive interviewing to validate the PISA assessment of student self-efficacy in mathematics |
| title_short | Think aloud: using cognitive interviewing to validate the PISA assessment of student self-efficacy in mathematics |
| title_sort | think aloud: using cognitive interviewing to validate the pisa assessment of student self-efficacy in mathematics |
| topic | Cognitive interviewing think aloud validation assessment PISA international self-efficacy mathematics |
| url | https://eprints.nottingham.ac.uk/38673/ https://eprints.nottingham.ac.uk/38673/ https://eprints.nottingham.ac.uk/38673/ |