Reviewing and extending the five-user assumption: a grounded procedure for interaction evaluation
The debate concerning how many participants represents a sufficient number for interaction testing is well-established and long-running, with prominent contributions arguing that five users provide a good benchmark when seeking to discover interaction problems. We argue that adoption of five users i...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Published: |
ACM
2013
|
| Online Access: | https://eprints.nottingham.ac.uk/29453/ |
| _version_ | 1848793788718252032 |
|---|---|
| author | Borsci, Simone Macredie, Robert D. Barnett, Julie Martin, Jennifer L. Kuljis, Jasna Young, Terry |
| author_facet | Borsci, Simone Macredie, Robert D. Barnett, Julie Martin, Jennifer L. Kuljis, Jasna Young, Terry |
| author_sort | Borsci, Simone |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | The debate concerning how many participants represents a sufficient number for interaction testing is well-established and long-running, with prominent contributions arguing that five users provide a good benchmark when seeking to discover interaction problems. We argue that adoption of five users in this context is often done with little understanding of the basis for, or implications of, the decision. We present an analysis of relevant research to clarify the meaning of the five-user assumption and to examine the way in which the original research that suggested it has been applied. This includes its blind adoption and application in some studies, and complaints about its inadequacies in others. We argue that the five-user assumption is often misunderstood, not only in the field of Human-Computer Interaction, but also in fields such as medical device design, or in business and information applications. The analysis that we present allows us to define a systematic approach for monitoring the sample discovery likelihood, in formative and summative evaluations, and for gathering information in order to make critical decisions during the interaction testing, while respecting the aim of the evaluation and allotted budget. This approach -- which we call the Grounded Procedure -- is introduced and its value argued. |
| first_indexed | 2025-11-14T19:05:52Z |
| format | Article |
| id | nottingham-29453 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| last_indexed | 2025-11-14T19:05:52Z |
| publishDate | 2013 |
| publisher | ACM |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-294532020-05-04T16:39:04Z https://eprints.nottingham.ac.uk/29453/ Reviewing and extending the five-user assumption: a grounded procedure for interaction evaluation Borsci, Simone Macredie, Robert D. Barnett, Julie Martin, Jennifer L. Kuljis, Jasna Young, Terry The debate concerning how many participants represents a sufficient number for interaction testing is well-established and long-running, with prominent contributions arguing that five users provide a good benchmark when seeking to discover interaction problems. We argue that adoption of five users in this context is often done with little understanding of the basis for, or implications of, the decision. We present an analysis of relevant research to clarify the meaning of the five-user assumption and to examine the way in which the original research that suggested it has been applied. This includes its blind adoption and application in some studies, and complaints about its inadequacies in others. We argue that the five-user assumption is often misunderstood, not only in the field of Human-Computer Interaction, but also in fields such as medical device design, or in business and information applications. The analysis that we present allows us to define a systematic approach for monitoring the sample discovery likelihood, in formative and summative evaluations, and for gathering information in order to make critical decisions during the interaction testing, while respecting the aim of the evaluation and allotted budget. This approach -- which we call the Grounded Procedure -- is introduced and its value argued. ACM 2013-11-01 Article PeerReviewed Borsci, Simone, Macredie, Robert D., Barnett, Julie, Martin, Jennifer L., Kuljis, Jasna and Young, Terry (2013) Reviewing and extending the five-user assumption: a grounded procedure for interaction evaluation. ACM Transactions on Computer-Human Interaction (TOCHI), 20 (5). 29/1-29/23. ISSN 1073-0516 http://dl.acm.org/citation.cfm?id=2506210&CFID=697895576&CFTOKEN=48685461 doi:10.1145/2506210 doi:10.1145/2506210 |
| spellingShingle | Borsci, Simone Macredie, Robert D. Barnett, Julie Martin, Jennifer L. Kuljis, Jasna Young, Terry Reviewing and extending the five-user assumption: a grounded procedure for interaction evaluation |
| title | Reviewing and extending the five-user assumption: a grounded procedure for interaction evaluation |
| title_full | Reviewing and extending the five-user assumption: a grounded procedure for interaction evaluation |
| title_fullStr | Reviewing and extending the five-user assumption: a grounded procedure for interaction evaluation |
| title_full_unstemmed | Reviewing and extending the five-user assumption: a grounded procedure for interaction evaluation |
| title_short | Reviewing and extending the five-user assumption: a grounded procedure for interaction evaluation |
| title_sort | reviewing and extending the five-user assumption: a grounded procedure for interaction evaluation |
| url | https://eprints.nottingham.ac.uk/29453/ https://eprints.nottingham.ac.uk/29453/ https://eprints.nottingham.ac.uk/29453/ |