Listenership in human-agent collectives: a study of unidirectional instruction-giving
Research in nonverbal listenership behaviour and instruction-giving has focused on interaction with people while paying inadequate attention to human-agent interaction even as recent research indicates that, increasing pervasive computing is significantly changing how humans interact with intelligen...
| Main Author: | |
|---|---|
| Format: | Thesis (University of Nottingham only) |
| Language: | English |
| Published: |
2018
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/49255/ |
| _version_ | 1848797957854330880 |
|---|---|
| author | Ofemile, Abdulmalik Caxton Yusuf |
| author_facet | Ofemile, Abdulmalik Caxton Yusuf |
| author_sort | Ofemile, Abdulmalik Caxton Yusuf |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | Research in nonverbal listenership behaviour and instruction-giving has focused on interaction with people while paying inadequate attention to human-agent interaction even as recent research indicates that, increasing pervasive computing is significantly changing how humans interact with intelligent software agents and extending the boundaries of discourse to contexts including satellite navigation systems giving directions to drivers, self-checkout machines in supermarkets and intelligent personal assistants on smartphones. This thesis reports studies that use spontaneous listener facial actions and gestures to understand the nature and pattern of spontaneous nonverbal listenership behaviours, identification and communication in instruction-giving contexts. The research methodology used is as follows. Participants who are all L1 speakers of English (forty-eight in Study 1, six in Study 2) were tasked with assembling two Lego models using vague verbal instructions from a computer interface in Study 1 and a human instructor in Study 2 with a 15-minute time limit per iteration. The interface in study1uses three voices of which two are synthesised and one is non-synthesised human recording by a voice actor while Study 2 used a live human voice. A 24-hour long multimodal corpus was built and analysed from interactions between participants and the interface in Study 1 while a 3-hour multimodal corpus was developed from Study 2. The multimodal corpus was annotated for marked facial actions and gestures occurring at points when participants requested that instructors repeat instructions. Participant requests were nonverbal in HAI and a combination of nonverbal and verbal instructions in HHI contexts. The repetitions were quantified and classified into nine typologies. The results reveal key findings regarding the use of spontaneous nonverbal listenership behaviours as pragmatic markers indicating listener comprehension or incomprehension of instructions, perception of instructor-identities, projection of attitudes, meaning-development, task-execution strategies and interaction management even though, the agent could not attend to them in the same way a human can. Using these results, the thesis submits that there are potentials for applied linguistics theories and research to be used to identify and understand pathways to make agents more responsive to human behaviour, make human-agent interaction more credible and provides a theoretical foundation for future multidisciplinary research. |
| first_indexed | 2025-11-14T20:12:08Z |
| format | Thesis (University of Nottingham only) |
| id | nottingham-49255 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-14T20:12:08Z |
| publishDate | 2018 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-492552025-02-28T13:58:44Z https://eprints.nottingham.ac.uk/49255/ Listenership in human-agent collectives: a study of unidirectional instruction-giving Ofemile, Abdulmalik Caxton Yusuf Research in nonverbal listenership behaviour and instruction-giving has focused on interaction with people while paying inadequate attention to human-agent interaction even as recent research indicates that, increasing pervasive computing is significantly changing how humans interact with intelligent software agents and extending the boundaries of discourse to contexts including satellite navigation systems giving directions to drivers, self-checkout machines in supermarkets and intelligent personal assistants on smartphones. This thesis reports studies that use spontaneous listener facial actions and gestures to understand the nature and pattern of spontaneous nonverbal listenership behaviours, identification and communication in instruction-giving contexts. The research methodology used is as follows. Participants who are all L1 speakers of English (forty-eight in Study 1, six in Study 2) were tasked with assembling two Lego models using vague verbal instructions from a computer interface in Study 1 and a human instructor in Study 2 with a 15-minute time limit per iteration. The interface in study1uses three voices of which two are synthesised and one is non-synthesised human recording by a voice actor while Study 2 used a live human voice. A 24-hour long multimodal corpus was built and analysed from interactions between participants and the interface in Study 1 while a 3-hour multimodal corpus was developed from Study 2. The multimodal corpus was annotated for marked facial actions and gestures occurring at points when participants requested that instructors repeat instructions. Participant requests were nonverbal in HAI and a combination of nonverbal and verbal instructions in HHI contexts. The repetitions were quantified and classified into nine typologies. The results reveal key findings regarding the use of spontaneous nonverbal listenership behaviours as pragmatic markers indicating listener comprehension or incomprehension of instructions, perception of instructor-identities, projection of attitudes, meaning-development, task-execution strategies and interaction management even though, the agent could not attend to them in the same way a human can. Using these results, the thesis submits that there are potentials for applied linguistics theories and research to be used to identify and understand pathways to make agents more responsive to human behaviour, make human-agent interaction more credible and provides a theoretical foundation for future multidisciplinary research. 2018-03-15 Thesis (University of Nottingham only) NonPeerReviewed application/pdf en cc_by https://eprints.nottingham.ac.uk/49255/8/Abdulmalik%20Ofemile%20PhD%202018.pdf Ofemile, Abdulmalik Caxton Yusuf (2018) Listenership in human-agent collectives: a study of unidirectional instruction-giving. PhD thesis, University of Nottingham. Listenership; Human-Agent Interaction; Facial Actions; Gestures; Instruction-Giving;Multimodal Interaction; Assembly Task;Listening Comprehension; Vague Language; Nonverbal Feedback; Third Space of Hybridity; Hybrid space |
| spellingShingle | Listenership; Human-Agent Interaction; Facial Actions; Gestures; Instruction-Giving;Multimodal Interaction; Assembly Task;Listening Comprehension; Vague Language; Nonverbal Feedback; Third Space of Hybridity; Hybrid space Ofemile, Abdulmalik Caxton Yusuf Listenership in human-agent collectives: a study of unidirectional instruction-giving |
| title | Listenership in human-agent collectives: a study of unidirectional instruction-giving |
| title_full | Listenership in human-agent collectives: a study of unidirectional instruction-giving |
| title_fullStr | Listenership in human-agent collectives: a study of unidirectional instruction-giving |
| title_full_unstemmed | Listenership in human-agent collectives: a study of unidirectional instruction-giving |
| title_short | Listenership in human-agent collectives: a study of unidirectional instruction-giving |
| title_sort | listenership in human-agent collectives: a study of unidirectional instruction-giving |
| topic | Listenership; Human-Agent Interaction; Facial Actions; Gestures; Instruction-Giving;Multimodal Interaction; Assembly Task;Listening Comprehension; Vague Language; Nonverbal Feedback; Third Space of Hybridity; Hybrid space |
| url | https://eprints.nottingham.ac.uk/49255/ |