Message vs. messenger effects on cross-modal matching for spoken phrases
A core issue in speech perception and word recognition research is the nature of information perceivers use to identify spoken utterances across indexical variations in their phonetic details, such as talker and accent differences. Separately, a crucial question in audio-visual research is the natur...
| Main Authors: | , , , , , |
|---|---|
| Other Authors: | |
| Format: | Conference Paper |
| Published: |
ISCA
2015
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/20.500.11937/45277 |
| _version_ | 1848757237434023936 |
|---|---|
| author | Best, C. Kroos, Christian Mulak, K. Halovic, S. Fort, M. Kitamura, C. |
| author2 | - |
| author_facet | - Best, C. Kroos, Christian Mulak, K. Halovic, S. Fort, M. Kitamura, C. |
| author_sort | Best, C. |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | A core issue in speech perception and word recognition research is the nature of information perceivers use to identify spoken utterances across indexical variations in their phonetic details, such as talker and accent differences. Separately, a crucial question in audio-visual research is the nature of information perceivers use to recognize phonetic congruency between the audio and visual (talking face) signals that arise from speaking. We combined these issues in a study examining how differences between connected speech utterances (messages) versus between talkers and accents (messenger characteristics) contribute to recognition of crossmodal articulatory congruence between audio-only (AO) and video-only (VO) components of spoken utterances. Participants heard AO phrases in their native regional English accent or another English accent, and then saw two synchronous VO displays of point-light talking faces from which they had to select the one that corresponded to the audio target. The incorrect video in each pair was either the same or a different phrase as the audio target, produced by the same or a different talker, who spoke in either the same or a different English accent. Results indicate that cross-modal articulatory correspondence is more accurately and quickly detected for message content than for messenger details, suggesting that recognising the linguistic message is more fundamental than messenger features is to cross-modal detection of audio-visual articulatory congruency. Nonetheless, messenger characteristics, especially accent, affected performance to some degree, analogous to recent findings in AO speech research. |
| first_indexed | 2025-11-14T09:24:54Z |
| format | Conference Paper |
| id | curtin-20.500.11937-45277 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T09:24:54Z |
| publishDate | 2015 |
| publisher | ISCA |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-452772017-01-30T15:19:48Z Message vs. messenger effects on cross-modal matching for spoken phrases Best, C. Kroos, Christian Mulak, K. Halovic, S. Fort, M. Kitamura, C. - cross-modal congruency articulatory information point-light talkers talker and accent effects A core issue in speech perception and word recognition research is the nature of information perceivers use to identify spoken utterances across indexical variations in their phonetic details, such as talker and accent differences. Separately, a crucial question in audio-visual research is the nature of information perceivers use to recognize phonetic congruency between the audio and visual (talking face) signals that arise from speaking. We combined these issues in a study examining how differences between connected speech utterances (messages) versus between talkers and accents (messenger characteristics) contribute to recognition of crossmodal articulatory congruence between audio-only (AO) and video-only (VO) components of spoken utterances. Participants heard AO phrases in their native regional English accent or another English accent, and then saw two synchronous VO displays of point-light talking faces from which they had to select the one that corresponded to the audio target. The incorrect video in each pair was either the same or a different phrase as the audio target, produced by the same or a different talker, who spoke in either the same or a different English accent. Results indicate that cross-modal articulatory correspondence is more accurately and quickly detected for message content than for messenger details, suggesting that recognising the linguistic message is more fundamental than messenger features is to cross-modal detection of audio-visual articulatory congruency. Nonetheless, messenger characteristics, especially accent, affected performance to some degree, analogous to recent findings in AO speech research. 2015 Conference Paper http://hdl.handle.net/20.500.11937/45277 ISCA restricted |
| spellingShingle | cross-modal congruency articulatory information point-light talkers talker and accent effects Best, C. Kroos, Christian Mulak, K. Halovic, S. Fort, M. Kitamura, C. Message vs. messenger effects on cross-modal matching for spoken phrases |
| title | Message vs. messenger effects on cross-modal matching for spoken phrases |
| title_full | Message vs. messenger effects on cross-modal matching for spoken phrases |
| title_fullStr | Message vs. messenger effects on cross-modal matching for spoken phrases |
| title_full_unstemmed | Message vs. messenger effects on cross-modal matching for spoken phrases |
| title_short | Message vs. messenger effects on cross-modal matching for spoken phrases |
| title_sort | message vs. messenger effects on cross-modal matching for spoken phrases |
| topic | cross-modal congruency articulatory information point-light talkers talker and accent effects |
| url | http://hdl.handle.net/20.500.11937/45277 |