Seeing Is Believing: Neural Representations of Visual Stimuli in Human Auditory Cortex Correlate with Illusory Auditory Perceptions

Seeing Is Believing: Neural Representations of Visual Stimuli in Human Auditory Cortex Correlate with Illusory Auditory Perceptions

In interpersonal communication, the listener can often see as well as hear the speaker. Visual stimuli can subtly change a listener’s auditory perception, as in the McGurk illusion, in which perception of a phoneme’s auditory identity is changed by a concurrent video of a mouth articulating a differ...

Full description

Bibliographic Details
Main Authors: Smith, Elliot, Duede, Scott, Hanrahan, Sara, Davis, Tyler, House, Paul, Greger, Bradley
Format: Online
Language:English
Published: Public Library of Science 2013
Online Access:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3762867/
id pubmed-3762867
recordtype oai_dc
spelling pubmed-37628672013-09-10 Seeing Is Believing: Neural Representations of Visual Stimuli in Human Auditory Cortex Correlate with Illusory Auditory Perceptions Smith, Elliot Duede, Scott Hanrahan, Sara Davis, Tyler House, Paul Greger, Bradley Research Article In interpersonal communication, the listener can often see as well as hear the speaker. Visual stimuli can subtly change a listener’s auditory perception, as in the McGurk illusion, in which perception of a phoneme’s auditory identity is changed by a concurrent video of a mouth articulating a different phoneme. Studies have yet to link visual influences on the neural representation of language with subjective language perception. Here we show that vision influences the electrophysiological representation of phonemes in human auditory cortex prior to the presentation of the auditory stimulus. We used the McGurk effect to dissociate the subjective perception of phonemes from the auditory stimuli. With this paradigm we demonstrate that neural representations in auditory cortex are more closely correlated with the visual stimuli of mouth articulation, which drive the illusory subjective auditory perception, than the actual auditory stimuli. Additionally, information about visual and auditory stimuli transfer in the caudal–rostral direction along the superior temporal gyrus during phoneme perception as would be expected of visual information flowing from the occipital cortex into the ventral auditory processing stream. These results show that visual stimuli influence the neural representation in auditory cortex early in sensory processing and may override the subjective auditory perceptions normally generated by auditory stimuli. These findings depict a marked influence of vision on the neural processing of audition in tertiary auditory cortex and suggest a mechanistic underpinning for the McGurk effect. Public Library of Science 2013-09-04 /pmc/articles/PMC3762867/ /pubmed/24023823 http://dx.doi.org/10.1371/journal.pone.0073148 Text en https://creativecommons.org/publicdomain/zero/1.0/ This is an open-access article distributed under the terms of the Creative Commons Public Domain declaration, which stipulates that, once placed in the public domain, this work may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose.
repository_type Open Access Journal
institution_category Foreign Institution
institution US National Center for Biotechnology Information
building NCBI PubMed
collection Online Access
language English
format Online
author Smith, Elliot
Duede, Scott
Hanrahan, Sara
Davis, Tyler
House, Paul
Greger, Bradley
spellingShingle Smith, Elliot
Duede, Scott
Hanrahan, Sara
Davis, Tyler
House, Paul
Greger, Bradley
Seeing Is Believing: Neural Representations of Visual Stimuli in Human Auditory Cortex Correlate with Illusory Auditory Perceptions
author_facet Smith, Elliot
Duede, Scott
Hanrahan, Sara
Davis, Tyler
House, Paul
Greger, Bradley
author_sort Smith, Elliot
title Seeing Is Believing: Neural Representations of Visual Stimuli in Human Auditory Cortex Correlate with Illusory Auditory Perceptions
title_short Seeing Is Believing: Neural Representations of Visual Stimuli in Human Auditory Cortex Correlate with Illusory Auditory Perceptions
title_full Seeing Is Believing: Neural Representations of Visual Stimuli in Human Auditory Cortex Correlate with Illusory Auditory Perceptions
title_fullStr Seeing Is Believing: Neural Representations of Visual Stimuli in Human Auditory Cortex Correlate with Illusory Auditory Perceptions
title_full_unstemmed Seeing Is Believing: Neural Representations of Visual Stimuli in Human Auditory Cortex Correlate with Illusory Auditory Perceptions
title_sort seeing is believing: neural representations of visual stimuli in human auditory cortex correlate with illusory auditory perceptions
description In interpersonal communication, the listener can often see as well as hear the speaker. Visual stimuli can subtly change a listener’s auditory perception, as in the McGurk illusion, in which perception of a phoneme’s auditory identity is changed by a concurrent video of a mouth articulating a different phoneme. Studies have yet to link visual influences on the neural representation of language with subjective language perception. Here we show that vision influences the electrophysiological representation of phonemes in human auditory cortex prior to the presentation of the auditory stimulus. We used the McGurk effect to dissociate the subjective perception of phonemes from the auditory stimuli. With this paradigm we demonstrate that neural representations in auditory cortex are more closely correlated with the visual stimuli of mouth articulation, which drive the illusory subjective auditory perception, than the actual auditory stimuli. Additionally, information about visual and auditory stimuli transfer in the caudal–rostral direction along the superior temporal gyrus during phoneme perception as would be expected of visual information flowing from the occipital cortex into the ventral auditory processing stream. These results show that visual stimuli influence the neural representation in auditory cortex early in sensory processing and may override the subjective auditory perceptions normally generated by auditory stimuli. These findings depict a marked influence of vision on the neural processing of audition in tertiary auditory cortex and suggest a mechanistic underpinning for the McGurk effect.
publisher Public Library of Science
publishDate 2013
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3762867/
_version_ 1612008309636726784

Similar Items