What Happens In Your Brain When You Hear With Your Eyes? Neuroscientists Examine The McGurk Effect
Your hearing might not be as great as you think it is. And you can let your eyes take the blame. It’s what’s known as the McGurk Effect.
Unless you’re an avid watcher of dubbed movies, you’ve most likely experienced it. This video clip from the BBC illustrates how when sounds aren’t accompanied by the appropriate visuals, you may trust your eyes more than your ears.
Read: Audio Illusions Prove Your Hearing May Not Be As Good As You Thought
Although the McGurk Effect was discovered over four decades ago by two psychologists, new research published in PLOS Computational Biology gives insight into why sometimes we hear with our eyes. By creating an updated algorithm to predict how people would, or would not, be fooled by audiovisual speech syllables, the researchers discovered a better understanding of why incongruent audiovisual speech occurs.
"Understanding how the brain combines information from multiple senses will provide insight into ways to improve declines in speech perception due to typical aging and even to develop devices that could enhance hearing across the lifespan," senior author Dr. Michael Beauchamp, told MedicalXpress.
Despite the McGurk Effect being cited in thousands of papers since it was discovered, it was still unclear why the effect happens with some syllables and not others, which is what the authors sought to discover.
Their model was based on previous models; however, it was more comprehensive because it included the principle of causal inference. In their paper, the authors describe causal inference as “given a particular pair of auditory and visual syllables, the brain calculates the likelihood they are from a single vs. multiple talkers and uses this likelihood to determine the final speech percept.”
Their findings concluded causal inference to be a key component to incorporate in their algorithm and suggest it to be included in future research involving audiovisual speech.
See also: Brain Fart: How Auditory Illusions Trick Your Brain Into Hearing Something It's Not
Audio Illusion Reveals The Power Of Priming, The Brain’s Natural Tendency To Make Things Stick