You have to try this – the McGurk Effect
This is one of my favorite illusions, and it is a really interesting demonstration of something that our brains do all the time, every single day. However, the demonstration will be slightly less interesting if you do things out of order, so please so please follow my instructions exactly to get the full effect.
Okay…check that the sound on your computer is on and loud enough that you can hear it (without hurting your ears, obviously).
Once the video below has loaded so that you see a man’s face with a play symbol over the top, put your cursor over the play symbol. Then, CLOSE YOUR EYES and click your mouse to make the video start. Listen to what you hear.
What sound was the man repeating over and over?
The video clip is extremely short, so you may want to put your cursor over the word ‘Replay’ in the upper left corner of the video, close your eyes, and then click to play it again one or two more times until you know for sure which sound you are hearing.
You should have heard ‘Ba ba ba ba ba ba,’ like ‘Bob’ over and over without the final b. Is this what you heard?
Now, turn the sound on your computer OFF.
Go back to the video above (I won’t insert the video again below because then you may not trust me that it is the same one), and click ‘Replay’ again. This time, try to lip read and see what the man is saying without the sound. Replay the video a few times to double check.
Does it look like he is saying ‘ba ba ba ba ba ba’?
If you aren’t sure, watch the video without sound again, this time saying, ‘ba ba ba ba ba ba’ to yourself to see if your mouth moves the same way as the man in the video.
Hopefully you think it looks like he is saying something different. Can you tell what he is saying by looking at his mouth move?
You may be able to tell that his mouth is moving to say ‘da da da da da da,’ as in ‘dollar’ without the ‘llar’. This is correct.
Now turn the sound on your computer back ON. Watch the video one more time (or as many times as you like) with the sound on and your eyes open.
What does the man in the video appear to be saying now? Watch it as many times as you like, thinking about the sound you hear and the way he is moving his mouth.
Do you hear ‘ga ga ga ga ga ga,’ as in ‘golly’ without the ‘lly’?
Hopefully this is what you hear. If you weren’t sure, play the video again with your eyes open and the sound on, and see what you think. I think ‘ga ga ga ga ga ga’ should be the closest fit.
So why do we hear three different sounds from the same audio depending on whether we just listen, just watch, or do both at the same time?
This illusion was created by showing a person saying ‘da’ while playing the audio clip of him saying ‘ba’. This is why you heard ‘ba ba ba ba ba ba’ when you listened without watching, and why you saw him saying ‘da da da da da da’ when you watched without listening.
When you listened and watched at the same time, you heard a sound (‘ga’) which is halfway in between ‘ba’ and ‘da’ in terms of the way the sounds are formed in our mouth and the way they sound.
The McGurk Effect is an example of the way our brains work to integrate the stimuli we encounter every day. We do not hear sounds based purely on the audio information which enters our ears. Whenever possible, we link this information with the way a person’s mouth moves, what they point at, their gestures, and our pre-existing knowledge.
The McGurk Effect is therefore often used as evidence that our brains do not function in a completely modular way. For example (as in the McGurk Effect), our brains do not process sounds completely independently from sights. Although different areas of our brains are specialized to deal with different types of information, this demonstration suggests that the information is somehow integrated in order to give us the best possible interpretation. For example, in a noisy room, we may be able to hear someone better if we can also see the way the person’s mouth is moving. However, it can also lead to confusion when there is a mis-match between the stimuli that enters our ears and what we expect to hear, like in this demonstration, which is an extreme example because the sound and the mouth movement really are different.
I would love to know whether the demonstration worked for you, as it has always worked great when I have done it in courses with students…but it is harder to do in a blog post. In addition, it is always fun to see if other people have heard/seen the same thing you have, but we can’t do that by a show of hands online. This is why I have included the very quick poll at the bottom of this post. If you wish to participate, simply click on your answer and click ‘Submit’. Then you can click ‘View Results’ to see what other people have said. It is totally anonymous, so no one else can see who has answered, but it should be interesting to see what everyone heard (shown in terms of percentages, not names).
For my other ‘You have to try this’ post, check out: