People often move their hands up and down to 'highlight' what they are saying. Are such 'beat gestures' important for communication? Researchers created words with an ambiguous stress pattern and asked listeners what they heard (DIScount or disCOUNT?). The beat gestures people saw influenced what they heard, showing that listeners quickly integrate verbal and visual information during speech recognition.
When politicians address an audience, they typically highlight important words with beat gestures, for example by moving their hands up and down.
In fact, we all seem to do it: Such 'flicks of the hands' are among the most common gestures in everyday conversations. People align these gestures very precisely to the prominent words in speech. But do beat gestures help listeners to understand the speaker? Researchers tested whether what we see shapes what we hear.
Speakers make use of different channels (mouth, hands, and face) to get a message across. Scientists want to understand how listeners make use of these different streams of information when they are listening to someone." In a well-known illusion called the 'McGurk effect', people hear a sound (like the 'b' in 'ba') as a different sound (for instance 'pa' or 'fa'), depending on the lip movements they see. But is there also a manual McGurk effect? Does what we hear depend on the gestures we see?
To investigate this question, the researchers chose a set of Dutch words that differed only in stress pattern. For instance, the word "PLAto" -- with stress on the first syllable -- refers to the philosopher from ancient Greece. However, "plaTO," pronounced with stress on the second syllable, refers to a plateau.
Participants watched a video of Bosker producing the words (with ambiguous stress) while making beat gestures ("Now I say the word ... plato"). Participants then had to decide which word they heard (PLAto or plaTEAU?). Would it matter whether beat gestures occurred at the first or second syllable?
Listeners were more likely to hear stress on a syllable if there was a beat gesture on that syllable. This 'manual McGurk effect' occurred for both words and non-words ("BAAGpif" or "baagPIF"?).
Even more surprisingly, beat gestures influenced what vowel people heard (long or short 'a' in 'baagpif / bagpif'), as vowel length is typically associated with the stress pattern of a word.
ความคิดเห็น