Format

Send to

Choose Destination
J Exp Psychol Hum Percept Perform. 2011 Aug;37(4):1193-209. doi: 10.1037/a0023100.

Psychophysics of the McGurk and other audiovisual speech integration effects.

Author information

1
Division of Communication and Auditory Neuroscience, House Ear Institute, Los Angeles, California, USA. jjiang@gwu.edu

Abstract

When the auditory and visual components of spoken audiovisual nonsense syllables are mismatched, perceivers produce four different types of perceptual responses, auditory correct, visual correct, fusion (the so-called McGurk effect), and combination (i.e., two consonants are reported). Here, quantitative measures were developed to account for the distribution of the four types of perceptual responses to 384 different stimuli from four talkers. The measures included mutual information, correlations, and acoustic measures, all representing audiovisual stimulus relationships. In Experiment 1, open-set perceptual responses were obtained for acoustic /bɑ/ or /lɑ/ dubbed to video /bɑ, dɑ, gɑ, vɑ, zɑ, lɑ, wɑ, ðɑ/. The talker, the video syllable, and the acoustic syllable significantly influenced the type of response. In Experiment 2, the best predictors of response category proportions were a subset of the physical stimulus measures, with the variance accounted for in the perceptual response category proportions between 17% and 52%. That audiovisual stimulus relationships can account for perceptual response distributions supports the possibility that internal representations are based on modality-specific stimulus relationships.

PMID:
21574741
PMCID:
PMC3149717
DOI:
10.1037/a0023100
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for American Psychological Association Icon for PubMed Central
Loading ...
Support Center