|
Sign In to gain access to subscriptions and/or personal tools.
|
Trends in Amplification, Vol. 11, No. 4,
233-241 (2007)
DOI: 10.1177/1084713807307409
Auditory and Visual Lexical Neighborhoods in Audiovisual Speech Perception
Nancy Tye-Murray, PhD
Washington University School of Medicine, murrayn{at}ent.wustl.edu
Mitchell Sommers, PhD
Department of Psychology, Washington University in St Louis St Louis, Missouri
Brent Spehar, PhD
Washington University School of Medicine
Much evidence suggests that the mental lexicon is organized into auditory neighborhoods, with words that are phonologically similar belonging to the same neighborhood. In this investigation, we considered the existence of visual neighborhoods. When a receiver watches someone speak a word, a neighborhood of homophenes (ie, words that look alike on the face, such as pat and bat) is activated. The simultaneous activation of a word's auditory and visual neighborhoods may, in part, account for why individuals recognize speech better in an auditory—visual condition than what would be predicted by their performance in audition-only and vision-only conditions. A word test was administered to 3 groups of participants in audition-only, vision-only, and auditory—visual conditions, in the presence of 6-talker babble. Test words with sparse visual neighborhoods were recognized more accurately than words with dense neighborhoods in a vision-only condition. Densities of both the acoustic and visual neighborhoods as well as their intersection overlap were predictive of how well the test words were recognized in the auditory—visual condition. These results suggest that visual neighborhoods exist and that they affect auditory—visual speech perception. One implication is that in the presence of dual sensory impairment, the boundaries of both acoustic and visual neighborhoods may shift, adversely affecting speech recognition.
Key Words: Lexical neighborhood • auditory—visual speech perception • integration
References
- Grant KW, Walden BE, Seitz PF Auditory-visual speech recognition by hearing-impaired subjects: consonant recognition, sentence recognition, and auditory-visual integration. J Acoust Soc Am. 1198;103:2677-2690.
- Sumby WH, Pollack I. Visual contributions to speech intelligibility in noise. J Acoust Soc Am. 1954;26:212-215.[Medline]
[Order article via Infotrieve]
- Sommers MS, Tye-Murray N., Spehar B. Auditory-visual speech perception and auditory-visual enhancement in normal-hearing younger and older adults. Ear Hear. 2005; 26:263-275.[CrossRef][ISI][Medline]
[Order article via Infotrieve]
- Tyler RD, Preece J., Tye-Murray N. The Iowa Laser Videodisk Tests. Iowa City, Iowa: University of Iowa Hospitals; 1986.
- Summerfield Q. Visual perception of phonetic gestures. In: Mattingly IG, ed. Modularity and the Motor Theory of Speech Perception. Hillsdale, NJ: Laurence Erlbaum; 1989:117-137.
- Massaro D. Perceiving Talking Faces: From Speech Perception to a Behavioral Principle. Cambridge, MA: MIT Press; 1998.
- Ouni S., Cohen M., Ishak H., Massaro D. Visual contribution to speech perception: measuring the intelligibility of animated talking heads. Eurasip J Audio Speech Music Process. In press.
- Cienkowski KM, Carney AE Auditory—visual speech perception and aging. Ear Hear. 2002;23:439-449.[CrossRef][ISI][Medline]
[Order article via Infotrieve]
- Tye-Murray N., Sommers M., Spehar B. Audiovisual integration and lipreading abilities of older adults with normal and impaired hearing. Ear Hear. In press.
- Braida L. Crossmodal integration in the identification of consonant segments. Psych Quart J Exp Psych. 1991;43: 647-677.
- Grant KW Measures of auditory-visual integration for speech understanding: a theoretical perspective. J Acoust Soc Am. 2002;112:30-33.[CrossRef][ISI][Medline]
[Order article via Infotrieve]
- Sommers MS, Spehar B., Tye-Murray N. The effects of signal-to-noise ratio on auditory-visual integration: integration and encoding are not independent. J Acoust Soc Am. 2005;117:2574.
- Blamey PJ, Cowan Rsc, Alcantara JI, Whitford LA, Clark GM Speech perception using combinations of auditory, visual, and tactile information. J Rehab Res Devel. 1989;26:15-24.[Medline]
[Order article via Infotrieve]
- Luce PA, Pisoni DB Recognizing spoken words: the neighborhood activation model. Ear Hear. 1998; 19:1-36.[Medline]
[Order article via Infotrieve]
- Mattys SL, Bernstein LE, Auer ET Stimulus-based lexical distinctiveness as a general word-recognition mechanism. Percept Psychophys. 2002;64:667-679.[ISI][Medline]
[Order article via Infotrieve]
- Lesner SA, Sandridge SA, Kricos PB Training influences on visual consonant and sentence recognition. Ear Hear. 1987;8:283-287.[ISI][Medline]
[Order article via Infotrieve]
- Berger KW Speechreading: Principles and methods. Kent, Ohio: Herald; 1972.
- Tye-Murray N., Geers A. Children's Audiovisual Enhancement Test. St. Louis, MO: Central Institute for the Deaf; 2001.
- Kaiser AR, Kirk KI, Lachs L., Pisoni DB Talker and lexical effects on audiovisual word recognition by adults with cochlear implants. J Speech Lang Hear Res. 2003; 46:390-404.
- Spehar B., Tye-Murray N., Sommers M. Time-compressed visual speech and age: a first report. Ear Hear. 2004;25:565-572.[CrossRef][ISI][Medline]
[Order article via Infotrieve]
- American Speech-Language-Hearing Association. Guidelines for determining threshold level for speech. ASHA. 1988; 30:85-89.[Medline]
[Order article via Infotrieve]
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati What's this?
|
