Trends in Amplification

 

Advanced Search

Journal Navigation

Journal Home

Subscriptions

Archive

Contact Us

Table of Contents

Click here for more information

Sign In to gain access to subscriptions and/or personal tools.
This Article
Right arrow Abstract Freely available
Right arrow Free Full Text (Free PDF) Free
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Xin Luo,
Right arrow Articles by Galvin, J. J.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Xin Luo,
Right arrow Articles by Galvin, J. J.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?
Trends in Amplification, Vol. 11, No. 4, 301-315 (2007)
DOI: 10.1177/1084713807305301

Vocal Emotion Recognition by Normal-Hearing Listeners and Cochlear Implant Users

Xin Luo, PhD

Department of Auditory Implants and Perception, House Ear Institute, Los Angeles, California, xluo{at}hei.org

Qian-Jie Fu, PhD

Department of Auditory Implants and Perception, House Ear Institute, Los Angeles, California

John J. Galvin, BA

Department of Auditory Implants and Perception, House Ear Institute, Los Angeles, California

The present study investigated the ability of normal-hearing listeners and cochlear implant users to recognize vocal emotions. Sentences were produced by 1 male and 1 female talker according to 5 target emotions: angry, anxious, happy, sad, and neutral. Overall amplitude differences between the stimuli were either preserved or normalized. In experiment 1, vocal emotion recognition was measured in normal-hearing and cochlear implant listeners; cochlear implant subjects were tested using their clinically assigned processors. When overall amplitude cues were preserved, normal-hearing listeners achieved near-perfect performance, whereas listeners with cochlear implant recognized less than half of the target emotions. Removing the overall amplitude cues significantly worsened mean normal-hearing and cochlear implant performance. In experiment 2, vocal emotion recognition was measured in listeners with cochlear implant as a function of the number of channels (from 1 to 8) and envelope filter cutoff frequency (50 vs 400 Hz) in experimental speech processors. In experiment 3, vocal emotion recognition was measured in normal-hearing listeners as a function of the number of channels (from 1 to 16) and envelope filter cutoff frequency (50 vs 500 Hz) in acoustic cochlear implant simulations. Results from experiments 2 and 3 showed that both cochlear implant and normal-hearing performance significantly improved as the number of channels or the envelope filter cutoff frequency was increased. The results suggest that spectral, temporal, and overall amplitude cues each contribute to vocal emotion recognition. The poorer cochlear implant performance is most likely attributable to the lack of salient pitch cues and the limited functional spectral resolution.

Key Words: vocal emotion • cochlear implant • normal hearing • spectral resolution • temporal resolution

References

  • Fu QJ, Shannon RV, Wang X. Effects of noise and spectral resolution on vowel and consonant recognition: acoustic and electric hearing. J Acoust Soc Am. 1998; 104:3586-3596.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Friesen LM, Shannon RV, Baskent D., Wang X. Speech recognition in noise as a function of the number of spectral channels: comparison of acoustic hearing and cochlear implants. J Acoust Soc Am. 2001;110:1150-1163.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Fu QJ, Nogaki G. Noise susceptibility of cochlear implant users: the role of spectral resolution and smearing. J Assoc Res Otolaryngol. 2005;6:19-27.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Fu QJ, Chinchilla S., Galvin JJ III. The role of spectral and temporal cues in voice gender discrimination by normal-hearing listeners and cochlear implant users. J Assoc Res Otolaryngol. 2004;5:253-260.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Fu QJ, Chinchilla S., Nogaki G., Galvin JJ III. Voice gender identification by cochlear implant users: the role of spectral and temporal resolution. J Acoust Soc Am. 2005;118:1711-1718.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Gonzalez J., Oliver JC Gender and speaker identification as a function of the number of channels in spectrally reduced speech. J Acoust Soc Am. 2005;118:461-470.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Vongphoe M., Zeng FG Speaker recognition with temporal cues in acoustic and electric hearing. J Acoust Soc Am. 2005;118:1055-1061.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Shannon RV, Zeng FG, Kamath V., Wygonski J., Ekelid M. Speech recognition with primarily temporal cues. Science. 1995;270:303-304.[Abstract/Free Full Text]
  • Banse R., Scherer KR Acoustic profiles in vocal emotion expression. J Pers Soc Psychol. 1996;70:614-636.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Fernald A. Intonation and communicative intent in mothers' speech to infants: is the melody the message? Child Dev. 1989;60:1497-1510.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Cooper RP, Aslin RN Preference for infant-directed speech in the first month after birth. Child Dev. 1990; 61:1584-1595.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Trainor LJ, Austin CM, Desjardins RN Is infant-directed speech prosody a result of the vocal expression of emotion? Psychol Sci. 2000;11:188-195.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Williams CE, Stevens KN Emotions and speech: some acoustical correlates. J Acoust Soc Am. 1972;52:1238-1250.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Murray IR, Arnott JL Toward the simulation of emotion in synthetic speech: a review of the literature on human vocal emotion. J Acoust Soc Am. 1993;93:1097-1108.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Scherer KR Vocal communication of emotion: A review of research paradigms. Speech Communication. 2003; 40:227-256.
  • Dellaert F., Polzin T., Waibel A. Recognizing emotion in speech. In: Proceedings of the International Conference on Spoken Language Processing. 1996;1970-1973. Online archive of International Speech Communication Association proceedings available at: http://www.iscaspeech.org.
  • Petrushin VA Emotion recognition in speech signal: experimental study, development, and application. In: Proceedings of the International Conference on Spoken Language Processing. 2000;2:222-225. Online archive of International Speech Communication Association proceedings available at: http://www.isca-speech.org.
  • Cowie R., Douglas-Cowie E., Tsapatsoulis N., et al. Emotion recognition in human-computer interaction. IEEE Signal Process Mag 2001;18:32-80.[CrossRef]
  • Lee CM, Narayanan SS Toward detecting emotions in spoken dialogs. IEEE Trans Speech Audio Process. 2005;13:293-303.[CrossRef]
  • Yildirim S., Bulut M., Lee CM, et al. An acoustic study of emotions expressed in speech. In: Proceedings of the International Conference on Spoken Language Processing. 2004;2193-2196. Online archive of International Speech Communication Association proceedings available at: http://www.isca-speech.org.
  • Moore BC Perceptual consequences of cochlear hearing loss and their implications for the design of hearing aids. Ear Hear. 1996;17:133-161.[ISI][Medline] [Order article via Infotrieve]
  • Most T., Weisel A., Zaychik A. Auditory, visual and auditory-visual identification of emotions by hearing and hearing-impaired adolescents. Br J Audiol. 1993;27: 247-253.[ISI][Medline] [Order article via Infotrieve]
  • House D. Perception and production of mood in speech by cochlear implant users. In: Proceedings of the International Conference on Spoken Language Processing. 1994;2051-2054. Online archive of International Speech Communication Association proceedings available at: http://www.isca-speech.org.
  • Pereira C. The perception of vocal affect by cochlear implantees. In: Waltzman SB, Cohen NL, eds. Cochlear Implants. New York, NY: Thieme Medical; 2000: 343-345.
  • Shinall AR Emotion Perception in Pre-kindergarten School Children at Central Institute for the Deaf [master's thesis]. St Louis, Mo: Washington University School of Medicine; 2005.
  • Schorr EA Social and Emotional Functioning of Children With Cochlear Implants [dissertation]. College Park: University of Maryland; 2005.
  • Peters KP Emotion Perception in Speech: Discrimination, Identification, and the E fects of Talker and Sentence Variability [dissertation]. St Louis, Mo: Washington University School of Medicine; 2006.
  • Wilson BS, Finley CC, Lawson DT, Wolford RD, Eddington DK, Rabinowitz WM Better speech recognition with cochlear implants. Nature. 1991;352:236-238.[CrossRef][Medline] [Order article via Infotrieve]
  • Sjölander K., Beskow J. Wavesurfer speech processing software. Department of Speech, Music and Hearing, KTH, Sweden, 2006. Available at: http://www.speech.kth.se/wavesurfer/. Accessed November 1, 2005.
  • Shannon RV, Adams DD, Ferrel RL, Palumbo RL, Grandgenett M. A computer interface for psychophysical and speech research with the Nucleus cochlear implant. J Acoust Soc Am. 1990;87:905-907.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Wygonski J., Robert ME HEI Nucleus Research Interface Specification. Los Angeles, Calif: House Ear Institute; 2001.
  • Greenwood DD A cochlear frequency-position function for several species—29 years later. J Acoust Soc Am. 1990;87:2592-2605.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Fu QJ, Shannon RV Effects of amplitude nonlinearity on phoneme recognition by cochlear implant users and normal-hearing listeners. J Acoust Soc Am. 1998;104: 2570-2577.[CrossRef][ISI][Medline] [Order article via Infotrieve]

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati    What's this?



This Article
Right arrow Abstract Freely available
Right arrow Free Full Text (Free PDF) Free
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Xin Luo,
Right arrow Articles by Galvin, J. J.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Xin Luo,
Right arrow Articles by Galvin, J. J.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?