UTD Home
Human-Centered Research Lab

Publications at MSP lab


Journal Articles

  1. Soroosh Mariooryad and Carlos Busso, "Correcting time-continuous emotional labels by modeling the reaction lag of evaluators," IEEE Transactions on Affective Computing, vol. To appear, 2014, Special Issue Best of ACII. [soon pdf][soon cited][soon bib] Special Issue Best of ACII
  2. Nanxiang Li and Carlos Busso, "Predicting perceived visual and cognitive distractions of drivers with multimodal features," IEEE Transactions on Intelligent Transportation Systems, vol. To appear, 2014. [soon pdf][soon cited] [bib]
  3. Soroosh Mariooryad and Carlos Busso, "Compensating for speaker or lexical variabilities in speech for emotion recognition," Speech Communication, vol. 57, pp. 1-12, February 2014. [pdf] [cited] [bib]
  4. Juan Pablo Arias, Carlos Busso, and Nestor Becerra Yoma, "Shape-based modeling of the fundamental frequency contour for emotion detection in speech," Computer Speech and Language, vol. 28, no. 1, pp. 278-294, January 2014. [pdf] [cited] [bib]
  5. Carlos Busso, Soroosh Mariooryad, Angeliki Metallinou, and Shrikanth S. Narayanan, "Iterative feature normalization scheme for automatic emotion detection from speech," IEEE Transactions on Affective Computing, vol. 4, no. 4, pp. 386-397, October-December 2013. [pdf] [cited] [bib]
  6. Soroosh Mariooryad and Carlos Busso, "Exploring cross-modality affective reactions for audiovisual emotion recognition," IEEE Transactions on Affective Computing, vol. 4, no. 2, pp. 183-196, April-June 2013. [pdf] [cited] [bib]
  7. Nanxiang Li, Jinesh J. Jain, and Carlos Busso, "Modeling of driver behavior in real world scenarios using multiple noninvasive sensors," IEEE Transactions on Multimedia, vol. 15, no. 5, pp. 1213-1225, August 2013. [pdf] [cited] [bib]
  8. Soroosh Mariooryad and Carlos Busso, "Generating human-like behaviors using joint, speech-driven models for conversational agents," IEEE Transactions on Audio, Speech and Language Processing, vol. 20, no. 8, pp. 2329-2340, October 2012. [pdf] [cited] [bib]
  9. Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee, and Shrikanth S. Narayanan, "Emotion recognition using a hierarchical binary decision tree approach," Speech Communication, vol. 53, no. 9-10, pp. 1162-1171, November-December 2011. [pdf] [cited] [bib]
  10. Carlos Busso, Sungbok Lee, and Shrikanth S. Narayanan, "Analysis of emotionally salient aspects of fundamental frequency for emotion detection," IEEE Transactions on Audio, Speech and Language Processing, vol. 17, no. 4, pp. 582-596, May 2009. [pdf] [cited] [bib]
  11. Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette Chang, Sungbok Lee, and Shrikanth S. Narayanan, "IEMOCAP: Interactive emotional dyadic motion capture database," Journal of Language Resources and Evaluation, vol. 42, no. 4, pp. 335-359, December 2008. [pdf] [cited] [bib]
  12. Carlos Busso and Shrikanth S. Narayanan, "Interrelation between speech and facial gestures in emotional utterances: a single subject study," IEEE Transactions on Audio, Speech and Language Processing, vol. 15, no. 8, pp. 2331-2347, November 2007. [pdf] [cited] [bib]
  13. Carlos Busso, Zhigang Deng, Michael Grimm, Ulrich Neumann, and Shrikanth S. Narayanan, "Rigid head motion in expressive speech animation: Analysis and synthesis," IEEE Transactions on Audio, Speech and Language Processing, vol. 15, no. 3, pp. 1075-1086, March 2007. [pdf] [cited] [bib]
  14. Nestor Becerra Yoma, Carlos Molina, Jorge Silva, and Carlos Busso, "Modeling, estimating, and compensating low-bit rate coding distortion in speech recognition," IEEE Transactions on Audio, Speech and Language Processing, vol. 14, no. 1, pp. 246-255, January 2006. [pdf] [cited] [bib]
  15. Carlos Busso, Zhigang Deng, Ulrich Neumann, and Shrikanth S. Narayanan, "Natural head motion synthesis driven by acoustic prosodic features," Computer Animation and Virtual Worlds, vol. 16, no. 3-4, pp. 283-290, July 2005. [pdf] [cited] [bib] [slides]
  16. Nestor Becerra Yoma, Carlos Busso, and Ismael Soto, "Packet-loss modelling in IP networks with state-duration constraints," Communications, IEE Proceedings, vol. 152, no. 1, pp. 1-5, Feb 2005. [pdf] [cited] [bib]
  17. Nestor Becerra Yoma, Juan Hood, and Carlos Busso, "A real-time protocol for the internet based on the least mean square algorithm," Transactions on Multimedia, IEEE, vol. 6, no. 1, pp. 174-184, Feb 2004. [pdf] [cited] [bib]
  18. Nestor Becerra Yoma, Jorge Silva, Carlos Busso, and Ivan Brito, "Compensating additive noise and CS-CELP distortion in speech recognition using stochastic weighted Viterbi algorithm," Electronics Letters, IEE, vol. 39, no. 4, pp. 409-411, Feb 2003. [pdf] [cited] [bib]

Book chapters

  1. Chi-Chun Lee, Jangwon Kim, Angeliki Metallinou, Carlos Busso, Sungbok Lee, and Shrikanth S. Narayanan, "Speech in Affective Computing," in The Oxford Handbook of Affective Computing, R. Calvo, S. D'Mello, J. Gratch, and A. Kappas, Eds. Oxford University Press. To appear 2014. [link-to-pdf] [soon cited] [bib]
  2. Nanxiang Li and Carlos Busso, "Using perceptual evaluation to quantify cognitive and visual driver distractions," To appear in Smart Mobile In-Vehicle Systems - Next Generation Advancements, G. Schmidt, H. Abut, K. Takeda, and J. H. L. Hansen, Eds. pp. 183-207. Springer, New York, NY, USA, January 2014. [link-to-pdf] [cited] [bib]
  3. Carlos Busso, Murtaza Bulut, and Shrikanth S. Narayanan, "Toward effective automatic recognition systems of emotion in speech," in Social emotions in nature and artifact: emotions in human and human-computer interaction, S. Marsella J. Gratch, Eds., pp. 110-127. Oxford University Press, New York, NY, USA, November 2013. [link-to-pdf] [cited] [bib]
  4. Carlos Busso and Jinesh J. Jain, "Advances in multimodal tracking of driver distraction," in DSP for In-Vehicle Systems & Safety, J. Hansen, P. Boyraz, K. Takeda, and H. Abut, Eds., p. In Press. Springer, New York, NY, USA, 2012. [link-to-pdf] [cited] [bib]
  5. Carlos Busso, Murtaza Bulut, Sungbok Lee, and Shrikanth S. Narayanan, "Fundamental frequency analysis for speech emotion processing," in The Role of Prosody in Affective Speech, Sylvie Hancil, Ed., pp. 309-337. Peter Lang Publishing Group, Berlin, Germany, 2009. [link-to-pdf] [cited] [bib]
  6. Carlos Busso, Zhigang Deng, Ulrich Neumann, and Shrikanth S. Narayanan, "Learning expressive human-like head motion sequences from speech," in Data-Driven 3D Facial Animations, Zhigang Deng and Ulrich Neumann, Eds. Surrey,United Kingdom: Springer-Verlag London Ltd, 2007, pp. 113-131. [pdf] [cited] [bib]

Conference Proceedings

  1. Mohammed Abdelwahab and Carlos Busso, "Evaluation of syllable rate estimation in expressive speech and its contribution to emotion recognition," in IEEE Spoken Language Technology Workshop (SLT), South Lake Tahoe, CA, USA, December 2014. [soon pdf][soon cited] [bib]
  2. Najmeh Sadoughi, Yang Liu, and Carlos Busso, "Speech-driven animation constrained by appropriate discourse functions," in International conference on multimodal interaction (ICMI 2014), Istanbul, Turkey, November 2014. [soon pdf][soon cited] [bib]
  3. Nanxiang Li and Carlos Busso, "User-independent gaze estimation by exploiting similarity measures in the eye pair appearance eigenspace," in International conference on multimodal interaction (ICMI 2014), Istanbul, Turkey, November 2014. [soon pdf][soon cited] [bib]
  4. Fei Tao and Carlos Busso, "Lipreading approach for isolated digits recognition under whisper and neutral speech," in Interspeech 2014, Singapore, September 2014, pp. 1154-1158. [pdf] [soon cited] [bib] [poster]
  5. Soroosh Mariooryad, Reza Lotfian, and Carlos Busso, "Building a naturalistic emotional speech corpus by retrieving expressive behaviors from existing speech corpora," in Interspeech 2014, Singapore, September 2014, pp. 238-242. [pdf] [soon cited] [bib] [poster]
  6. Nanxiang Li and Carlos Busso, "Evaluating the robustness of an appearance-based gaze estimation method for multimodal interfaces," in International conference on multimodal interaction (ICMI 2013), Sydney, Australia, December 2013, pp. 91-98. [pdf] [soon cited] [bib] [poster]
  7. Nanxiang Li and Carlos Busso, "Driver mirror-checking action detection using multi-modal signals," in The 6th Biennial Workshop on Digital Signal Processing for In-Vehicle Systems, Seoul, Korea, September-October 2013, pp. 101-108. [pdf] [cited] [bib] [slides]
  8. Nanxiang Li, Amardeep Sathyanarayana, Carlos Busso, and John H.L. Hansen, "Rear-end collision prevention using mobile devices," in The 6th Biennial Workshop on Digital Signal Processing for In-Vehicle Systems, Seoul, Korea, September-October 2013, pp. 36-43. [pdf] [soon cited] [bib] [poster]
  9. Soroosh Mariooryad and Carlos Busso, "Analysis and compensation of the reaction lag of evaluators in continuous emotional annotations," in Affective Computing and Intelligent Interaction (ACII 2013), Geneva, Switzerland, September 2013, pp. 85-90. [pdf] [cited] [bib] [slides]
    Nominated for Best Student Paper at ACII 2013!
  10. Juan Pablo Arias, Carlos Busso, and Nestor Becerra Yoma, "Energy and F0 contour modeling with functional data analysis for emotional speech detection," in Interspeech 2013, Lyon, France, August 2013, pp. 2871-2875. [pdf] [cited] [bib] [poster]
  11. Nanxiang Li and Carlos Busso, "Analysis of facial features of drivers under cognitive and visual distractions," in IEEE International Conference on Multimedia and Expo (ICME 2013), San Jose, CA, USA, July 2013. [pdf] [cited] [bib] [slides]
  12. Tam Tran, Soroosh Mariooryad, and Carlos Busso, "Audiovisual corpus to analyze whisper speech," in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2013), Vancouver, BC, Canada, May 2013. [pdf] [cited] [bib] [poster]
  13. Soroosh Mariooryad and Carlos Busso, "Feature and model level compensation of lexical content for facial emotion recognition," in IEEE International Conference on Automatic Face and Gesture Recognition (FG 2013), Shanghai, China, April 2013. [pdf] [cited] [bib] [slides]
  14. Carlos Busso and Tauhidur Rahman, "Unveiling the Acoustic Properties that Describe the Valence Dimension," in Interspeech 2012, Portland, OR, USA, September 2012, pp. 1179-1182. [pdf] [cited] [bib] [poster]
  15. Soroosh Mariooryad and Carlos Busso, "Factorizing speaker, lexical and emotional variabilities observed in facial expressions," in IEEE International Conference on Image Processing (ICIP 2012), Orlando, FL, USA, September-October 2012. [pdf] [cited] [bib] [poster]
  16. David Tick, Tauhidur Rahman, Carlos Busso, and Nicholas Gans, "Indoor robotic terrain classification via angular velocity based hierarchical classifier selection," in IEEE International Conference on Robotics and Automation (ICRA 2012), St. Paul, MN, USA, May 2012, pp. 3594-3600. [pdf] [cited] [bib] [slides]
  17. Tauhidur Rahman and Carlos Busso, "A personalized emotion recognition system using an unsupervised feature adaptation scheme," in International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2012), Kyoto, Japan, March 2012, pp. 5117-5120. [pdf] [cited] [bib] [poster]
  18. Jinesh J. Jain and Carlos Busso, "Assessment of driver's distraction using perceptual evaluations, self assessments and multimodal feature analysis," in 5th Biennial Workshop on DSP for In-Vehicle Systems, Kiel, Germany, September 2011. [pdf] [cited] [bib] [slides]
  19. Tauhidur Rahman, Soroosh Mariooryad, Shalini Keshavamurthy, Gang Liu, John H.L. Hansen, and Carlos Busso, "Detecting sleepiness by fusing classifiers trained with novel acoustic features," in 12th Annual Conference of the International Speech Communication Association (Interspeech-2011), Florence, Italy, August 2011, pp. 3285-3288. [pdf] [cited] [bib] [slides]
  20. Xing Fan, Carlos Busso, and John H.L. Hansen, "Audio-visual isolated digit recognition for whispered speech," in European Signal Processing Conference (EUSIPCO-2011), Barcelona, Spain, August-September 2011. [pdf] [cited] [bib] [slides]
  21. Jinesh J. Jain and Carlos Busso, "Analysis of driver behaviors during common tasks using frontal video camera and CAN-Bus information," IEEE International Conference on Multimedia and Expo (ICME 2011), Barcelona, Spain, July 2011. [pdf] [cited] [bib] [poster] [slides] [Youtube]
    Hewlett Packard Best Paper Award at ICME2011!
  22. Carlos Busso, Angeliki Metallinou, and Shrikanth S. Narayanan, "Iterative feature normalization for emotional speech detection," in International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2011) Prague, Czech Republic, May 2011, pp. 5692-5695. [pdf] [cited] [bib] [poster]
  23. Angeliki Metallinou, Chi-Chun Lee, Carlos Busso, Sharon Carnicke, and Shrikanth S. Narayanan, "The USC CreativeIT database: A multimodal database of theatrical improvisation," in Workshop on Multimodal Corpora: Advances in Capturing, Coding and Analyzing Multimodality (MMC 2010), Valletta, Malta, May 2010. [pdf] [cited] [bib]
  24. Angeliki Metallinou, Carlos Busso, Sungbok Lee, and Shrikanth S. Narayanan, "Visual emotion recognition using compact facial representations and viseme information," in International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2010), Dallas, TX, USA, March 2010, pp. 2474-2477. [pdf] [cited] [bib] [poster]
  25. Emily Mower, Angeliki Metallinou, Chi-Chun Lee, Abe Kazemzadeh, Carlos Busso, Sungbok Lee, and Shrikanth S. Narayanan, "Interpreting ambiguous emotional expressions," in International Conference on Affective Computing and Intelligent Interaction (ACII 2009), Amsterdam, The Netherlands, September 2009. [pdf] [cited] [bib]
  26. Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee, and Shrikanth S. Narayanan, "Emotion recognition using a hierarchical binary decision tree approach," in Interspeech 2009, Brighton, UK, September 2009, pp. 320-323. [pdf] [cited] [bib] [slides]
  27. Chi-Chun Lee, Carlos Busso, Sungbok Lee, and Shrikanth S. Narayanan, "Modeling mutual influence of interlocutor emotion states in dyadic spoken interactions," in Interspeech 2009, Brighton, UK, September 2009, pp. 1983-1986. [pdf] [cited] [bib] [poster]
  28. Carlos Busso and Shrikanth S. Narayanan, "The expression and perception of emotions: Comparing assessments of self versus others," in Interspeech 2008 - Eurospeech, Brisbane, Australia, September 2008, pp. 257-260. [pdf] [cited] [bib] [poster]
  29. Carlos Busso and Shrikanth S. Narayanan, "Scripted dialogs versus improvisation: Lessons learned about emotional elicitation techniques from the IEMOCAP database," in Interspeech 2008 - Eurospeech, Brisbane, Australia, September 2008, pp. 1670-1673. [pdf] [cited] [bib] [poster]
  30. Carlos Busso and Shrikanth S. Narayanan, "Recording audio-visual emotional databases from actors: a closer look," in Second International Workshop on Emotion: Corpora for Research on Emotion and Affect, International conference on Language Resources and Evaluation (LREC 2008), Marrakech, Morocco, May 2008, pp. 17-22. [pdf] [cited] [bib] [slides]
  31. Carlos Busso and Shrikanth S. Narayanan, "Joint analysis of the emotional fingerprint in the face and speech: A single subject study," in International Workshop on Multimedia Signal Processing (MMSP 2007), Chania, Crete, Greece, October 2007, pp. 43-47. [pdf] [cited] [bib] [poster]
  32. Viktor Rozgic, Carlos Busso, Panayiotis G. Georgiou, and Shrikanth S. Narayanan, "Multimodal meeting monitoring: Improvements on speaker tracking and segmentation through a modified mixture particle filter," in International Workshop on Multimedia Signal Processing (MMSP 2007), Chania, Crete, Greece, October 2007, pp. 60-65. [pdf] [cited] [bib]
  33. Carlos Busso, Sungbok Lee, and Shrikanth S. Narayanan, "Using neutral speech models for emotional speech analysis," in Interspeech 2007 - Eurospeech, Antwerp, Belgium, August 2007, pp. 2225-2228. [pdf] [cited] [bib] [poster]
  34. Carlos Busso, Panayiotis G. Georgiou, and Shrikanth S. Narayanan, "Real-time monitoring of participants interaction in a meeting using audio-visual sensors," in International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2007), vol. 2, Honolulu, HI, USA, April 2007, pp. 685-688. [pdf] [cited] [bib] [slides]
  35. Carlos Busso and Shrikanth S. Narayanan, "Interplay between linguistic and affective goals in facial expression during emotional utterances," in 7th International Seminar on Speech Production (ISSP 2006), Ubatuba-SP, Brazil, December 2006, pp. 549-556. [pdf] [cited] [bib] [poster]
  36. Murtaza Bulut, Carlos Busso, Serdar Yildirim, Abe Kazemzadeh, Chul Min Lee, Sungbok Lee, and Shrikanth S. Narayanan, "Investigating the role of phoneme-level modifications in emotional speech resynthesis," in 9th European Conference on Speech Communication and Technology (Interspeech-2005 - Eurospeech), Lisbon, Portugal, September 2005, pp. 801-804. [pdf] [cited] [bib]
  37. Carlos Busso, Sergi Hernanz, Chi-Wei Chu, Soon-il Kwon, Sungbok Lee, Panayiotis G. Georgiou, Isaac Cohen, and Shrikanth S. Narayanan, "Smart Room: Participant and speaker localization and identification," in International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2005), vol. 2, Philadelphia, PA, USA, March 2005, pp. 1117-1120. [pdf] [cited] [bib] [poster]
  38. Nestor Becerra Yoma, Carlos Busso, Juan Inzunza, and Fernando Huenupan, "Packet-loss modeling with state duration constraints and VoIP based on perceptual quality maximization," in 10th International Conference on Speech and Computer (SPECOM 2005), Patras, Greece, October 2005, pp. 757-760. [pdf] [soon cited] [bib]
  39. Carlos Busso, Zhigang Deng, Serdar Yildirim, Murtaza Bulut, Chul Min Lee, Abe Kazemzadeh, Sungbok Lee, Ulrich Neumann, and Shrikanth S. Narayanan, "Analysis of emotion recognition using facial expressions, speech and multimodal information," in Sixth International Conference on Multimodal Interfaces ICMI 2004. State College, PA: ACM Press, October 2004, pp. 205-211. [pdf] [cited] [bib] [poster] [slides]
  40. Zhigang Deng, Carlos Busso, Shrikanth S. Narayanan, and Ulrich Neumann, "Audio-based head motion synthesis for avatar-based telepresence systems," in ACM SIGMM 2004 Workshop on Effective Telepresence (ETP 2004). New York, NY: ACM Press, October 2004, pp. 24-30. [pdf] [cited] [bib]
  41. Chul Min Lee, Serdar Yildirim, Murtaza Bulut, Abe Kazemzadeh, Carlos Busso, Zhigang Deng, Sungbok Lee, and Shrikanth S. Narayanan, "Emotion recognition based on phoneme classes," in 8th International Conference on Spoken Language Processing (ICSLP 04), Jeju Island, Korea, October 2004, pp. 889-892. [pdf] [cited] [bib]
  42. Serdar Yildirim, Murtaza Bulut, Chul Min Lee, Abe Kazemzadeh, Carlos Busso, Zhigang Deng, Sungbok Lee, and Shrikanth S. Narayanan, "An acoustic study of emotions expressed in speech," in 8th International Conference on Spoken Language Processing (ICSLP 04), Jeju Island, Korea, October 2004, pp. 2193-2196. [pdf] [cited] [bib]
  43. Nestor Becerra Yoma, Juan Hood, and Carlos Busso, "An UDP-based real time protocol for the Internet," in International Telecommunications Symposium(ITS 2002), Natal, Brazil, September 2002. [pdf] [soon cited] [bib]

Abstracts

  1. Serdar Yildirim, Murtaza Bulut, Carlos Busso, Chul Min Lee, Abe Kazemzadeh, Sungbok Lee, and Shrikanth S. Narayanan, "Study of acoustic correlates associate with emotional speech," J. Acoust. Soc. Am., vol. 116, p. 2481, 2004. [pdf] [bib]
  2. Chul Min Lee, Serdar Yildirim, Murtaza Bulut, Carlos Busso, Abe Kazemzadeh, Sungbok Lee, and Shrikanth S. Narayanan, "Effects of emotion on different phoneme classes," J. Acoust. Soc. Am., vol. 116, p. 2481, 2004. [pdf] [bib]
  3. Murtaza Bulut, Serdar Yildirim, Sungbok Lee, Chul Min Lee, Carlos Busso, Abe Kazemzadeh, and Shrikanth S. Narayanan, "Emotion to emotion speech conversion in phoneme level," J. Acoust. Soc. Am., vol. 116, p. 2481, 2004. [pdf] [bib]

Copyright Notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

(c) Copyrights. All rights reserved.