Affordable Access

deepdyve-link
Publisher Website

Affective Latent Representation of Acoustic and Lexical Features for Emotion Recognition.

Authors
  • Kim, Eesung1
  • Song, Hyungchan2
  • Shin, Jong Won2
  • 1 AI R&D Team, Kakao Enterprise, 235, Pangyoyeok-ro, Bundang-gu, Seongnam-si, Gyeonggi-do 13494, Korea. , (North Korea)
  • 2 School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, 123 Cheomdan-gwagiro, Buk-gu, Gwangju 61005, Korea. , (North Korea)
Type
Published Article
Journal
Sensors
Publisher
MDPI AG
Publication Date
May 04, 2020
Volume
20
Issue
9
Identifiers
DOI: 10.3390/s20092614
PMID: 32375342
Source
Medline
Keywords
Language
English
License
Unknown

Abstract

In this paper, we propose a novel emotion recognition method based on the underlying emotional characteristics extracted from a conditional adversarial auto-encoder (CAAE), in which both acoustic and lexical features are used as inputs. The acoustic features are generated by calculating statistical functionals of low-level descriptors and by a deep neural network (DNN). These acoustic features are concatenated with three types of lexical features extracted from the text, which are a sparse representation, a distributed representation, and an affective lexicon-based dimensions. Two-dimensional latent representations similar to vectors in the valence-arousal space are obtained by a CAAE, which can be directly mapped into the emotional classes without the need for a sophisticated classifier. In contrast to the previous attempt to a CAAE using only acoustic features, the proposed approach could enhance the performance of the emotion recognition because combined acoustic and lexical features provide enough discriminant power. Experimental results on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) corpus showed that our method outperformed the previously reported best results on the same corpus, achieving 76.72% in the unweighted average recall.

Report this publication

Statistics

Seen <100 times