Dynamic difficulty awareness training for continuous emotion prediction
File(s)
Accepted version
Author(s)
Zhang, Z
Han, Jing
Coutinho, Eduardo
Schuller, Björn
Type
Journal Article
Abstract
Time-continuous emotion prediction has become an increasingly compelling task in machine learning. Considerable efforts have been made to advance the performance of these systems. Nonetheless, the main focus has been the development of more sophisticated models and the incorporation of different expressive modalities (e.g., speech, face, and physiology). In this paper, motivated by the benefit of difficulty awareness in a human learning procedure, we propose a novel machine learning framework, namely, Dynamic Difficulty Awareness Training (DDAT), which sheds fresh light on the research - directly exploiting the difficulties in learning to boost the machine learning process. The DDAT framework consists of two stages: information retrieval and information exploitation. In the first stage, we make use of the reconstruction error of input features or the annotation uncertainty to estimate the difficulty of learning specific information. The obtained difficulty level is then used in tandem with original features to update the model input in a second learning stage with the expectation that the model can learn to focus on high difficulty regions of the learning process. We perform extensive experiments on a benchmark database (RECOLA) to evaluate the effectiveness of the proposed framework. The experimental results show that our approach outperforms related baselines as well as other well-established time-continuous emotion prediction systems, which suggests that dynamically integrating the difficulty information for neural networks can help enhance the learning process.
Date Issued
2019-05-01
Date Acceptance
2018-09-02
Citation
IEEE Transactions on Multimedia, 2019, 21 (5), pp.1289-1301
ISSN
1941-0077
Publisher
Institute of Electrical and Electronics Engineers
Start Page
1289
End Page
1301
Journal / Book Title
IEEE Transactions on Multimedia
Volume
21
Issue
5
Copyright Statement
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Subjects
Science & Technology
Technology
Computer Science, Information Systems
Computer Science, Software Engineering
Telecommunications
Computer Science
Emotion prediction
difficulty awareness learning
dynamic learning
RECOGNITION
ATTENTION
cs.LG
cs.LG
cs.AI
cs.HC
stat.ML
08 Information and Computing Sciences
09 Engineering
Artificial Intelligence & Image Processing
Publication Status
Published
Date Publish Online
2018-09-24