首页|Emotion recognition based on convolutional neural networks and heterogeneous bio-signal data sources
Emotion recognition based on convolutional neural networks and heterogeneous bio-signal data sources
扫码查看
点击上方二维码区域，可以放大扫码查看
原文链接
NSTL
Elsevier
Emotion recognition is a crucial application in human-computer interaction. It is usually conducted using facial expressions as the main modality, which might not be reliable. In this study, we proposed a multimodal approach that uses 2-channel electroencephalography (EEG) signals and eye modality in addition to the face modality to enhance the recognition performance. We also studied the use of facial images versus facial depth as the face modality and adapted the common arousal-valence model of emotions and the convolutional neural network, which can model the spatiotemporal information from the modality data for emotion recognition. Extensive experiments were conducted on the modality and emotion data, the results of which showed that our system has high accuracies of 67.8% and 77.0% in valence recognition and arousal recognition, respectively. The proposed method outperformed most state-of-the-art systems that use similar but fewer modalities. Moreover, the use of facial depth has outperformed the use of facial images. The proposed method of emotion recognition has significant potential for integration into various educational applications.
Emotion recognitionElectroencephalogramArousal-valence model of emotions3D convolutional neural network
Ngai, Wang Kay、Xie, Haoran、Zou, Di、Chou, Kee-Lee
展开 >
Educ Univ Hong Kong, Dept Asian & Policy Studies, Hong Kong, Peoples R China
Lingnan Univ, Dept Comp & Decis Sci, Hong Kong, Peoples R China
Educ Univ Hong Kong, Dept English Language Educ, Hong Kong, Peoples R China
NSTL
Elsevier
