结合多模态数据和混合模型的情绪识别
Combine Multimodal Data and Hybrid Models for Emotion Recognition
宗静 1熊馨 1周建华 1张琪 1周雕 1吉瑛2
作者信息
- 1. 昆明理工大学信息工程与自动化学院,昆明 650500
- 2. 昆明医科大学研究生院,昆明 650500
- 折叠
摘要
如何准确的识别人类情绪是一项颇具挑战性且十分有意义的任务,然而,由于情绪的复杂性,单一模态信号很难全面地描述情绪,而且基于生理信号的情绪识别准确率仍有可提升的空间.因此,本文提出了一种新颖的多模态轻量化情绪识别混合模型,名为FCAN-FFM-LightGBM,该模型主要由FCAN-FFM和LightGBM两部分构成,前者作为特征处理器,后者作为分类器.使用脑电信号(EEG)、眼电信号(EOG)和肌电信号(EMG)进行情绪识别.在DEAP公共数据集上进行广泛的实验验证,在四分类、唤醒维度和效价维度实验中分别取得了 95.92％、97.22％和97.16％的准确率.结果表明,混合模型有助于提升情绪识别准确率且多模态的情绪识别准确率明显优于单模态.与其它方法相比,本文方法在取得较高情绪分类精度的同时降低了计算成本.
Abstract
The precise and reliable identification of human emotions represents a challenging yet profoundly meaningful undertaking.However,it is difficult to comprehensively describe emotions with a single modal signal due to their intricate nature,and there is still room to improve the accuracy of emotion recognition based on physiological signals.Therefore,this research paper introduces a novel hybrid model for multimodal emotion recognition,denoted as FCAN-FFM-LightGBM.This model comprises two key components:FCAN-FFM,serving as a feature processor,and LightGBM,functioning as a classifier.Emotion recognition is conducted utilizing elec-troencephalogram(EEG),electrooculogram(EOG),and electromyogram(EMG)signals.Through extensive experimental evaluation on the DEAP public dataset,notable accuracies of 95.92％,97.22％,and 97.16％were achieved in four-class classification,arousal,and validity dimension experiments,respectively.These outcomes demonstrate the efficacy of multimodal in enhancing emotion recog-nition accuracy,surpassing the performance of unimodal approaches.Furthermore,compared with other methods,the method in this pa-per reduces computational consumption while achieving higher accuracy in emotion classification.
关键词
情绪识别/多模态/融合/EEG/EMG/EOGKey words
emotion recognition/multimodal/fusion/EEG/EMG/EOG引用本文复制引用
出版年
2025
NETL
NSTL
万方数据