基于颜色和光流的多注意力机制微表情识别
Multi-attention micro-expression recognition based on color and optical flow
黄凯 1王峰 2王晔 1常亦婷1
作者信息
- 1. 太原理工大学 电子信息与光学工程学院,山西 晋中 030606
- 2. 太原理工大学 电气与动力工程学院,山西 太原 030024
- 折叠
摘要
针对光流法无法充分利用微表情面部颜色信息,导致识别准确率不高的问题,本文提出一种基于颜色和光流的多注意力双流网络方法.首先,提出以CIE Luv色差图的形式,初步提取人脸情感生理特征,弥补微表情光流特征的单一性和局限性;然后,将PAM模块和ECA block并行组合得到轻量化的双注意力模块,提取空间和通道关键特征;最后,设计一种交叉注意力机制以获取颜色和光流混合特征,将其与空间通道关键特征融合用于分类.本模型在实验中采用留一交叉验证法进行评估,在SAMM数据集上的准确率和F1分数分别达到69.18%和67.04%,在CASME Ⅱ数据集上的准确率和F1分数分别达到72.38%和70.85%.实验结果均优于目前主流算法,进一步证明本文模型及其模块在识别微表情方面的有效性.
Abstract
The optical flow method cannot fully exploit the facial color information of micro-expressions,resulting in low recognition accuracy.Therefore,this paper proposes a multi-attention dual-flow network method based on color and optical flow.Firstly,the facial color difference maps are obtained in the CIE Luv color space,and the emotional-physiological features are extracted to compensate for the singularity and limitation of the micro-expression optical flow features.Then,the PAM module and ECA block are combined in parallel to obtain the lightweight dual-attention module,which extracts the spatial and channel key features.Finally,a cross-attention mechanism is designed to obtain mixed features of color and optical flow.The mixed features are fused with spatial channel key features for micro-expression classification.The model is evaluated experimentally using leave-one-out cross-validation.The accuracy and F1 scores reach 69.18%and 67.04%on the SAMM dataset,and 72.38%and 70.85%on the CASME Ⅱ dataset.The experimental results are superior to the current mainstream algorithms,further proving the effectiveness of the proposed model and its modules in micro-expression recognition.
关键词
计算机视觉/微表情识别/CIE/Luv/颜色特征/光流特征/双流网络Key words
computer vision/micro-expression recognition/CIE Luv/color features/optical flow features/two-stream network引用本文复制引用
基金项目
山西省回国留学人员科研资助项目(2020-042)
山西省留学回国人员科技活动择优资助基金(20200017)
山西省基础研究计划(20210302123186)
国家级重点支持领域大创项目(20220058)
出版年
2024
国家科技期刊平台
NSTL
万方数据