时空关联与图注意力引导的微表情识别网络
Spatiotemporal association and graph-attention guided micro-expression recognition network
于洋 1王晓民 1岑世欣 1李扬 2孙芳芳1
作者信息
- 1. 河北工业大学 人工智能与数据科学学院,天津 300401
- 2. 天津市农业科学院 信息研究所,天津 300192
- 折叠
摘要
微表情作为典型非自发表情可以揭示人类的真实意图,在公安测谎、情感识别以及医学辅助诊断等领域有着重要的应用.现有微表情识别方法主要关注面部局部区域肌肉运动,往往忽略了与微表情产生相关的全局区域关联性.针对上述问题,提出基于时空区域关联与图注意力引导的微表情识别网络.网络由时空关联模块和图注意力模块组成,时空关联模块利用AU结合时空图卷积聚合面部全局区域不同节点的时空特征,并使用Transformer Encoder进一步构建面部远距离节点间的区域相关性,用以加强全局区域间的时空联系.图注意力模块以面部局部区域为基础建立图结构,并指导自注意力的运算,从而获得精确的局部区域关联特征.最后将全局时空和局部关联特征相融合用于微表情的识别.在CASME、CASME Ⅱ、SAMM 3个主流的微表情数据集上进行了实验,分别取得了82.46%、86.59%和80.88%的优秀识别结果.实验结果表明,本文提出的方法与其他方法相比有更好的表现.
Abstract
As a typical non-spontaneous expression,micro-expressions can reflect the true intentions of human beings and have important applications in polygraphs and other fields.In order to solve the problem of weak global region corre-lation related to micro-expression generation,a micro-expression recognition network based on spatiotemporal regional correlation and graph attention guidance is proposed.The network consists of spatiotemporal association module and graph attention module.Spatiotemporal association module aggregates the spatiotemporal features of different nodes in the global region of the face by combining AU with spatiotemporal graph convolution,and uses Transformer encoder to further construct regional correlations among distant nodes of the face to strengthen the spatiotemporal association from a global perspective.The graph attention module builds the graph structure based on the facial local areas,and includes the calculation of self-attention,to obtain accurate local area association features.Finally,the global spatiotemporal and local correlation features are fused to recognize micro-expressions.Experiments were conducted on three mainstream mi-cro-expression datasets,CASME,CASME Ⅱ,and SAMM,and achieved excellent recognition results of 82.46%,86.59%,and 80.88%,respectively.The experimental results show that the proposed method has better performance than other methods.
关键词
微表情识别/Vision/Transformer/AU动作单元/时空图卷积/自注意力Key words
micro-expression recognition/Vision Transformer/AU action unit/spatiotemporal graph convolution/self-attention引用本文复制引用
基金项目
国家自然科学基金(61806071)
国家自然科学基金(62102129)
出版年
2024
NETL
NSTL
万方数据