指挥信息系统与技术2024,Vol.15Issue(4) :69-77,94.DOI:10.15908/j.cnki.cist.2024.04.011

结合文本监督和多粒度特征的姿态情绪识别

Posture-Based Emotion Recognition via Text-Supervision and Multi-granularity Features

邹卓华 李新德 胡川飞 徐建平
指挥信息系统与技术2024,Vol.15Issue(4) :69-77,94.DOI:10.15908/j.cnki.cist.2024.04.011

结合文本监督和多粒度特征的姿态情绪识别

Posture-Based Emotion Recognition via Text-Supervision and Multi-granularity Features

邹卓华 1李新德 2胡川飞 1徐建平3
扫码查看

作者信息

  • 1. 东南大学自动化学院 南京 210096;南京应用数学中心 南京 211135
  • 2. 东南大学自动化学院 南京 210096;南京应用数学中心 南京 211135;东南大学深圳研究院 广东深圳 518063
  • 3. 信息系统工程重点实验室 南京 210023
  • 折叠

摘要

通过分析人体3D骨架的连续运动规律,建模姿态情绪识别方法,在人机交互等领域具有广泛的应用前景.针对姿态特征易混淆导致的骨架局部情绪性特征难捕获问题,提出了一种结合文本监督和多粒度特征的姿态情绪识别方法.通过多粒度特征融合网络模块,对情绪识别问题中人体的动态姿态进行建模.同时,采用文本化描述的标签对融合情绪特征进行监督训练,引导网络提取骨架各个部位的情绪性特征.在MPI情绪身体表达数据集上对该方法进行评估,试验结果表明,该方法准确率达80.50%,验证了其有效性.

Abstract

By analyzing the continuous motion patterns of the 3D pose skeleton,modeling a posture-based emotion recognition method has a wide application prospect in human-computer interaction.Aiming at the difficulty of capturing the local emotional features caused by easily-confused posture fea-tures,a posture-based emotion recognition method via text-supervision and multi-granularity features is proposed.With multi-granularity feature fusion network module,the dynamic posture of the human body in the emotion recognition problem is modeled.Meanwhile,the text description of the label is used to supervise and train the fused emotional features,guiding the network to extract the emotional features of each skeleton part.The method is evaluated on the MPI emotional body expression datas-et.The experimental results show that the accuracy of the method reaches 80.50%.Thus,its effec-tiveness is verified.

关键词

姿态情绪识别/特征融合/多粒度/文本监督

Key words

posture-based emotion recognition/feature fusion/multi-granularity/text-supervision

引用本文复制引用

出版年

2024
指挥信息系统与技术
中国电子科技集团公司第二十八研究所

指挥信息系统与技术

影响因子:0.707
ISSN:1674-909X
段落导航相关论文