首页|FaSRnet:用于人体姿态估计的特征和语义修正网络

FaSRnet:用于人体姿态估计的特征和语义修正网络

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
由于运动模糊、视频失焦和遮挡等因素,多帧人体姿态估计是一项有挑战性的任务.利用连续帧之间的时间一致性是解决这一问题的有效方法.目前,大多数方法通过修正最终热图来利用时间一致性.热图包含了关键点的语义信息,可在一定程度上提高检测质量.它们由特征生成,但这些方法很少考虑特征级别的修正.本文提出一种人体姿态估计框架,该框架在特征和语义层面进行了改进.将辅助特征与当前帧的特征对齐,以减少不同特征分布带来的损失.使用注意力机制将辅助特征与当前特征融合.在语义方面,使用相邻热图之间的差异作为辅助特征来修正当前热图.在大型基准数据集PoseTrack2017和PoseTrack2018上验证了该方法的有效性.
FaSRnet:a feature and semantics refinement network for human pose estimation
Due to factors such as motion blur,video out-of-focus,and occlusion,multi-frame human pose estimation is a challenging task.Exploiting temporal consistency between consecutive frames is an efficient approach for addressing this issue.Currently,most methods explore temporal consistency through refinements of the final heatmaps.The heatmaps contain the semantics information of key points,and can improve the detection quality to a certain extent.However,they are generated by features,and feature-level refinements are rarely considered.In this paper,we propose a human pose estimation framework with refinements at the feature and semantics levels.We align auxiliary features with the features of the current frame to reduce the loss caused by different feature distributions.An attention mechanism is then used to fuse auxiliary features with current features.In terms of semantics,we use the difference information between adjacent heatmaps as auxiliary features to refine the current heatmaps.The method is validated on the large-scale benchmark datasets PoseTrack2017 and PoseTrack2018,and the results demonstrate the effectiveness of our method.

Human pose estimationMulti-frame refinementHeatmap and offset estimationFeature alignmentMulti-person

仲元红、徐乾锋、钟代笛、杨勋、王姗姗

展开 >

重庆大学微电子与通信工程学院,中国重庆市,400044

重庆大学生物工程学院,中国重庆市,400044

中国科学技术大学信息科学与技术学院,中国 合肥市,230039

安徽大学物质科学与信息技术研究院,中国 合肥市,230039

展开 >

人体姿态估计 多帧修正 热图和偏移估计 特征对齐 多人

国家重点研发计划国家重点研发计划Special Project of Technological Innovation and Application Development of Chongqing,China

2021YFC20092002023YFC3606100cstc2019jscxmsxmX0167

2024

10.1631/FITEE.2200639

信息与电子工程前沿(英文)
浙江大学

信息与电子工程前沿(英文)

CSTPCD
影响因子：0.371
ISSN：2095-9184
年,卷(期)：2024.25(4)
  • 48