FaSRnet:用于人体姿态估计的特征和语义修正网络
FaSRnet:a feature and semantics refinement network for human pose estimation
仲元红 1徐乾锋 1钟代笛 2杨勋 3王姗姗4
作者信息
- 1. 重庆大学微电子与通信工程学院,中国重庆市,400044
- 2. 重庆大学生物工程学院,中国重庆市,400044
- 3. 中国科学技术大学信息科学与技术学院,中国 合肥市,230039
- 4. 安徽大学物质科学与信息技术研究院,中国 合肥市,230039
- 折叠
摘要
由于运动模糊、视频失焦和遮挡等因素,多帧人体姿态估计是一项有挑战性的任务.利用连续帧之间的时间一致性是解决这一问题的有效方法.目前,大多数方法通过修正最终热图来利用时间一致性.热图包含了关键点的语义信息,可在一定程度上提高检测质量.它们由特征生成,但这些方法很少考虑特征级别的修正.本文提出一种人体姿态估计框架,该框架在特征和语义层面进行了改进.将辅助特征与当前帧的特征对齐,以减少不同特征分布带来的损失.使用注意力机制将辅助特征与当前特征融合.在语义方面,使用相邻热图之间的差异作为辅助特征来修正当前热图.在大型基准数据集PoseTrack2017和PoseTrack2018上验证了该方法的有效性.
Abstract
Due to factors such as motion blur,video out-of-focus,and occlusion,multi-frame human pose estimation is a challenging task.Exploiting temporal consistency between consecutive frames is an efficient approach for addressing this issue.Currently,most methods explore temporal consistency through refinements of the final heatmaps.The heatmaps contain the semantics information of key points,and can improve the detection quality to a certain extent.However,they are generated by features,and feature-level refinements are rarely considered.In this paper,we propose a human pose estimation framework with refinements at the feature and semantics levels.We align auxiliary features with the features of the current frame to reduce the loss caused by different feature distributions.An attention mechanism is then used to fuse auxiliary features with current features.In terms of semantics,we use the difference information between adjacent heatmaps as auxiliary features to refine the current heatmaps.The method is validated on the large-scale benchmark datasets PoseTrack2017 and PoseTrack2018,and the results demonstrate the effectiveness of our method.
关键词
人体姿态估计/多帧修正/热图和偏移估计/特征对齐/多人Key words
Human pose estimation/Multi-frame refinement/Heatmap and offset estimation/Feature alignment/Multi-person引用本文复制引用
基金项目
国家重点研发计划(2021YFC2009200)
国家重点研发计划(2023YFC3606100)
Special Project of Technological Innovation and Application Development of Chongqing,China(cstc2019jscxmsxmX0167)
出版年
2024
NETL
NSTL
万方数据