时空自适应图卷积与Transformer结合的动作识别网络
Action Recognition Network Combining Spatio-Temporal Adaptive Graph Convolution and Transformer
韩宗旺 1杨涵 1吴世青 1陈龙1
作者信息
- 1. 上海理工大学机械工程学院 上海 200093
- 折叠
摘要
在一个以人为中心的智能工厂中,感知和理解工人的行为是至关重要的,不同工种类别往往与工作时间和工作内容相关.该文通过结合自适应图和Transformer两种方式使模型更关注骨架的时空信息来提高模型识别的准确率.首先,采用一个自适应的图方法去关注除人体骨架之外的连接关系.进一步,采用Transformer框架去捕捉工人骨架在时间维度上的动态变化信息.为了评估模型性能,制作了智能生产线装配任务中6种典型的工人动作数据集,并进行验证,结果表明所提模型在Top-1精度上与主流动作识别模型相当.最后,在公开的NTU-RGBD和Skeleton-Kinetics数据集上,将该文模型与一些主流方法进行对比,实验结果表明,所提模型具有良好鲁棒性.
Abstract
In a human-centered smart factory,perceiving and understanding workers'behavior is crucial,as different job categories are often associated with work time and tasks.In this paper,the accuracy of the model's recognition is improved by combining two approaches,namely adaptive graphs and Transformers,to focus more on the spatiotemporal information of the skeletal structure.Firstly,an adaptive graph method is employed to capture the connectivity relationships beyond the human body skeleton.Furthermore,the Transformer framework is utilized to capture the dynamic temporal variations of the worker's skeleton.To evaluate the model's performance,six typical worker action datasets are created for intelligent production line assembly tasks and validated.The results indicate that the model proposed in this article has a Top-1 accuracy comparable to mainstream action recognition models.Finally,the proposed model is compared with several mainstream methods on the publicly available NTU-RGBD and Skeleton-Kinetics datasets,and the experimental results demonstrate the robustness of the model proposed in this paper.
关键词
智能工厂/工人动作识别/深度学习/自适应图/TransformerKey words
Intelligent manufacturing/Recognition of worker activity/Deep learning/Adaptive graph/Transformer引用本文复制引用
出版年
2024
NETL
NSTL
万方数据