首页|时空自适应图卷积与Transformer结合的动作识别网络

时空自适应图卷积与Transformer结合的动作识别网络

扫码查看
在一个以人为中心的智能工厂中,感知和理解工人的行为是至关重要的,不同工种类别往往与工作时间和工作内容相关.该文通过结合自适应图和Transformer两种方式使模型更关注骨架的时空信息来提高模型识别的准确率.首先,采用一个自适应的图方法去关注除人体骨架之外的连接关系.进一步,采用Transformer框架去捕捉工人骨架在时间维度上的动态变化信息.为了评估模型性能,制作了智能生产线装配任务中6种典型的工人动作数据集,并进行验证,结果表明所提模型在Top-1精度上与主流动作识别模型相当.最后,在公开的NTU-RGBD和Skeleton-Kinetics数据集上,将该文模型与一些主流方法进行对比,实验结果表明,所提模型具有良好鲁棒性.
Action Recognition Network Combining Spatio-Temporal Adaptive Graph Convolution and Transformer
In a human-centered smart factory,perceiving and understanding workers'behavior is crucial,as different job categories are often associated with work time and tasks.In this paper,the accuracy of the model's recognition is improved by combining two approaches,namely adaptive graphs and Transformers,to focus more on the spatiotemporal information of the skeletal structure.Firstly,an adaptive graph method is employed to capture the connectivity relationships beyond the human body skeleton.Furthermore,the Transformer framework is utilized to capture the dynamic temporal variations of the worker's skeleton.To evaluate the model's performance,six typical worker action datasets are created for intelligent production line assembly tasks and validated.The results indicate that the model proposed in this article has a Top-1 accuracy comparable to mainstream action recognition models.Finally,the proposed model is compared with several mainstream methods on the publicly available NTU-RGBD and Skeleton-Kinetics datasets,and the experimental results demonstrate the robustness of the model proposed in this paper.

Intelligent manufacturingRecognition of worker activityDeep learningAdaptive graphTransformer

韩宗旺、杨涵、吴世青、陈龙

展开 >

上海理工大学机械工程学院 上海 200093

智能工厂 工人动作识别 深度学习 自适应图 Transformer

国家自然科学基金

52005338

2024

电子与信息学报
中国科学院电子学研究所 国家自然科学基金委员会信息科学部

电子与信息学报

CSTPCD北大核心
影响因子:1.302
ISSN:1009-5896
年,卷(期):2024.46(6)