首页|Segment differential aggregation representation and supervised compensation learning of ConvNets for human action recognition

Segment differential aggregation representation and supervised compensation learning of ConvNets for human action recognition

扫码查看
原文链接
  • 万方数据
  • 维普
With more multi-modal data available for visual classification tasks,human action recognition has become an increasingly at-tractive topic.However,one of the main challenges is to effectively extract complementary features from different modalities for action recognition.In this work,a novel multimodal supervised learning framework based on convolution neural networks(ConvNets)is proposed to facilitate extracting the compensation features from different modalities for human action recogni-tion.Built on information aggregation mechanism and deep ConvNets,our recognition framework represents spatial-temporal information from the base modalities by a designed frame difference aggregation spatial-temporal module(FDA-STM),that the networks bridges information from skeleton data through a multimodal supervised compensation block(SCB)to supervise the extraction of compensation features.We evaluate the proposed recognition framework on three human action datasets,including NTU RGB+D 60,NTU RGB+D 120,and PKU-MMD.The results demonstrate that our model with FDA-STM and SCB achieves the state-of-the-art recognition performance on three benchmark datasets.

action recognitionsegment frame difference aggregationsupervised compensation learningConvNets

REN ZiLiang、ZHANG QieShi、CHENG Qin、XU ZhenYu、YUAN Shuai、LUO DeLin

展开 >

School of Computer Science and Technology,Dongguan University of Technology,Dongguan 523808,China

Shenzhen Institute of Advanced Technology,Chinese Academy of Sciences,Shenzhen 518055,China

School of Electronic Engineering and Automation,Guilin University of Electronic Technology,Guilin 541004,China

Department of the mechanical engineering,Erlangen-Nuremberg University,Erlangen 91508,Germany

School of Aerospace Engineering,Xiamen University Xiamen 361102,China

展开 >

Natural Science Foundation of Guangdong ProvinceNatural Science Foundation of Guangdong ProvinceNational Key Laboratory of Airbased Information Perception and FusionAeronautic Science Foundation of ChinaDongguan Science and Technology Special Commissioner ProjectNational Natural Science Foundation of ChinaNational Natural Science Foundation of ChinaNational Natural Science Foundation of China

2022A15151401192023A151501130720220001068001202218005003626237626161972090U21A20487

2024

10.1007/s11431-023-2491-4

中国科学:技术科学(英文版)
中国科学院

中国科学:技术科学(英文版)

CSTPCDEI
影响因子：1.056
ISSN：1674-7321
年,卷(期)：2024.67(1)
  • 1