首页|非对称的分层特征融合的RGBT跟踪网络

非对称的分层特征融合的RGBT跟踪网络

扫码查看
为了解决可见光图像和热红外图像由于成像原理不同而导致的模态存在异质性的问题,提出一种非对称的分层特征融合的RGBT跟踪网络。首先通过双流网络分别提取可见光和热红外的特征;然后通过模态特征提取模块挖掘不同模态特征,并对获得的特征进行自适应聚合,以获得有利于增强可见光模态的特征;最后将各层获得的聚合特征与双流网络获得的可见光特征进行增强融合,获得更具有鲁棒性的特征。在GTOT,RGBT234和LasHeR数据集上的实验结果表明,所提网络的跟踪精度(PR)和成功率(SR)分别达到92。2%/77。2%,82。9%/61。1%和52。7%/40。3%,与目前主流的RGBT目标跟踪网络相比,PR和SR均有所提高,验证了该网络的有效性。
Asymmetric Hierarchical Feature Fusion Network for RGBT Tracking
In order to solve the problem of modal heterogeneity between visible light images and thermal in-frared images due to different imaging principles,an asymmetric hierarchical feature fusion RGBT tracking network is proposed.Firstly,a two-stream network is used to extract visual light and thermal infrared fea-tures;then through the modal feature extraction module to mine different modal features and adaptively ag-gregate the obtained features to obtain features that are conducive to enhancing the visible light mode;fi-nally,the aggregated features obtained by each layer and the visible light features obtained by the two-stream network perform enhanced fusion to obtain more robust features.Experimental results on GTOT,RGBT234 and LasHeR datasets show that the tracking precision rate(PR)and success rate(SR)of the network reach 92.2%/77.2%,82.9%/61.1%and 52.7%/40.3%,compared with the current mainstream RGBT target tracking network,both PR and SR have been improved,which verifies the effectiveness of the network.

asymmetric structurehierarchical feature fusionRGBT target trackingTransformer

吴习惠、李婷、葛洪伟

展开 >

江南大学人工智能与计算机学院 无锡 214122

江苏省模式识别与计算机智能工程实验室(江南大学) 无锡 214122

非对称结构 分层特征融合 RGBT目标跟踪 Transformer

2024

计算机辅助设计与图形学学报
中国计算机学会

计算机辅助设计与图形学学报

CSTPCD北大核心
影响因子:0.892
ISSN:1003-9775
年,卷(期):2024.36(11)