计算机辅助设计与图形学学报2024,Vol.36Issue(11) :1700-1709.DOI:10.3724/SP.J.1089.2024.20162

非对称的分层特征融合的RGBT跟踪网络

Asymmetric Hierarchical Feature Fusion Network for RGBT Tracking

吴习惠 李婷 葛洪伟
计算机辅助设计与图形学学报2024,Vol.36Issue(11) :1700-1709.DOI:10.3724/SP.J.1089.2024.20162

非对称的分层特征融合的RGBT跟踪网络

Asymmetric Hierarchical Feature Fusion Network for RGBT Tracking

吴习惠 1李婷 1葛洪伟1
扫码查看

作者信息

  • 1. 江南大学人工智能与计算机学院 无锡 214122;江苏省模式识别与计算机智能工程实验室(江南大学) 无锡 214122
  • 折叠

摘要

为了解决可见光图像和热红外图像由于成像原理不同而导致的模态存在异质性的问题,提出一种非对称的分层特征融合的RGBT跟踪网络.首先通过双流网络分别提取可见光和热红外的特征;然后通过模态特征提取模块挖掘不同模态特征,并对获得的特征进行自适应聚合,以获得有利于增强可见光模态的特征;最后将各层获得的聚合特征与双流网络获得的可见光特征进行增强融合,获得更具有鲁棒性的特征.在GTOT,RGBT234和LasHeR数据集上的实验结果表明,所提网络的跟踪精度(PR)和成功率(SR)分别达到92.2%/77.2%,82.9%/61.1%和52.7%/40.3%,与目前主流的RGBT目标跟踪网络相比,PR和SR均有所提高,验证了该网络的有效性.

Abstract

In order to solve the problem of modal heterogeneity between visible light images and thermal in-frared images due to different imaging principles,an asymmetric hierarchical feature fusion RGBT tracking network is proposed.Firstly,a two-stream network is used to extract visual light and thermal infrared fea-tures;then through the modal feature extraction module to mine different modal features and adaptively ag-gregate the obtained features to obtain features that are conducive to enhancing the visible light mode;fi-nally,the aggregated features obtained by each layer and the visible light features obtained by the two-stream network perform enhanced fusion to obtain more robust features.Experimental results on GTOT,RGBT234 and LasHeR datasets show that the tracking precision rate(PR)and success rate(SR)of the network reach 92.2%/77.2%,82.9%/61.1%and 52.7%/40.3%,compared with the current mainstream RGBT target tracking network,both PR and SR have been improved,which verifies the effectiveness of the network.

关键词

非对称结构/分层特征融合/RGBT目标跟踪/Transformer

Key words

asymmetric structure/hierarchical feature fusion/RGBT target tracking/Transformer

引用本文复制引用

出版年

2024
计算机辅助设计与图形学学报
中国计算机学会

计算机辅助设计与图形学学报

CSTPCDCSCD北大核心
影响因子:0.892
ISSN:1003-9775
段落导航相关论文