基于混合注意力的双模态融合目标跟踪网络
Dual mode fusion object tracking network based on hybrid attention
权家锐 1何乐生 1晏开祥 1尹恒 1余圣涛 1廖伟1
作者信息
- 1. 云南大学信息学院,云南 昆明 650000
- 折叠
摘要
目标跟踪通常在面对亮度变化、背景混杂和快速移动等复杂场景时难以取得良好的跟踪性能,为此提出了一种结合自适应特征融合和注意力机制的红外与可见光融合的目标跟踪算法.利用红外光与可见光的互补性优势,增强传统目标跟踪算法在复杂场景下的跟踪性能.首先在前3层卷积层中结合注意力机制对红外光与可见光模态特征进行特征筛选,同时针对不同通道特征进行动态权重分配实现自适应特征融合,然后将不同通道特征进行融合,并经过实例分类模块实现对目标的跟踪.在GTOT数据集和RGBT234数据集上的实验结果表明,该算法的精度和成功率分别到达了 90.4％和73.2％、79.6％和56.1％,优于目前主流算法.
Abstract
Object tracking is often difficult to achieve good tracking performance in complex scenes,such as brightness changes,background interference and fast movement.Therefore,we propose an object tracking algorithm that combines infrared and visible light with adaptive feature fusion and an attention mechanism to improve tracking performance.By leveraging the complementary strengths of infrared and visible light,we enhance the performance of traditional object tracking algorithms in complex scenes.To achieve this,we first employ an attention mechanism in the initial three convolution layers to select relevant features from both the infrared and visible modalities.Simultaneously,we dynamically allocate weights to the features of different channels,enabling adaptive feature fusion.Subsequently,the features from different channels are fused,and the object is tracked using the instance classification module.Experimental results obtained from the GTOT dataset and RGBT234 dataset demonstrate the effectiveness of our proposed algorithm.The accuracy and success ratio achieved 90.4％and 73.2％on the GTOT dataset,and 79.6％and 56.1％on the RGBT234 dataset,respectively.These results surpass those of current mainstream algorithms.
关键词
目标跟踪/红外与可见光融合/注意力机制/特征融合Key words
object tracking/infrared and visible light fusion/attention mechanism/feature fusion引用本文复制引用
出版年
2024
NETL
NSTL
万方数据