首页|基于融合注意力和特征增强的跨模态行人重识别

基于融合注意力和特征增强的跨模态行人重识别

扫码查看
跨模态行人重识别是一项具有挑战性的任务,目的是在可见光和红外模式之间匹配行人图像,以便在犯罪调查和智能视频监控应用中发挥重要作用.为了解决跨模态行人重识别任务中对细粒度特征提取能力不强的问题,本文提出一种基于融合注意力和特征增强的行人重识别模型.首先,利用自动数据增强技术缓解不同摄像机的视角、尺度差异,并基于交叉注意力多尺度Vision Transformer,通过处理多尺度特征生成具有更强区分性的特征表示;接着,提出通道注意力和空间注意力机制,在融合可见光和红外图像特征时学习对区分特征重要的信息;最后,设计损失函数,采用基于自适应权重的难三元组损失,增强了每个样本之间的相关性,提高了可见光和红外图像对不同行人的识别能力.在SYSU-MM01和RegDB数据集上进行大量实验,结果表明,本文提出方法的mAP分别达到了 68.05%和85.19%,相较之前的工作性能有所提升,且通过消融实验和对比分析验证了本文模型的先进性和有效性.
Cross-modal person re-identification based on fused attention and feature enhancement
RGB-Infrared person re-identification(Re-ID)is a challenging task which aims to match person images between visible and infrared modalities,playing a crucial role in criminal investigation and intelligent video surveil-lance.To address the weak feature extraction capability for fine-grained features in current cross-modal person Re-ID tasks,this paper proposes a person re-identification model based on fused attention and feature enhancement.First,automatic data augmentation techniques are employed to mitigate the differences in perspectives and scales among different cameras,and a cross-attention multi-scale Vision Transformer is proposed to generate more discriminative feature representations by processing multi-scale features.Then the channel attention and spatial attention mecha-nisms are introduced to learn information important for distinguishing features when fusing visible and infrared image features.Finally,a loss function is designed,which adopts the adaptive weight based hard triplet loss,to enhance the correlation between each sample and improve the capability of identifying different persons from visible and infrared images.Extensive experiments conducted on the SYSU-MM01 and RegDB datasets show that the proposed approach achieves mAP of 68.05%and 85.19%,respectively,outperforming many state-of-the-art approaches.Moreover,ab-lation experiments and comparative analysis validate the superiority and effectiveness of the proposed model.

person re-identification(Re-ID)cross-modalcross attentionfeature extractionmulti-scale

黄驰涵、沈肖波

展开 >

南京理工大学设计艺术与传媒学院,南京,210094

南京理工大学计算机科学与工程学院,南京,210094

行人重识别 跨模态 交叉注意力 特征提取 多尺度

国家自然科学基金江苏省自然科学基金优秀青年基金

62176126BK20230095

2024

南京信息工程大学学报
南京信息工程大学

南京信息工程大学学报

CSTPCD北大核心
影响因子:0.737
ISSN:1674-7070
年,卷(期):2024.16(4)