基于梯度注意力机制与交叉神经网络的红外与可见光图像融合
Infrared and visible image fusion based on an improved gradient attention mechanism and the cross neural network
孙希霞 1邓林威 1潘甦1
作者信息
- 1. 南京邮电大学 物联网学院,江苏 南京 210003
- 折叠
摘要
针对现有的基于深度学习的红外与可见光图像融合算法存在的难以区分重要信息与无关信息的问题,提出了一种基于梯度注意力机制与细节保留交叉神经网络(Detail Preserving Cross Network,DPCN)的红外与可见光图像融合方法.首先,将改进的梯度注意力机制引入到DPCN,引导神经网络尽可能关注可见光图像的纹理细节和红外图像的目标信息,同时利用DPCN加强红外图像和可见光图像之间的信息交互.然后,提出了一种基于多尺度细节保留模块的解码器重建融合图像.最后,设计了一种基于辅助判别器的自适应损失函数.实验结果表明:所提方法可保留更清晰的边缘及目标信息,在主观和客观评价方面均优于对比方法.
Abstract
Since the current infrared and visible image fusion methods based on deep learning are difficult to distinguish important information from irrelevant information,a new infrared and visible image fusion method based on the gradient attention mechanism and the detail preserving cross network(DPCN)is proposed.First,an improved gradient attention mechanism is introduced into the DPCN to guide the network to focus on the texture details of the visible image and the target information of the infrared image as much as possible.The DPCN is used to enhance the information interaction between the infrared image and the visible image.Then,a decoder based on the multi-scale detail preserving module is proposed to reconstruct the merged features.Finally,an adaptive loss function based on an auxiliary discriminator is designed.The experimental results show that the fusion image of the proposed method can retain clearer edge and target information,and is superior to the compared methods in both subjective and objective evaluations.
关键词
图像融合/注意力机制/细节保留交叉神经网络/多尺度图像重建Key words
image fusion/attention mechanism/detail preserving cross network/multi-scale image reconstruction引用本文复制引用
基金项目
国家自然科学基金(62071244)
国家自然科学基金(62172235)
出版年
2024
国家科技期刊平台
NSTL
万方数据