基于自编码器的红外与可见光图像融合算法
Infrared and visible image fusion algorithm based on autoencoder
陈海秀 1房威志 2陆成 2陆康 2何珊珊 2黄仔洁2
作者信息
- 1. 南京信息工程大学 自动化学院,南京 210044;南京信息工程大学 江苏省大气环境与装备技术协同创新中心,南京 210044
- 2. 南京信息工程大学 自动化学院,南京 210044
- 折叠
摘要
针对目前红外与可见光图像融合过程中,图像特征提取不充分、中间层信息丢失以及融合图像细节不够清晰的问题,提出了一种基于自编码器的端到端图像融合网络结构.该网络由编码器、融合网络和解码器 3 部分组成.将高效通道注意力机制和混合注意力机制引入到编码器和融合网络中,利用卷积残差网络(convolutional residual network,CRN)基本块来提取并融合红外图像和可见光图像的基本特征,然后将融合后的特征图输入到解码器进行解码,重建出融合图像.选取目前具有典型代表性的5 种方法在主客观方面进行对比.在客观方面,较第 2 名平均梯度、空间频率和视觉保真度分别提升了21%、10.2%、7.2%.在主观方面,融合后的图像目标清晰、细节突出、轮廓明显,符合人类视觉感受.
Abstract
To address the current problems of inadequate image feature extraction,loss of information in the middle layer and insufficient details of fused images in the process of infrared and visible image fusion,this paper proposes an end-to-end image fusion network structure based on a self-encoder,which consists of three parts:encoder,fusion network and decoder.Firstly,the efficient channel attention mechanism and hybrid attention mechanism are introduced into the encoder and fusion network.The CRN(convolutional residual network)base blocks are used to extract and fuse the basic features of infrared images and visible images.The fused feature images are input to the decoder to reconstruct the fused images.Five representative methods are selected to compare with subjective and objective aspects.In the objective aspect,compared with the second place,AG、SF and VIF have increased by 21%,10.2%,and 7.2%.In the subjective aspect,significantly with clear targets,prominent details and obvious outline,which is in line with human visual perception.
关键词
红外图像/可见光图像/图像融合/注意力机制/编码解码结构Key words
infrared images/visible light images/image fusion/attention mechanism/autoencoder引用本文复制引用
基金项目
国家自然科学基金项目(61302189)
江苏省研究生科研与实践创新计划项目(SJCX23_0383)
出版年
2024
NETL
NSTL
万方数据