首页|基于无监督显著性掩码引导的红外与可见光图像融合网络

基于无监督显著性掩码引导的红外与可见光图像融合网络

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
在具有挑战性的拍摄环境中,使用单张红外或可见光图像很难捕获清晰详细的纹理信息以及热辐射信息.然而,红外和可见光图像融合允许保存来自红外图像的热辐射信息和来自可见光图像的纹理细节.现有的许多方法在融合过程中直接生成融合图像,忽略了对源图像像素级权重贡献的估计,强调了不同源图像之间的学习.为此,提出了基于无监督显著性掩码引导的红外与可见光图像融合网络,利用密集结构在源图像中进行全面的特征提取.它产生一个权重估计概率来评估每个源图像对融合图像的贡献.此外,由于红外与可见光图像缺乏真实标签,难以使用有监督学习,UMGN还引入了显著性掩码,便于网络集中学习红外图像的热辐射信息和可见光纹理信息.在训练过程中还引入了加权保真度项和梯度损失,以防止梯度退化.与大量其他最先进的方法进行对比实验,结果证明了所提出的UMGN方法的优越性和有效性.
UMGN:An Infrared and Visible Image Fusion Network Based on Unsupervised Significance Mask Guidance
In challenging shooting environments,it is difficult to capture clear and detailed texture information and thermal radia-tion information using a single infrared or visible image.However,infrared and visible image fusion allows the preservation of thermal radiation information in infrared images and texture details in visible light images.Many existing methods directly gener-ate fused images in the fusion process,ignore the estimation of pixel-level weight contribution of source images,and emphasize the learning between different source images.For this reason,an infrared and visible image fusion based on unsupervised significance mask guidance network is proposed,which uses DenseNet structure to extract comprehensive features from source images.It pro-duces a weight estimation probability to evaluate the contribution of each source image to the fused image.Since infrared and visi-ble images lack ground truth,it is difficult to use supervised learning.UMGN also introduces the significance mask to facilitate the network to focus on learning the thermal radiation information and visible light texture information of infrared images.A weighted fidelity term and gradient loss are also introduced in the training process to prevent gradient degradation.A large num-ber of comparative experiments with other advanced methods prove the superiority and effectiveness of the proposed UMGN method.

Unsupervised learningSignificance maskWeight estimation probabilityInfrared and visible image fusion

李东阳、聂仁灿、潘琳娜、李贺

展开 >

云南大学信息学院 昆明 650091

无监督学习 显著性掩码 权重估计概率 红外与可见光图像融合

国家自然科学基金云南省基础研究计划重点项目云南省基础研究计划重点项目国家重点研发计划云南省科技厅项目

61966037202301AS070025202401AT0704672020YFA07143012012105AF150011

2024

10.11896/jsjkx.230600170

计算机科学
重庆西南信息有限公司（原科技部西南信息中心）

计算机科学

CSTPCD北大核心
影响因子：0.944
ISSN：1002-137X
年,卷(期)：2024.51(z1)
  • 19