基于多尺度空洞U-Net的多聚焦图像融合算法
Multiscale Dilated U-Net Based Multifocus Image Fusion Algorithm
聂丰镐 1李梦霞 1周孟香 1董雨雪 1李志良 1李龙1
作者信息
- 1. 长江大学计算机科学学院,湖北 荆州 434023
- 折叠
摘要
针对目前多聚焦融合算法仅采用单一的图像特征提取尺度,从而导致成像中细节边缘丢失和局部模糊等的问题,提出一种基于多尺度空洞U-Net的多聚焦图像融合算法.首先,在U-Net的编码器部分,引入多尺度空洞模块替代传统的卷积模块,充分利用不同尺度的感受野,更全面地捕捉局部和全局信息.此外,为了进一步增强图像特征的表征能力,在U-Net的中间层采用了RFB-s模块,以优化多尺度特征的定位能力.所提融合算法采用深度学习中端到端的有监督学习方法,分为特征提取、特征融合和图像重建3个模块,其中特征提取部分使用了包含多尺度空洞模块的U-Net.实验结果表明,所提算法得到的融合图像细节纹理清晰,且无重叠伪影.在所有用于对比的多聚焦图像融合算法中,所提算法的平均梯度、视觉信息保真度和互信息评价指标均最优,并且边缘信息保持度指标取得了接近最优结果的次优结果.同时,消融实验的结果进一步验证了所提多尺度空洞模块能显著增强网络的特征提取能力,从而提高图像融合质量.
Abstract
Current multifocus fusion algorithms use only a single-image feature extraction scale,leading to problems such as loss of detail edges and local blurring in imaging.In response to these algorithms,this paper proposes a multifocus image fusion algorithm based on multiscale null U-Net.First,in the encoder part of U-Net,a multiscale null module was introduced to replace the traditional convolution module,which fully uses sensory fields with various scales to capture local and global information more comprehensively.In addition,to enhance the image feature characterization further,a RFB-s module was employed in the middle layer of U-Net to optimize the localization ability of multiscale features.The proposed fusion algorithm adopted the end-to-end supervised learning method in deep learning.This method was divided into three modules:feature extraction,feature fusion,and image reconstruction.Among these,the feature extraction module used U-Net containing multiscale null modules.Experimental results show that the fused images obtained using the proposed algorithm have clear detailed texture and are free of overlapping artifacts.Among all multifocus image fusion algorithms used for comparison,the proposed algorithm is optimal in terms of average gradient,visual information fidelity,and mutual information evaluation metrics.Additionally,this algorithm achieves suboptimal results close to the optimal results in edge information retention metrics.Meanwhile,the ablation experiment results further verify that the proposed multiscale null module can remarkably enhance the feature extraction capability of the network,thereby improving the quality of image fusion.
关键词
图像处理/多聚焦图像/图像融合/多尺度/空洞卷积Key words
image processing/multi-focus image/image fusion/multi-scale/dilated convolution引用本文复制引用
基金项目
国家自然科学基金(62173049)
国家自然科学基金(62273060)
湖北省教育厅科学研究计划资助项目(D20211302)
出版年
2024
NETL
NSTL
万方数据