基于双特征融合引导的深度图像超分辨率重建网络
Depth Image Super-Resolution Reconstruction Network Based on Dual Feature Fusion Guidance
耿浩文 1王宇 1辛彦玲1
作者信息
- 1. 长春理工大学电子信息工程学院,吉林 长春 130012
- 折叠
摘要
针对彩色图像引导的深度图像超分辨率重建算法中存在的纹理转移和深度流失的问题,提出一种基于双特征融合引导的深度图像超分辨率重建网络(DF-Net).为了充分利用深度和强度特征的关联性,在网络模型的深度恢复重建部分采用双通道融合模块(DCM)和双特征引导重建模块(DGM).利用输入金字塔结构提取深度信息和强度信息的多尺度特征:DCM基于通道注意力机制对深度特征和强度特征进行通道间的特征融合与增强;DGM将深度、强度特征自适应选择融合后实现重建的双特征引导,增加了深度特征的引导作用,改善了纹理转移和深度流失的问题.实验结果表明,所提方法的峰值信噪比(PSNR)和均方根误差(RMSE)优于RMRF、JBU和Depth-Net等方法,尤其4×超分辨率重建结果的PSNR值比其他方法平均提升6.79 dB,RMSE平均下降0.94,取得了较好的深度图像超分辨率重建效果.
Abstract
A depth image super-resolution reconstruction network(DF-Net)based on dual feature fusion guidance is proposed to address the issues of texture transfer and depth loss in color image guided deep image super-resolution reconstruction algorithms.To fully utilize the correlation between depth and intensity features,a dual channel fusion module(DCM)and a dual feature guided reconstruction module(DGM)are used to perform deep recovery and reconstruction in the network model.The multi-scale features of depth and intensity information are extracted using a input pyramid structure:DCM performs feature fusion and enhancement between channels based on a channel attention mechanism for depth and intensity features;DGM provides dual feature guidance for reconstruction by adaptively selecting and fusing depth and intensity features,increasing the guidance effect of depth features,and overcoming the issues of texture transfer and depth loss.The experimental results show that the peak signal-to-noise ratio(PSNR)and root mean square error(RMSE)of the proposed method are superior to those of methods such as RMRF,JBU,and Depth Net.Compared to the other methods,the PSNR value of the 4×super-resolution reconstruction results increased by an average of 6.79 dB,and the RMSE decreased by an average of 0.94,thus achieving good depth image super-resolution reconstruction results.
关键词
图像处理/图像超分辨率重建/卷积神经网络/深度图像/特征融合/通道注意力Key words
image processing/image super-resolution reconstruction/convolution neural network/depth image/feature fusion/channel attention引用本文复制引用
基金项目
吉林省自然科学基金(20210101180JC)
出版年
2024
NETL
NSTL
万方数据