多尺度信息融合的生成对抗网络壁画修复
Mural inpainting algorithm for generative adversarial network with multi-scale information fusion
胡升 1薛涛 1季虹1
作者信息
- 1. 西安工程大学计算机科学学院 西安 710048
- 折叠
摘要
针对现有壁画修复算法因缺乏对于图像远距离特征的捕获能力而导致修复结果结构紊乱,以及缺失边缘颜色不一致问题,提出一种多尺度信息融合的生成对抗网络壁画修复算法.首先,将多分支扩张卷积架构引入生成网络,各个子扩张卷积的卷积核以不同扩张率局部扩大感受野,提取图像的局部特征;其次结合快速傅里叶卷积基于全局感受野提取特征,实现壁画图像局部到全局的特征提取;最后引入自注意力与PatchGAN鉴别器以解决缺失边缘颜色不一致问题.根据自制壁画数据集进行模型的训练和测试,并与多组修复算法进行修复对比,实验结果表明,相较于对比算法,所提算法在峰值信噪比(PSNR)平均提升4.42 dB,结构相似性(SSIM)平均提升4.4%,学习感知图像块相似度(LPIPS)平均提升11.3%.实验证明所提算法能够有效修复破损壁画,修复后的壁画有较好的结构和纹理信息,为真实壁画的修复工作提供了支撑.
Abstract
Aiming at the existing mural inpating algorithms due to the lack of the ability to capture the remote features of the image,which leads to structural disorder in the inpating results,as well as the problem of inconsistent colour of the missing edges,a generative adversarial network mural restoration algorithm is proposed for the fusion of multiscale information.Firstly,the multi-branch dilated convolution architecture is introduced into the generative network,and the convolution kernel of each subdilated convolution locally expands the receptive field with different expansion rates to extract the local features of the image.Secondly,the fast Fourier convolution is combined to extract the features based on the global receptive field,so as to achieve the local-to-global feature extraction for the mural image.Finally,the self-attention and the PatchGAN discriminator are introduced to solve the problem of inconsistent colours of the missing edges.The model is trained and tested according to the homemade mural dataset,and compared with several groups of restoration algorithms for restoration.The experimental results show that compared with the comparison algorithms,the proposed algorithm improves PSNR by an average of 4.42 dB,SSIM by an average of 4.4%,and LPIPS by an average of 11.3%.The experiment proves that the proposed algorithm can effectively repair broken murals,and the repaired murals have better structure and texture information,and this method provides support for the restoration work of real murals.
关键词
多分支扩张卷积/快速傅里叶卷积/自注意力/生成对抗网络/壁画修复Key words
multi-branch dilated convolution/fast Fourier convolution/self-attention/generative adversarial network/mural inpainting引用本文复制引用
基金项目
国家自然科学基金(62106189)
陕西省技术创新引导专项计划(2020CGXNG-012)
出版年
2024
NETL
NSTL
万方数据