基于Transformer的文物图像修复方法
Transformer-Based Image Restoration Method for Cultural Relics
王真言 1蒋胜丞 1宋奇鸿 1刘波 1毕秀丽 1肖斌1
作者信息
- 1. 图像认知重庆市重点实验室(重庆邮电大学) 重庆 400065
- 折叠
摘要
文物极易因为保存不当而导致部分结构或纹理缺失,而现有的图像修复技术由于受到先验信息和卷积操作的局限而无法直接应用于文物图像修复,为更合理地恢复文物图像原貌,提出了一种新的文物图像修复方法,将文物图像修复工作分为 2个步骤:第 1步使用Transformer进行粗略的图像重建并恢复连贯的结构;第 2步使用卷积神经网络将粗略的重建图像进行上采样并恢复缺失区域的精细纹理.考虑到目前国内外没有高质量的大型文物数据库,因此也提出了一个新的高质量大型文物图像数据库.最终实验结果表明,在符合现实场景的破损修复实验和大面积破损修复实验中,修复效果在主观和客观评估中均优于当前图像修复算法.同时,支持多元化输出,为修复人员提供多样化参考,极大地提升了文物修复效率.
Abstract
Cultural relics are prone to partial losses of structure or texture due to improper preservation.In order to restore the original image of cultural relics,we propose a new method for restoring cultural relics by using Transformer's global structure understanding ability to restore the coherent structure of cultural relics and using convolutional neural networks'local texture understanding ability to restore the delicate texture of cultural relics.To achieve this goal,the restoration work is divided into two steps:the first step is to use Transformer to reconstruct the rough image and restore the coherent structure;the second step is to use a convolutional neural network to enlarge the rough image and restore the fine texture of the missing area.Considering that there is no high-quality,large-scale heritage database in China and abroad,a new heritage image database is also proposed.The experimental results show that the restoration results outperform the current image restoration algorithms in both subjective and objective evaluations in both breakage restoration experiments and large-area breakage restoration experiments that match realistic scenes.At the same time,the proposed method supports diversified output,which provides diverse references for restorers and improves the restoration efficiency.
关键词
文物数据库/文物图像补全/Transformer/卷积神经网络/超分辨/虚拟修复Key words
cultural relic database/cultural relic image completion/Transformer/convolutional neural networks/super-resolution/virtual restoration引用本文复制引用
基金项目
国家自然科学基金项目(62172067)
国家自然科学基金项目(61976031)
国家重点研发计划项目(2019YFE0110800)
重庆市杰出青年自然科学基金项目(CSTB2022NSCQ-JQX0001)
重庆市教委科技研究计划项目(KJQN202200635)
出版年
2024
NETL
NSTL
万方数据