针对多视图三维重建中存在的内存和时间消耗过大、高分辨率重建完整性差等问题,提出一种基于深度学习的多视图重建网络.网络由特征提取模块、级联的Patchmatch模块和深度图优化模块组成.首先,设计U型的特征提取模块,提取多阶段特征图,并在每个阶段引入相对位置编码的局部自注意力层,捕捉图像中的局部细节和全局上下文,提升网络特征提取性能.其次,设计深度残差网络,通过密集连接和残差结构对特征进行融合,充分利用彩色图像先验知识来约束深度图,提升深度估计的准确性.在公开数据集DTU(Technical University of Denmark)上进行测试,实验结果表明,三维重建质量到了有效的提升,与PatchmatchNet相比在完整性上提升了6.1%,在整体性上提升了2.5%,与其他的 SOTA(State-Of-The-Art)方法相比,在完整性和整体性上都得到了较大提升.
Multi-view Reconstruction with Local Self-attention and Deep Optimization
To address the issues of high memory and time consumption,low completeness and fidelity of high-resolution recon-struction in multi-view 3D reconstruction,we propose a deep learning-based multi-view reconstruction network.The network consists of a feature extraction module,a cascaded Patchmatch module and a depth map optimization module.First,we design a U-shaped feature extraction module to extract multi-stage feature maps,and introduce local self-attention layers with relative position encoding at each stage,which capture the local details and global context in the images,and enhance the feature extrac-tion performance of the network.Second,we design a deep residual network to fuse the features,and fully utilize the color image prior knowledge to constrain the depth map,and improve the accuracy of depth estimation.We test our network on the public da-taset DTU(Technical University of Denmark),and the experimental results show that our network achieves significant improve-ment in 3D reconstruction quality.Compared with PatchmatchNet,our network improves the completeness by 6.1%and the over-all by 2.5%.Compared with other SOTA(State-Of-The-Art)methods,our network also achieves better performance in both completeness and overall.
deep learning3D reconstructionlocal self-attention mechanismmulti-view stereodepth estimation
国家科技期刊平台
NSTL
万方数据
