首页|基于特征聚合Transformer的多视图立体重建方法

基于特征聚合Transformer的多视图立体重建方法

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
针对现有多视图立体(MVS)方法中缺乏对整体图像的理解以及图像之间的联系所造成的在弱纹理区域或非朗伯曲面匹配模糊的问题,提出一种基于特征聚合Transformer的多视图立体重建方法.首先,通过融合可变形卷积的特征金字塔网络对输入图像进行特征提取,自适应地调整感受野的大小和形状.接着引入基于Transformer的空间聚合模块,利用图像内部自注意力机制和图像间交叉注意力机制有效地获取视图内全局上下文信息以及视图间的信息交互关系,从而更精确地捕获场景的纹理特征进行特征聚合,实现可靠的特征匹配.最后,采用可见性成本聚合估计像素可见性信息,去除成本聚合中的噪声和错误匹配像素.在DTU和Tanks&Temples数据集上的实验结果表明,与其他方法相比,所提方法的重建性能更优越.
Multiview Stereo Reconstruction with Feature Aggregation Transformer
In this study,a multiview stereo network reconstruction method based on a feature aggregation transformer was proposed to address the problem of blurred matching in areas with weak textures or non-Lambertian surfaces.This is caused by the lack of understanding of the overall image and connections between images in existing multiview stereo methods.Initially,the input image extracted features by fusing deformable convolutional feature pyramid networks.Further,the size and shape of the receptive field were adaptively adjusted.Subsequently,a Transformer-based spatial aggregation module was introduced to capture the texture features of scenes more accurately for feature aggregation using the intra-image self-attention mechanism.This yielded the intra-view global contextual information and inter-image cross-attention mechanism to efficiently obtain inter-view information interactions,thereby achieving a reliable feature match by capturing the texture features of scenes more accurately.Finally,visibility cost aggregation was employed to estimate pixel visibility information to remove noisy and mismatched pixels from cost aggregation.Experimental results on the DTU and Tanks&Temples datasets show that the proposed method achieves superior reconstruction performance compared with other methods.

multiview stereoTransformerfeature aggregationcascade architecture3D reconstruction

王敏、赵明富、宋涛、李沩沩、田媛、李程、张渝

展开 >

光纤传感与光电检测重庆市重点实验室,重庆 400054

重庆理工大学电气与电子工程学院,重庆 400054

多视图立体 Transformer 特征聚合 级联架构 三维重建

重庆市科技局基础与前沿研究计划资助项目中国博士后科学基金重庆理工大学科研创新团队项目重庆理工大学研究生教育高质量发展行动计划

cstc2021jcyjmsxmX03482022M7105432023TDZ014gzlcx20233131

2024

10.3788/LOP232546

激光与光电子学进展
中国科学院上海光学精密机械研究所

激光与光电子学进展

CSTPCD北大核心
影响因子：1.153
ISSN：1006-4125
年,卷(期)：2024.61(14)
  • 5