基于改进多层感知机的神经辐射场三维重建方法
3D Reconstruction of Neural Radiation Field Based on Improved Multiple Layer Perceptron
侯耀斐 1黄海松 2范青松 1肖婧 1韩正功1
作者信息
- 1. 贵州大学现代制造技术教育部重点实验室,贵州 贵阳 550025
- 2. 贵州大学现代制造技术教育部重点实验室,贵州 贵阳 550025;重庆机电职业技术大学信息工程学院,重庆 402760
- 折叠
摘要
相比传统的三维重建方法,神经辐射场(NeRF)在隐式三维重建方面显示出了优异的性能,然而简单的多层感知机(MLP)模型在采样过程中缺乏局部信息,产生细节模糊的三维重建场景.为解决这一问题,提出一种基于MLP的多特征联合学习方法.首先,在NeRF嵌入层和采样层之间构造多特征联合学习(MFJL)模块,有效解码输入的多视图编码数据,补充MLP模型缺失的局部特征信息.然后,在NeRF采样层和推理层之间建立门控通道变换多层感知机(GCT-MLP)模块,学习高阶特征交互关系,并控制反馈给MLP层的信息流,实现对歧义特征的选择.实验结果表明:所提基于改进MLP的神经辐射场可以避免三维重建中的视图模糊和混叠现象;在Real Forward-Facing数据集部分场景上的平均峰值信噪比(PSNR)、结构相似度(SSIM)、学习感知图像块相似度(LPIPS)分别为 28.08 dB、0.887、0.061;在Realistic Synthetic 360°数据集部分场景上的PSNR、SSIM、LPIPS分别为 32.75 dB、0.960、0.026;在DTU数据集部分场景上的PSNR、SSIM、LPIPS分别为25.96 dB、0.807、0.208;与NeRF相比,具有更好的视图重建性能,并且在主观视觉效果上得到更加清晰的图像和细节纹理特征.
Abstract
Neural radiation field(NeRF)exhibits excellent performances in implicit 3D reconstruction compared with traditional 3D reconstruction methods.However,the simple multilayer perceptron(MLP)model lacks local information in the sampling process,resulting in a fuzzy 3D reconstruction scene.To solve this issue,a multifeature joint learning(MFJL)method based on MLP is proposed in this study.First,an MFJL module was constructed between the embedding layer and the sampling layer of NeRF to effectively decode the multiview encoded input and supplement the missing local information of MLP model.Then,a gated channel transformation MLP(GCT-MLP)module was built between the sampling layer and the inference layer of NeRF to learn the interaction relations between higher-order features and control the information flow fed back to the MLP layer for the selection of ambiguous features.The experimental results reveal that the NeRF based on the improved MLP can avoid blurred views and aliasing in 3D reconstruction.The average peak signal-to-noise ratio(PSNR),structural similarity(SSIM),and learned perceptual image patch similarity(LPIPS)values on the Real Forward-Facing dataset are 28.08 dB,0.887,and 0.061;on the Realistic Synthetic 360° dataset are 32.75 dB,0.960,and 0.026;and on the DTU dataset are 25.96 dB,0.807,and 0.208,respectively.Overall,the proposed method has a better view reconstruction performance and can obtain clearer images and detailed texture features in subjective visual effects compared with NeRF.
关键词
神经辐射场/多层感知机/联合学习/三维重建Key words
neural radiation field/multiple-layer perceptron/joint learning/3D reconstruction引用本文复制引用
基金项目
贵州省科技计划(黔科合平台人才-GCC[2022]006-1)
贵州省科技计划(黔科合支撑[2022]一般165)
贵州省科技计划(黔科合支撑[2021]一般445)
贵州省科技计划(黔科合支撑[2021]一般172)
贵州省科技计划(黔科合支撑[2021]一般397)
贵州省科技计划(黔科合支撑[202)
重庆市自然科学基金面上项目(CSTB2022NSCQ-MSX1600)
出版年
2024
NETL
NSTL
万方数据