首页|三维高斯溅射超分辨视觉场景构建算法

三维高斯溅射超分辨视觉场景构建算法

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
目前,机器人技术在工业化与自动化产业中扮演着举足轻重的角色,然而,机器人在视觉信息感知领域仍面临感知精度不足等挑战.为提升机器人在复杂工作场景下的视觉信息精确感知能力,该文基于三维高斯溅射(3D Gaussian splatting,3DGS)提出一种超分辨视觉场景构建算法.该算法引入真实世界增强型超分辨率生成对抗网络(real-world enhanced super-resolution generative adversarial networks,Real-ESRGAN)作为前置预处理技术生成高分辨率视图,并通过对生成的高分辨率视图下采样结果与原有的低分辨率视图对齐得到亚像素约束,进而增加三维重建细节部分的表征精细度.在场景构建过程中,自适应密度控制保证重建的精度,高斯快速可微光栅化器保证实时渲染速率.通过在具有复杂纹理、镜面反射等特征的多种场景实验验证表明,与传统 3DGS相比:峰值信噪比(PSNR)指标平均提高 7.81%,结构相似性指数(SSIM)指标平均提升 4.31%,学习感知图像块相似度(LPIPS)指标平均降低38.35%.该算法可显著改善传统 3DGS在低分辨率输入时出现的颜色渲染错误、针状伪影以及纹理信息缺失等问题,为机器人视觉信息感知提供新的技术支撑.
Super-resolution visual scene construction algorithm based on 3D Gaussian splatting
Currently,robotics technology plays a pivotal role in industrialization and automation industries.However,robots still face challenges in the field of visual information perception,such as insufficient perception accuracy.To enhance the precise perception capabilities of robots in complex work scenarios,this paper proposes a super-resolution visual scene construction algorithm based on 3D Gaussian splatting(3DGS).This algorithm introduces real-world enhanced super-resolution generative adversarial networks(Real-ESRGAN)as a pre-processing technique to generate high-resolution views.By downsampling the generated high-resolution views and aligning them with the original low-resolution views,sub-pixel constraints are obtained,which helps enhance the representational fineness of details in 3D reconstruction.During the scene construction process,adaptive density control ensures the accuracy of the reconstruction,and a Gaussian fast differentiable rasterizer guarantees real-time rendering rates.Experiments in various scenes with complex textures and mirror reflections have shown that,compared to traditional 3DGS,the peak signal-to-noise ratio(PSNR)has increased by an average of 7.81%,the structural similarity index(SSIM)has improved by an average of 4.31%,and the learned perceptual image patch similarity(LPIPS)has decreased by an average of 38.35%.This algorithm significantly improves upon the color rendering errors,needle-like artifacts,and missing texture information issues that occur with traditional 3DGS at low-resolution inputs,providing new technical support for robotic visual information perception.

3D reconstructionrobot perceptionneural radiance fieldssuper-resolution3D Gaussian splatting

侯礼杰、沈寅松、刘晓晨、陈雪梅、贾瑞才、范广伟、申冲

展开 >

中北大学仪器与电子学院,山西太原 030051

中北大学机电工程学院,山西太原 030051

北京理工大学前沿技术研究院,山东济南 250300

浙江科技大学信息与电子工程学院,浙江杭州 310000

中国电子科技集团公司第五十四研究所,河北石家庄 050000

展开 >

三维重建 机器人感知 神经辐射场 超分辨 三维高斯溅射

国家自然科学基金创新研究群体项目山西省优秀青年基金山西省重点研发计划山西省基础研究计划航空科学基金山西省量子传感与精密测量重点实验室项目

518210032021030212220112022020201010022023030212111502022Z0220U0002201905D121001

2024

10.11857/j.issn.1674-5124.2024070006

中国测试
中国测试技术研究院

中国测试

CSTPCD北大核心
影响因子：0.446
ISSN：1674-5124
年,卷(期)：2024.50(9)
  • 2