首页|UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene

UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene

扫码查看
原文链接
  • NETL
  • NSTL
  • IEEE
Photorealistic stylization of 3D scenes aims to generate photorealistic images from arbitrary novel views according to a given style image, while ensuring consistency when rendering video from different viewpoints. Some existing stylization methods using neural radiance fields can effectively predict stylized scenes by combining the features of the style image with multi-view images to train 3D scenes. However, these methods generate novel view images that contain undesirable artifacts. In addition, they cannot achieve universal photorealistic stylization for a 3D scene. Therefore, a stylization image needs to retrain a 3D scene representation network based on a neural radiation field. We propose a novel photorealistic 3D scene stylization transfer framework to address these issues. It can realize photorealistic 3D scene style transfer with a 2D style image for novel view video rendering. We first pre-trained a 2D photorealistic style transfer network, which can satisfy the photorealistic style transfer between any content image and style image. Then, we use voxel features to optimize a 3D scene and obtain the geometric representation of the scene. Finally, we jointly optimize a hypernetwork to realize the photorealistic style transfer of arbitrary style images. In the transfer stage, we use a pre-trained 2D photorealistic network to constrain the photorealistic style of different views and different style images in the 3D scene. The experimental results show that our method not only realizes the 3D photorealistic style transfer of arbitrary style images, but also outperforms the existing methods in terms of visual quality and consistency.

Three-dimensional displaysRendering (computer graphics)TrainingImage color analysisImage reconstructionFeature extractionTask analysis

Yaosen Chen、Qi Yuan、Zhiqiang Li、Yuegen Liu、Wei Wang、Chaoping Xie、Xuming Wen、Qien Yu

展开 >

Media Intelligence Laboratory, ChengDu Sobey Digital Technology Company, Ltd, Chengdu, China

Southwest Jiaotong University, Chengdu, China|Chengdu Sobey Digital Technology Company, Ltd, Chengdu, China

Media Intelligence Laboratory, Chengdu Sobey Digital Technology Company, Ltd, Chengdu, China|Peng Cheng Laboratory, Shenzhen, China

College of Computer Science, Sichuan University, Chengdu, China

展开 >

2025

10.1109/TVCG.2024.3378692

IEEE transactions on visualization and computer graphics

IEEE transactions on visualization and computer graphics

SCI
ISSN：
年,卷(期)：2025.31(4)
  • 68