Visual saliency-driven non-local denoising of rendered panoramic images
Objective Panoramic movie technology has experienced notable advancements to enrich the audiovisual experi-ence for viewers,resulting in a heightened sense of immersion within the visual environment.Nevertheless,the production of high-quality images poses a challenge for conventional rasterization techniques,necessitating the exploration of alterna-tive approaches.Monte Carlo path tracing algorithms have been proven effective in generating high-quality images,offering exceptional visual fidelity in various rendering applications.However,the computational overhead associated with this algorithm remains challenging.Thus,reducing the number of pixels sampled in Monte Carlo path tracing is a common approach to optimize computation.However,this reduction often introduces noticeable noise in the resulting images,com-promising their overall quality.This paper aims to address the issue of image noise in Monte Carlo path tracing by exploring and proposing advanced techniques for denoising.Two main denoising approaches are commonly used in the domain of Monte Carlo rendering.The first approach utilizes traditional filtering methods with artificially designed filters to remove image noise.This approach is versatile,but its effectiveness in noise removal may be limited,often resulting in residual noise.The second approach involves deep learning-based denoising methods,which can effectively eliminate noise but may exhibit performance limitations on specific image types.Most existing image denoising algorithms are currently devel-oped and studied for ordinary flat images,with limited research dedicated to denoising algorithms specifically designed for panoramic images.Panoramic images possess unique characteristics,including a 360°field of view in the horizontal direc-tion,a 180° field of view in the vertical direction,distorted edges,and varying prominence of equatorial and polar pixels as perceived by human observers.Conventional flat image denoising methods often fail to fully account for these panoramic image characteristics,leading to excessive blurring or residual noise in the equatorial,polar,and distorted edge regions after the denoising process.Therefore,this paper proposes a visual saliency-driven non-local means(VSD-NLM)filtering denoising algorithm explicitly tailored for Monte Carlo rendering of panoramic images.The algorithm aims to leverage the distinctive characteristics of panoramic images,such as the 360°field of view,distorted edges,and varying pixel promi-nence,to effectively reduce noise while preserving the essential features of panoramic images.Through comprehensive experimentation and evaluation,the proposed algorithm demonstrates its efficacy in enhancing the image quality of Monte Carlo-rendered panoramic images,providing a valuable contribution to the field of panoramic image denoising.Method This paper presents the design and optimization of the VSD-NLM filtering algorithm for denoising Monte Carlo-rendered panoramic images.The proposed algorithm comprises two key components aimed at effectively removing noise and enhancing image quality in panoramic scenes.The first component focuses on enhancing the non-local means filtering process specifically tailored for panoramic images.Initially,a panoramic image saliency detection model is utilized to gen-erate a saliency image,incorporating an equatorial bias to improve saliency accuracy.Subsequently,the saliency image is employed to delineate saliency and non-saliency regions within the panoramic image.In the saliency region,the deviation value of the gradient magnitude similarity between image blocks is calculated to refine the weights used in non-local means filtering.For the non-saliency region,parallel algorithms for non-local means filtering are devised to accelerate the filter reconstruction process.Finally,denoising results from the saliency and non-saliency regions are combined to produce the final denoised panoramic image.The second component of the algorithm focuses on optimized noise reduction,specifically addressing the distorted edge regions of the panoramic image.Improvements are made to the Canny algorithm to obtain a highly accurate edge gradient image.Such improvements involve optimizing the weights for the 45°and 135°directions of the image,generating adaptive high and low thresholds using an improved Otsu method,and enhancing the local thresholds to optimize the performance of the Canny operator.Subsequently,anisotropic diffusion filtering is combined with guided fil-tering by utilizing the gradient image as a guide to filter and enhance the combined images.The optimizations of the pro-posed algorithm collectively contribute to effective noise reduction in the distorted edge regions of panoramic images,result-ing in enhanced image quality and reduced noise artifacts.Result This paper presents a comprehensive performance evalua-tion of the proposed denoising algorithm for panoramic images and utilizes structural similarity(SSIM)and FLIP metrics as objective evaluation indicators.The performance of the VSD-NLM algorithm is compared with other algorithms such as non-local means filtering,multifeature non-local means filtering,and progressive denoising algorithms to assess its effective-ness in reducing noise and improving the visual quality of panoramic images.Experimental results reveal that the proposed algorithm outperforms the comparison algorithms in terms of objective evaluation indicators.The average FLIP value achieved by the proposed algorithm is 15.2％lower compared with other algorithms.Similarly,the average SSIM value attained by the proposed algorithm is 14.7％higher than other algorithms,indicating its enhanced SSIM preservation.Fur-thermore,the visual effects of the algorithm are assessed,demonstrating its capability to mitigate blurring artifacts in pan-oramic images and enhance visual perception quality.This paper also presents an experimental verification of the effective-ness of two denoising algorithms:gradient magnitude similarity deviation assisted non-local means(GMSDA-NLM)and parallel non-local means(P-NLM).The GMSDA-NLM algorithm combines the strengths of non-local mean filtering and gradient magnitude similarity deviation to achieve superior noise reduction capabilities while maintaining the integrity of image details.This algorithm effectively identifies and suppresses noise while preserving the essential image features.The P-NLM algorithm exhibits a notable average speed increase of approximately six times compared to the nonparallel algo-rithm,facilitating real-time or near-real-time noise reduction applications.The SSIM value between P-NLM and the image generated by the nonparallel algorithm can reach 0.996.Conclusion This paper introduces a specialized denoising algo-rithm tailored for panoramic images,specifically addressing the unique challenges associated with denoising in this domain.From a practical perspective,the proposed algorithm holds substantial value for panoramic film production.The algorithm enhances the visual quality and fidelity of panoramic films by markedly reducing noise in panoramic images.The remarkable results obtained through the proposed algorithm contribute to immersive visual storytelling,elevating the overall cinematic experience and capturing the attention of audiences.Overall,the exceptional results achieved through the algo-rithm present valuable theoretical advancements and provide practical implications for panoramic film production,enhanc-ing the quality and impact of visual narratives in the realm of immersive cinematography.
panoramic imagenon-local means filtergradient magnitude similarity deviation(GMSD)guided filteringimage denoising
NETL
NSTL
万方数据
