Survey of digital face rendering and appearance recovery methods
Digital human technology has attracted widespread attention in digital twins and metaverse fields.As an integral part of digital humans,people have started focusing on facial digitization and presentation.Consequently,the associated techniques find extensive applications in film,gaming,and virtual reality.A growing demand for facial realism rendering and high-quality facial inverse recovery has been observed.However,given the complex and multilayered material struc-ture of the face,facial realism rendering presents a challenge.Furthermore,the composition of internal skin chemicals,such as melanin and hemoglobin,highly influences skin rendering.Factors,such as temperature and blood flow rate,may influence the skin's appearance.The semitransparency of the skin introduces difficulties in the simulation of subsurface scattering effects,in addition to the wide presence of microscopic geometric features,such as pores and wrinkles on the face.All the issues mentioned above cause problems in the rendering domain and raise the demand for the quality of facial recovery.In addition,as a result of people's exposure to real human faces in daily life,a heightened sensitivity to the tex-ture and details of digital human faces has been observed,and this condition places greater demands on their realism and accuracy.Meanwhile,recovery of facial geometry and appearance is a crucial method for the construction of facial datas-ets.However,the high costs of acquisition equipment often constrain high-quality facial recovery,and most studies are lim-ited by the acquisition speed for facial data,which result in the challenging capture of dynamic facial appearance.Light-weight recovery methods also encounter challenges related to the lack of facial material datasets.This paper presents an overview of recent advances in rendering and recovery of digital human faces.First,we introduce methods for realistic facial rendering and categorize them based on diffusion approximation and Monte Carlo approaches.Methods based on dif-fusion approximation,which focus on the efficient achievement of the semitransparency effect of the skin,are constrained by strict assumptions and suffer from certain limitations in precision.However,their simplified subsurface scattering mod-els can render satisfactory images relatively quickly.Dynamic and interactive applications,such as games,often apply these methods.On the other hand,methods based on the Monte Carlo approach yield high precision and robust results via the meticulous and comprehensive simulation of the complex interactions between light and skin but require long computa-tion times to converge.In applications,such as movies,where highly realistic visual effects are needed,they often become the preferred choice.We emphasized the development and challenges of methods based on diffusion approximation and divided them into improvements in the diffusion profiles,with real-time implementation of subsurface scattering,and hybrid methods combined with Monte Carlo techniques for detailed discussion.A recent Monte Carlo research aimed at improving the convergence rate for applications in facial rendering,including zero-variance random walks,next-event esti-mation,and path guiding.Second,we divided facial recovery work into two categories:high-precision recovery based on specialized acquisition equipment and low-precision recovery based on deep learning.This paper further categorizes the for-mer based on the use of specialized lighting equipment,which distinguishes between active illumination and passive cap-ture techniques,with provided detailed explanations for each category.Active illumination relies on professional lighting equipment,such as the application of gradient lighting to recover high-precision normal maps,to improve recovery quality.Conversely,passive capture methods are independent of professional lighting equipment,and any artificially provided light-ing is limited to uniform illumination to reduce the interference of scene lighting on recovery and similar auxiliary roles.The exploration also focuses on low-precision facial recovery methods incorporating deep learning and classifies them into three categories,namely,geometric detail,texture mapping,and facial material information recoveries,to provide in-depth insights into each approach.We discuss a strategy for overcoming the limitations of geometric recovery based on para-metric models,introduced refined parametric expressions of models,and predicted a range of maps,including displace-ment maps,that represent the model surface's geometric details.For texture recovery,we explored the application of deep neural networks in generative tasks in the prediction of high-fidelity and personalized facial skin textures.Comprehensive reviews the various attempts to mitigate the ill-posed problem of separating reflectance information.In addition,we intro-duce the facial recovery work using multiview images and video sequences.These low-precision facial recovery methods can gain a wide application space given their flexibility and achieve improved recovery results with the rapid development of deep learning technology.Finally,the future trends in facial realism rendering and recovery methods based on the current state of research our outlined.In the realm of facial realism,existing works often represent the material properties of faces using texture maps and neglect the unique principles of skin coloration as a biological material.Furthermore,the rapid development of deep learning technology increases the importance of exploring of its integration with currently rendering techniques.In terms of inverse recovery,the lack of high-quality open-source datasets often poses limitations on data-based facial recovery methods.In addition,substantial improvement is needed in modeling and recovering details at the skin pore level.Combination of inverse recovery with text-based generative work also holds enormous potential and applica-tion scenarios.Hopefully,this paper can provide novice researchers in facial rendering and appearance recovery with valu-able background knowledge and inspiration from harmony and ideas.
NETL
NSTL
万方数据
