A Lightweight Method for Human Body and Hand Mesh Reconstruction
The use of 3D human body reconstruction shows substantial potential across various domains,including film and television production and virtual reality.Notably,the prevailing reconstruction methodologies predominantly em-phasize the refinement of reconstruction accuracy and texture articulation,often necessitating high-performance comput-ing or sophisticated acquisition apparatus.Nonetheless,the current landscape exhibits a dearth of investigations into cost-effective and lightweight reconstruction techniques.In response to the imperative to alleviate usage costs and hard-ware requisites associated with human body reconstruction,this study proposes a strategy that entails the disentangle-ment of body and hand components grounded in a parameterized human body model.Subsequently,distinct reconstruc-tion networks have been tailored to accommodate the distinctive movement characteristics of the body and hands,offer-ing a judicious balance between computational parsimony and performance robustness.Both the body and hand recon-struction modules adopt an encoder-decoder architecture.The encoder segment of the body reconstruction module fea-tures a dual-stage design.Initially,leveraging Litehrnet and Canny edge algorithms,we derive heatmaps and edge maps,which serve as surrogate representations for RGB images,facilitating the acquisition of preliminary features through downsampling and concatenation.Because of the challenges of directly extracting adequate features from RGB images via lightweight backbone networks,the images are represented using edge maps and heatmaps.Subsequently,global features are procured in the second stage by Shufflenet.To improve performance,the activation function has been modified.To reduce parameter count while ensuring reconstruction efficacy,low-dimensional MLPs are used to esti-mate parameters based on probability distributions.Shape parameters are derived via a single MLP based on the Gauss-ian distribution,and pose parameters are estimated sequentially for each joint point utilizing cascaded low-dimensional MLPs guided by the Fisher matrix distribution.For the hand reconstruction branch,reconstruction is conducted based on vertex regression,and parameters are obtained via hand vertices.Conversely,the encoder of the hand reconstruction branch employs Litehrnet to yield multi-resolution feature branches.Although high-resolution features coupled with shal-low features exhibit enhanced granularity expression and low-resolution features afford superior global perception,we employ interpolation for pose pooling and fuse high-and low-resolution features to reconcile these disparate characteris-tics.Subsequently,the decoder employs a DSConv and upsample network to derive hand vertices.Shape parameters are estimated via MLP based on hand vertices,and joint rotation parameters are derived from vertex coordinates employing inverse topology mathematics.Compared to extant methodologies,the proposed method yields a notable reduction in pa-rameter and computational requisites,with an overall parameter count of 6.12M and a computational load of 433M.Evaluation of the Human3.6M dataset showcases an MPJPE of 86.7 mm for the body reconstruction branch,outperform-ing the classical method HMR(88.0 mm)with a parameter size representing only 11.6%of HMR.Moreover,the recon-structed mesh PA-MPJPE of 10.8 mm for the hand reconstruction branch surpasses regression-based full-body recon-struction methods such as ExPose and PIXIE,with parameter quantities of 4.7%and 3.1%,respectively.Furthermore,deployment on mobile devices for real-time inference,facilitated by Android Studio and PyTorch Android,yields an in-ference speed of 79.7 ms(12.5 fps)on Snapdragon 8Gen3,thereby meeting the exigencies of real-time inference applications.
human body reconstructionlightweight networkSMPL+HMANO
NETL
NSTL
万方数据
