Abstract:

In autonomous driving simulation and industrial virtual reality simulation, there is a high demand for accuracy and robustness in 3D human body modeling. However, current joint-based human modeling approaches suffer from issues such as continuous modeling jitter, local distortion, and poor adaptability to occlusion, which degrade model quality and limit the development of practical applications such as intelligent driving and digital factories. To address these challenges, this paper proposes a multi-view vision-based inverse kinematics 3D human modeling method using a vector quantized variational autoencoder(IK-VQ-VAE). By integrating joint training with an automatic variational gradient descent approach, the proposed method achieves multi-view temporal fusion and enhanced occlusion adaptability, resulting in a more robust and realistic human pose reconstruction. Experiments conducted on the public Shelf dataset demonstrate that the proposed method achieves a maximum improvement of 23.7% and an average improvement of 8.7% in the percentage of correct parts(PCP)compared with recent optimized approaches. Qualitative results further confirm that our method produces superior 3D human modeling performance compared to existing methods.

Key words: multi-view vision, human mesh recovery, vector quantized variational autoencoder(VQ-VAE), 3D human modeling, human pose