多目视觉下的逆运动学三维人体建模仿真

doi:10.16182/j.issn1004731x.joss.25-0742

摘要/Abstract

摘要：

自动驾驶仿真和工业虚拟现实仿真技术中对三维人体建模的准确性和鲁棒性具有较高的需求，现阶段基于关节点进行人体建模存在连续建模抖动、局部扭曲、遮挡适应性差等影响人体模型质量的问题，制约了智能驾驶和数字工厂等实际应用的发展。针对上述问题，提出一种多目视觉下基于向量量化变分自编码器的逆运动学三维人体建模方法，通过梯度下降自动变分方法的联合训练与IK-VQ-VAE(inverse kinematics vector quantised-variational auto encoder)方法相结合，得到了多视角时序融合、遮挡适应且更具鲁棒性的方法，满足更加符合真实人体姿态的需求。在公开数据集Shelf上进行实验，结果显示所提方法的正确部件百分比(PCP)相比近年的优化工作最高提升23.7%，平均提升了8.7%，同时，定性实验分析结果也表明了所提方法对人体3D建模效果优于其他方法。

关键词: 多目视觉, 人体网格恢复, 向量量化变分自编码器, 三维人体建模, 人体姿态

Abstract:

In autonomous driving simulation and industrial virtual reality simulation, there is a high demand for accuracy and robustness in 3D human body modeling. However, current joint-based human modeling approaches suffer from issues such as continuous modeling jitter, local distortion, and poor adaptability to occlusion, which degrade model quality and limit the development of practical applications such as intelligent driving and digital factories. To address these challenges, this paper proposes a multi-view vision-based inverse kinematics 3D human modeling method using a vector quantized variational autoencoder(IK-VQ-VAE). By integrating joint training with an automatic variational gradient descent approach, the proposed method achieves multi-view temporal fusion and enhanced occlusion adaptability, resulting in a more robust and realistic human pose reconstruction. Experiments conducted on the public Shelf dataset demonstrate that the proposed method achieves a maximum improvement of 23.7% and an average improvement of 8.7% in the percentage of correct parts(PCP)compared with recent optimized approaches. Qualitative results further confirm that our method produces superior 3D human modeling performance compared to existing methods.

Key words: multi-view vision, human mesh recovery, vector quantized variational autoencoder(VQ-VAE), 3D human modeling, human pose

中图分类号:

TP391.9

方国宇,李琰泽,陈凯等 . 多目视觉下的逆运动学三维人体建模仿真[J]. 系统仿真学报, 2026, 38(1): 99-111.

Fang Guoyu,Li Yanze,Chen Kai,et al . Inverse Kinematics 3D Human Modeling Simulation based on Multi-view Vision[J]. Journal of System Simulation, 2026, 38(1): 99-111.

图/表 9

图1

图2

图3

表1

图4

图5

图6

表2

图7

参考文献 40

[1]	Loper Matthew, Mahmood Naureen, Romero Javier, et al. SMPL: A Skinned Multi-person Linear Model[C]//Seminal Graphics Papers: Pushing the Boundaries. New York: ACM, 2023: 851-866.
[2]	Grochow K, Martin S L, Hertzmann Aaron, et al. Style-based Inverse Kinematics[C]//ACM SIGGRAPH 2004 Papers. New York: ACM, 2004: 522-531.
[3]	Chharia A, Gou Wenbo, Dong Haoye. MV-SSM: Multi-view State Space Modeling for 3D Human Pose Estimation[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2025: 11590-11599.
[4]	殷贤涛, 胡波, 李思照. 基于YOLOv8n-Pose关键点特征增强估计算法[J]. 无线电通信技术, 2025, 51(5): 1025-1035.
	Yin Xiantao, Hu Bo, Li Sizhao. Enhanced Estimation Algorithm Based on YOLOv8n-Pose Key Point Features[J]. Radio Communications Technology, 2025, 51(5): 1025-1035.
[5]	张嘉欣, 赵子月, 陈猛. 低光动态场景下的轻量小目标人体姿态估计[J]. 无线电工程, 2025, 55(8): 1607-1617.
	Zhang Jiaxin, Zhao Ziyue, Chen Meng. A Lightweight Network for Small-object Human Pose Estimation in Low-light Dynamic Conditions[J]. Radio Engineering, 2025, 55(8): 1607-1617.
[6]	Song Yupei, Wu Xiao, Yuan Zhaoquan, et al. PostureHMR: Posture Transformation for 3D Human Mesh Recovery[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2024: 9732-9741.
[7]	吴桢. 基于时空网络学习的人体运动预测方法研究[D]. 成都: 电子科技大学, 2025.
	Wu Zhen. Research on Human Motion Prediction Method Based on Spatiotemporal Network Learning[D]. Chengdu: University of Electronic Science and Technology of China, 2025.
[8]	Akhter Ijaz, Black Michael J. Pose-conditioned Joint Angle Limits for 3D Human Pose Reconstruction[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2015: 1446-1455.
[9]	Wang Chunyu, Wang Yizhou, Yuille A L. An Approach to Pose-based Action Recognition[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2013: 915-922.
[10]	金梓桥. 基于单帧图像以及SMPL的三维人体重建方法研究[D]. 杭州: 浙江工商大学, 2023.
	Jin Ziqiao. Research on 3D Human Body Reconstruction Based on Single Frame Image and SMPL[D]. Hangzhou: Zhejiang Gongshang University, 2023.
[11]	Toshev Alexander, Szegedy Christian. DeepPose: Human Pose Estimation via Deep Neural Networks[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2014: 1653-1660.
[12]	Baradel Fabien, Armando Matthieu, Galaaoui Salma, et al. Multi-HMR: Multi-person Whole-body Human Mesh Recovery in a Single Shot[C]//Computer Vision – ECCV 2024. Cham: Springer Nature Switzerland, 2025: 202-218.
[13]	Sun Xiao, Xiao Bin, Wei Fangyin, et al. Integral Human Pose Regression[C]//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 536-553.
[14]	范铭贤. 基于测试时优化的单目图像三维人体网格重建[D]. 广州: 华南理工大学, 2024.
	Fan Mingxian. Monocular Images 3D Human Mesh Reconstruction Based on Test-time Optimization[D]. Guangzhou: South China University of Technology, 2024.
[15]	Bartol Kristijan, Bojanić David, Petković Tomislav. Generalizable Human Pose Triangulation[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 11018-11027.
[16]	Tu Hanyue, Wang Chunyu, Zeng Wenjun. VoxelPose: Towards Multi-camera 3D Human Pose Estimation in Wild Environment[C]//Computer Vision – ECCV 2020. Cham: Springer International Publishing, 2020: 197-212.
[17]	Wang Tao, Zhang Jianfeng, Cai Yujun, et al. Direct Multi-view Multi-person 3D Pose Estimation[C]//Proceedings of the 35th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2021: 13153-13164.
[18]	Liao Ziwei, Zhu Jialiang, Wang Chunyu, et al. Multiple View Geometry Transformers for 3D Human Pose Estimation[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2024: 708-717.
[19]	Anguelov D, Srinivasan P, Koller D, et al. SCAPE: Shape Completion and Animation of People[C]//Seminal Graphics Papers: Pushing the Boundaries. New York: ACM, 2005: 408-416.
[20]	Kanazawa A, Black Michael J, Jacobs D W, et al. End-to-end Recovery of Human Shape and Pose[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7122-7131.
[21]	Sun Ke, Xiao Bin, Liu Dong, et al. Deep High-resolution Representation Learning for Human Pose Estimation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2019: 5686-5696.
[22]	Kolotouros N, Pavlakos G, Black M, et al. Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 2252-2261.
[23]	A A Osman Ahmed, Bolkart Timo, Black Michael J. STAR: Sparse Trained Articulated Human Body Regressor[C]//Computer Vision – ECCV 2020. Cham: Springer International Publishing, 2020: 598-613.
[24]	Sai Kumar Dwivedi, Sun Yu, Patel Priyanka, et al. TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2024: 1323-1333.
[25]	Saleem M U, Pinyoanuntapong E, WANG Pu, et al. GenHMR: Generative Human Mesh Recovery[C]//Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intelligence. Palo Alto: AAAI Press, 2025: 6749-6757.
[26]	Srivastav Vinkle, Chen Keqi, Padoy Nicolas. SelfPose3D: Self-supervised Multi-person Multi-view 3D Pose Estimation[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2024: 2502-2512.
[27]	Bogo F, Kanazawa A, Lassner Christoph, et al. Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image[C]//Computer Vision – ECCV 2016. Cham: Springer International Publishing, 2016: 561-578.
[28]	Aaron van den Oord, Vinyals O, Kavukcuoglu K. Neural Discrete Representation Learning[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 6309-6318.
[29]	Li Jiefeng, Bian Siyuan, Zeng Ailing, et al. Human Pose Regression with Residual Log-likelihood Estimation[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2021: 11005-11014.
[30]	Geng Zigang, Wang Chunyu, Wei Yixuan, et al. Human Pose as Compositional Tokens[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2023: 660-671.
[31]	Zhang Jianrong, Zhang Yangsong, Xiaodong Cun, et al. Generating Human Motion from Textual Descriptions with Discrete Representations[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2023: 14730-14740.
[32]	Belagiannis Vasileios, Amin Sikandar, Andriluka Mykhaylo, et al. 3D Pictorial Structures for Multiple Human Pose Estimation[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2014: 1669-1676.
[33]	赵晓冬, 陈凯, 黄煜杰, 等. 多目视觉下基于自动变分校正的人体3D姿态检测与建模[J/OL]. 北京航空航天大学学报. (2024-06-14) [2025-07-22]. .
	Zhao Xiaodong, Chen Kai, Huang Yujie, et al. Human 3D Posture Detection and Modeling Based on Variational Autoencoder in Multi-vision[J/OL]. Journal of Beijing University of Aeronautics and Astronautics. (2024-06-14) [2025-07-22]. .
[34]	Chen Kai, Zhao Xiaodong, Huang Yujie, et al. A Multiview Approach for Pedestrian 3D Pose Detection and Reconstruction[C]//Methods and Applications for Modeling and Simulation of Complex Systems. Singapore: Springer Nature Singapore, 2024: 87-100.
[35]	Kingma D P, Welling M. Auto-encoding Variational Bayes[EB/OL]. (2013-12-20) [2025-07-22]. .
[36]	Mahmood N, Ghorbani N, Troje N F, et al. AMASS: Archive of Motion Capture as Surface Shapes[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 5441-5450.
[37]	Ershadi-Nasab Sara, Noury Erfan, Kasaei Shohreh, et al. Multiple Human 3D Pose Estimation from Multiview Images[J]. Multimedia Tools and Applications, 2018, 77(12): 15573-15601.
[38]	Wu Size, Jin Sheng, Liu Wentao, et al. Graph-based 3D Multi-person Pose Estimation Using Multi-view Images[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2021: 11128-11137.
[39]	Belagiannis Vasileios, Amin Sikandar, Andriluka M, et al. 3D Pictorial Structures Revisited: Multiple Human Pose Estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(10): 1929-1942.
[40]	Dong Junting, Jiang Wen, Huang Qixing, et al. Fast and Robust Multi-person 3D Pose Estimation from Multiple Views[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2019: 7784-7793.

方法		PCP
方法		Act1	Act2	Act3	AVG
FS	MHPEMI^[37]	93.3	75.9	94.8	88.0
	G-MPEUMI^[38]	99.3	96.5	97.3	97.7
	MvP^[17]	99.3	95.1	97.8	97.4
	VoxelPose^[16]	99.3	94.1	97.6	97.0
OB	3DPS^[39]	75.3	69.7	87.6	77.5
	MvPose^[40]	98.8	94.1	92.8	95.2
	ours	99.1	93.5	97.6	96.7

方法	MPJPE/mm	PCP/%
梯度下降 + VAE	92.1	76.0
VAE + IK-VQ-VAE	86.3	93.2
联合训练+ IK-VQ-VAE	88.7	96.7