Research on Real-time Animatable Human Avatar Generation via 3D Gaussian Splatting

doi:10.16182/j.issn1004731x.joss.25-0829

Abstract

Abstract:

Real-time animatable 3D human avatar generation technology hold significant application value in fields such as virtual reality and remote collaboration. To address the limitations of existing methods in detail modeling, real-time performance, and robustness under novel pose driving, an efficient human avatar generation and driving method based on 3D Gaussian splatting (3DGS) is proposed. This method integrates optimized parametric human reconstruction, tri-plane feature encoding, and dynamic offset prediction to achieve efficient modeling from monocular video input. By introducing a skeleton binding and visibility analysis strategy, while designing a multi-scale regularization loss to address the overfitting problem. Simulation experiments demonstrate that the proposed method achieves outstanding performance across all evaluation metrics, particularly in novel pose driving and occluded scenarios, validating its effectiveness and superiority.

Key words: 3D Gaussian splatting (3DGS), animatable human avatars, monocular video, real-time rendering, parametric model

CLC Number:

TP391.9

Zhong Yuyou, Shen Xukun, Hu Yong. Research on Real-time Animatable Human Avatar Generation via 3D Gaussian Splatting[J]. Journal of System Simulation, 2026, 38(1): 200-210.

Figures/Tables 10

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Comparative experimental results

方法	NeuMan			ZJU-MoCap			People-Snapshot			FPS $↑$
方法	PSNR $↑$	SSIM $↑$	LPIPS $↓$	PSNR $↑$	SSIM $↑$	LPIPS $↓$	PSNR $↑$	SSIM $↑$	LPIPS $↓$	FPS $↑$
HumanNeRF	27.06	0.967	0.019	29.29	0.967	0.030	25.68	0.956	0.023	0.2
InstantAvatar	28.47	0.972	0.028	29.73	0.938	0.068	28.60	0.971	0.018	4.15
NeuMan	29.32	0.972	0.014	28.96	0.948	0.061	—	—	—	0.1
Vid2Avatar	31.18	0.982	0.025	15.72	0.942	0.060	28.70	0.975	0.042	0.005
GaussianAvatar	29.94	0.980	0.012	24.38	0.932	0.081	30.27	0.978	0.019	27.2
3DGS-Avatar	28.99	0.974	0.016	29.76	0.970	0.028	32.42	0.965	0.018	40.3
本文	32.09	0.982	0.012	31.65	0.989	0.011	32.82	0.982	0.017	45.8

Table 1

Fig. 5

Fig. 6

Table 2

Ablation results of parametric human model

参数是否优化	Epoch	PSNR $↑$	SSIM $↑$	LPIPS $↓$
不优化	100	27.82	0.966	0.020
不优化	300	31.05	0.974	0.015
优化	100	31.47	0.978	0.015
优化	300	32.09	0.982	0.012

Table 2

Fig. 7

Fig. 8

References 26

[1]	Weng C Y, Curless B, Kemelmacher-Shlizerman I. Photo Wake-up: 3D Character Animation from a Single Photo[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2019: 5901-5910.
[2]	Zackariasson P, Wilson T L. The Video Game Industry: Formation, Present State, and Future[M]. New York: Routledge, 2012.
[3]	Li Nianlong, Zhang Zhengquan, Liu Can, et al. vMirror: Enhancing the Interaction with Occluded or Distant Objects in VR with Virtual Mirrors[C]//Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. New York: Association for Computing Machinery, 2021: 132.
[4]	Liu Yang, Qiu Changzhen, Zhang Zhiyong. Deep Learning for 3D Human Pose Estimation and Mesh Recovery: A Survey[J]. Neurocomputing, 2024, 596: 128049.
[5]	Mildenhall B, Srinivasan P P, TANCIK M, et al. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis[C]//Computer Vision – ECCV 2020. Cham: Springer International Publishing, 2020: 405-421.
[6]	Kerbl B, Kopanas G, Leimkuehler T, et al. 3D Gaussian Splatting for Real-time Radiance Field Rendering[J]. ACM Transactions on Graphics, 2023, 42(4): 139.
[7]	Li Zhe, Zheng Zerong, Wang Lizhen, et al. Animatable Gaussians: Learning Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2024: 19711-19722.
[8]	Hu Shoukang, Hu Tao, Liu Ziwei. GauHuman: Articulated Gaussian Splatting from Monocular Human Videos[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2024: 20418-20431.
[9]	Hu Liangxiao, Zhang Hongwen, Zhang Yuxiang, et al. GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2024: 634-644.
[10]	Bhatnagar B L, Sminchisescu C, Theobalt C, et al. LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2020: 12909-12922.
[11]	Loper M, Mahmood N, Romero J, et al. SMPL: A Skinned Multi-person Linear Model[J]. ACM Transactions on Graphics, 2015, 34(6): 248.
[12]	Weng C Y, Curless B, Srinivasan P P, et al. HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 16189-16199.
[13]	Jiang Wei, Yi K M, Samei G, et al. NeuMan: Neural Human Radiance Field from a Single Video[C]//Computer Vision – ECCV 2022. Cham: Springer Nature Switzerland, 2022: 402-418.
[14]	Guo Chen, Jiang Tianjian, Chen Xu, et al. Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2023: 12858-12868.
[15]	Peng Sen, Xie Weixing, Wang Zilong, et al. RMAvatar: Photorealistic Human Avatar Reconstruction from Monocular Video Based on Rectified Mesh-embedded Gaussians[EB/OL]. (2025-01-13) [2025-09-13]. .
[16]	Chen Jianchuan, Hu Jingchuan, Wang Gaige, et al. TaoAvatar: Real-time Lifelike Full-body Talking Avatars for Augmented Reality via 3D Gaussian Splatting[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2025: 10723-10734.
[17]	Pavlakos G, Choutas V, Ghorbani N, et al. Expressive Body Capture: 3D Hands, Face, and Body from a Single Image[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2019: 10967-10977.
[18]	Baradel F, Armando M, Galaaoui S, et al. Multi-HMR: Multi-person Whole-body Human Mesh Recovery in a Single Shot[C]//Computer Vision – ECCV 2024. Cham: Springer Nature Switzerland, 2025: 202-218.
[19]	Jiang Tao, Lu Peng, Zhang Li, et al. RTMPose: Real-time Multi-person Pose Estimation based on MMPose[EB/OL]. (2023-07-03) [2025-09-13]. .
[20]	Ravi N, Gabeur V, Hu Yuanting, et al. SAM 2: Segment Anything in Images and Videos[EB/OL]. (2024-10-28) [2025-09-13]. .
[21]	Jin Sheng, Xu Lumin, Xu Jin, et al. Whole-body Human Pose Estimation in the Wild[C]//Computer Vision – ECCV 2020. Cham: Springer International Publishing, 2020: 196-214.
[22]	Zhang R, Isola P, Efros A A, et al. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 586-595.
[23]	Peng Sida, Zhang Yuanqing, Xu Yinghao, et al. Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 9050-9059.
[24]	Alldieck T, Magnor M, Xu Weipeng, et al. Video Based Reconstruction of 3D People Models[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 8387-8397.
[25]	Jiang Tianjian, Chen Xu, Song Jie, et al. InstantAvatar: Learning Avatars from Monocular Video in 60 Seconds[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2023: 16922-16932.
[26]	Qian Zhiyin, Wang Shaofei, Mihajlovic M, et al. 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2024: 5020-5030.