Research on Real-Time Motion Matching of Shadow Play Based on Kinect

doi:10.16182/j.issn1004731x.joss.20-0767

Abstract

Abstract:

In the inheritance of shadow play culture, due to the aging of the audience and the discontinuity of inheritance, the shadow play culture is gradually facing decline. Real-time matching of shadow play movements based on Kinect can inject new vitality into traditional shadow play culture. According to the characteristics of shadow play, a joint point shadow play model is constructed, and the static digitization of shadow play is realized. The human body depth image is obtained based on Kinect, and the human skeleton point coordinates are obtained through segmentation mask and machine learning to generate the human skeleton.Bone binding and weight setting are performed on the joint point shadow play model, and the human body data collected by Kinect are bound to the shadow play. Through the traversal of the skeleton tree and the action matching algorithm, the coordinate mapping is completed to realize the real-time matching between the human body and the shadow play action, thereby completing the digitalization of the dynamic performance of the shadow play.

Key words: joint point shadow model, Kinect, depth data collection, motion capture, real-time motion matching

CLC Number:

TP391.9

Chuanqian Tang, Zhiqiang Liu, Yijun Su, Xiaojing Liu. Research on Real-Time Motion Matching of Shadow Play Based on Kinect[J]. Journal of System Simulation, 2022, 34(3): 633-639.

Figures/Tables 17

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Fig. 11

Fig. 12

Fig. 13

Fig. 14

Fig. 15

Fig. 16

References 16

1	孟宪宁. 数字动画语境下民间皮影艺术的传承与创新[J]. 大众文艺, 2018(6): 170.
	Meng Xianning. Inheritance and Innovation of Folk Shadow Art in the Context of Digital Animation[J]. Popular Literature and Art, 2018(6): 170.
2	杨军燕. 对皮影戏衰落的思考[J]. 艺术教育, 2006(5): 14-15.
	Yang Junyan. Thoughts on the Decline of Shadow Play[J]. Art Education, 2006(5): 14-15.
3	李保强. 面向皮影戏动画制作的运动捕捉技术研究[D].泉州: 华侨大学, 2016.
	Li Baoqiang. Motion Capture Technology Research for Shadow Play Animation[D]. Quanzhou: Huaqiao University, 2016.
4	金波. 基于3ds max的角色动画制作研究[J]. 三峡大学学报(人文社会科学版), 2017, 39(增2): 162-163.
	Jin Bo. Research on Character Animation Production Based on 3ds Max[J]. Journal of China Three Gorges University (Humanities and Social Sciences Edition), 2017, 39(S2): 162-163.
5	单超杰. 皮影人物造型与三维建模技术结合的创新研究[D]. 上海: 东华大学, 2013.
	Shan Chaojie. Innovative Research on the Combination of Shadow Figure Modeling and 3D Modeling Technology[D]. Shanghai: Donghua University, 2013.
6	Vogel D, Lubos P, Steinicke F. AnimationVR - Interactive Controller-Based Animating in Virtual Reality[C]// 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 2018.
7	Chen Z, Liu W, Li W, et al. Kinect-Based Behavior Measurement in Group Discussion[C]// 2019 the World Symposium on Software Engineering (WSSE 2019). Wuhan: SCIence and Engineering Institute (SCIEI), 2019: 126-130.
8	刘俊毅. 彩色图像引导的深度图像增强[D]. 杭州: 浙江大学, 2014.
	Liu Junyi.Depth Map Enhancement Under the Guidance of Color Image[D]. Hangzhou: Zhejiang University, 2014.
9	邱望标, 李超. 基于运动捕捉技术的中国少数民族舞蹈艺术保护方法研究[J]. 电子科技大学学报(社科版), 2009, 11(4): 101-104.
	Qiu Wangbiao, Li Chao. Research of Protection Methods about the Dances Artistic of National Minorities Based on Motion Capture Technology[J]. Journal of University of Electronic Science and Technology of China (Social Science Edition), 2009, 11(4): 101-104.
10	王亮, 胡卫明, 谭铁牛. 人运动的视觉分析综述[J]. 计算机学报, 2002(3): 225-237.
	Wang Liang, Hu Weiming, Tan Tieniu. A Survey of Visual Analysis of Human Motion[J]. Chinese Journal of Computers, 2002(3): 225-237.
11	朱洪堃, 殷佳炜, 冯文宇, 等. 一种轻量化实时人体姿势检测模型研究与应用[J/OL]. 系统仿真学报: 1-14[2020-10-11]..
	Zhu Hongkun, Yin Jiawei, Feng Wenyu, et al. Research and Application of a Lightweight Real-Time Human Posture Detection Model[J/OL]. Journal of System Simulation: 1-14 [2020-10-11] ..
12	陈璇, 张明敏, 潘志庚. 数字皮影的交互仿真和动画绘制[J]. 中国图象图形学报, 2014, 19(10): 1490-1499.
	Chen Xuan, Zhang Mingmin, Pan Zhigeng. Interaction and Animation Simulation of Digital Shadow Play[J]. Journal of Image and Graphics, 2014, 19(10): 1490-1499.
13	赵瑛, 王冬晖, 李琦, 等. 仿真假体视觉下基于深度图像的手势识别研究[J]. 现代电子技术, 2019, 42(16): 131-135, 139.
	Zhao Ying, Wang Donghui, Li Qi, et al. Research on Gesture Recognition Based on Depth Image Under Simulated Prosthetic Vision[J]. Modern Electronic Technology, 2019, 42(16): 131-135, 139.
14	Pan Y, Mitchell K. PoseMMR: A Collaborative Mixed Reality Authoring Tool for Character Animation[C]// 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). Atlanta, GA,USA: IEEE, 2020.
15	朱大诚. 皮影风格动画研究[D]. 济南: 山东大学, 2014.
	Zhu Dacheng. Research of Shadow Graph-style Animation [D]. Jinan: Shandong University, 2014.
16	冀南. 涉足三维领域创新民族动画古老皮影幻化新生[J]. 国家人文历史, 2018(2): 12-13.
	Ji Nan. Involved in the Three-Dimensional Field, Innovating National Animation, Ancient Shadow Puppets Turned into a New Life[J]. National Humanities History, 2018(2): 12-13.

Depth data	Bit
Bits of Depth	D15~D3
Bits of Index	D2~D0

[1]	Muqing Wang, Lei Zhang, Xiumin Fan, Xiaomeng Luo, Wenmin Zhu. Simulation Method of Virtual Human Pose Optimization Based On VR Peripherals [J]. Journal of System Simulation, 2022, 34(6): 1296-1303.
[2]	Li Qi, Wang Xiangdong, Li Hua. 3D Human Pose Tracking Approach Based on Double Kinect Sensors [J]. Journal of System Simulation, 2020, 32(8): 1446-1454.
[3]	Xu Lianrui, Zhang Jinming. Modeling Indoor environment with Kinect [J]. Journal of System Simulation, 2019, 31(12): 2643-2651.
[4]	Ji Xianli, Ding Yulong, Yang Shuanghua. Intelligent Sandbox for Geography Teaching [J]. Journal of System Simulation, 2019, 31(12): 2816-2828.
[5]	Tianyu Huang, Yunying Guo. Research of Nonlinear Time Series Prediction Method for Motion Capture [J]. Journal of System Simulation, 2018, 30(7): 2808-2815.
[6]	Wei Yi, Xia Shihong. Synthesizing Rotational Human Motion with Physical Reality [J]. Journal of System Simulation, 2018, 30(6): 2190-2197.
[7]	Zhu Chaozheng, He Ming, Yang Sheng, Yang Cheng, Song Zhenhua. Real-time Reconstruction and Simulation of Police Dog Posture Based on Multi-sensor Data Vest [J]. Journal of System Simulation, 2018, 30(12): 4738-4746.
[8]	He Peng, Hu Shaojun, He Dongjian. Trajectory Capture and Sway Frequency Analysis of Trees Based on Kinect [J]. Journal of System Simulation, 2017, 29(7): 1411-1418.
[9]	Fang Zhili, Wen Weiliang, Guo Xinyu, Wu Jianwei. Skeleton Extraction from Three-dimensional Maize Based on Kinect [J]. Journal of System Simulation, 2017, 29(3): 524-530.
[10]	Pan Zhigeng, Liu Congjin, Ge Yingying, Yuan Qingshu. Design and Implementation of Virtual Treadmill System Supporting Natural Interaction [J]. Journal of System Simulation, 2017, 29(11): 2753-2759.
[11]	Ban Rui, Ding Dandan, Zhang Mingmin, Shen Huaqing. Design and Implementation of Online Interactive Educational Game Based on Somatosensory [J]. Journal of System Simulation, 2017, 29(11): 2890-2897.
[12]	Cui Lijun, Huang Tianyu, Feng Feng, Zhang Jie, Yang Kai, Liu Dong. Motion Reconstruction and Simulation Using Sparse Inertial Sensors [J]. Journal of System Simulation, 2017, 29(10): 2261-2267.
[13]	Zhang Xiaoli, Yao Junfeng, Huang Ping. 360-Degree Virtual Fitting Based on Kinect [J]. Journal of System Simulation, 2016, 28(10): 2378-2385.
[14]	Chen Hao, Liu Zhen, Chen Tian, Liu Tingting, Liu Cuijuan, Zhu Xianying. Kinect Based Serious Game of Entertainment and Body Exercise for Elderly [J]. Journal of System Simulation, 2016, 28(10): 2586-2592.
[15]	Huo Yuping, Zhang Xiu’e, Li Bing, Li Weiqing. Research on Motion Capture in Substation Virtual Environment [J]. Journal of System Simulation, 2016, 28(10): 2632-2637.