基于Kinect的皮影动作实时匹配研究

doi:10.16182/j.issn1004731x.joss.20-0767

摘要/Abstract

摘要：

皮影文化传承中由于受众老龄化和传承断代化使得皮影文化逐渐面临没落，基于Kinect实现皮影动作实时匹配能够为传统皮影文化注入新的活力。根据皮影特点构建关节点皮影模型，实现了皮影的静态数字化。基于Kinect获取人体深度图像，通过分割遮罩和机器学习获取人体骨骼点坐标，生成人体骨架。对关节点皮影模型进行骨骼绑定与权重设置，将Kinect采集到的人体数据绑定到皮影上。通过骨骼树的遍历和动作匹配算法完成坐标映射，实现人体与皮影动作的实时匹配，从而完成皮影的动态表演数字化。

关键词: 关节点皮影模型, Kinect, 深度数据采集, 动作捕捉, 动作实时匹配

Abstract:

In the inheritance of shadow play culture, due to the aging of the audience and the discontinuity of inheritance, the shadow play culture is gradually facing decline. Real-time matching of shadow play movements based on Kinect can inject new vitality into traditional shadow play culture. According to the characteristics of shadow play, a joint point shadow play model is constructed, and the static digitization of shadow play is realized. The human body depth image is obtained based on Kinect, and the human skeleton point coordinates are obtained through segmentation mask and machine learning to generate the human skeleton.Bone binding and weight setting are performed on the joint point shadow play model, and the human body data collected by Kinect are bound to the shadow play. Through the traversal of the skeleton tree and the action matching algorithm, the coordinate mapping is completed to realize the real-time matching between the human body and the shadow play action, thereby completing the digitalization of the dynamic performance of the shadow play.

Key words: joint point shadow model, Kinect, depth data collection, motion capture, real-time motion matching

中图分类号:

TP391.9

唐传谦, 刘志强, 苏毅俊, 刘晓静. 基于Kinect的皮影动作实时匹配研究[J]. 系统仿真学报, 2022, 34(3): 633-639.

Chuanqian Tang, Zhiqiang Liu, Yijun Su, Xiaojing Liu. Research on Real-Time Motion Matching of Shadow Play Based on Kinect[J]. Journal of System Simulation, 2022, 34(3): 633-639.

图/表 17

图1

图2

图3

图4

表1

图5

图6

图7

图8

图9

图10

图11

图12

图13

图14

图15

图16

参考文献 16

1	孟宪宁. 数字动画语境下民间皮影艺术的传承与创新[J]. 大众文艺, 2018(6): 170.
	Meng Xianning. Inheritance and Innovation of Folk Shadow Art in the Context of Digital Animation[J]. Popular Literature and Art, 2018(6): 170.
2	杨军燕. 对皮影戏衰落的思考[J]. 艺术教育, 2006(5): 14-15.
	Yang Junyan. Thoughts on the Decline of Shadow Play[J]. Art Education, 2006(5): 14-15.
3	李保强. 面向皮影戏动画制作的运动捕捉技术研究[D].泉州: 华侨大学, 2016.
	Li Baoqiang. Motion Capture Technology Research for Shadow Play Animation[D]. Quanzhou: Huaqiao University, 2016.
4	金波. 基于3ds max的角色动画制作研究[J]. 三峡大学学报(人文社会科学版), 2017, 39(增2): 162-163.
	Jin Bo. Research on Character Animation Production Based on 3ds Max[J]. Journal of China Three Gorges University (Humanities and Social Sciences Edition), 2017, 39(S2): 162-163.
5	单超杰. 皮影人物造型与三维建模技术结合的创新研究[D]. 上海: 东华大学, 2013.
	Shan Chaojie. Innovative Research on the Combination of Shadow Figure Modeling and 3D Modeling Technology[D]. Shanghai: Donghua University, 2013.
6	Vogel D, Lubos P, Steinicke F. AnimationVR - Interactive Controller-Based Animating in Virtual Reality[C]// 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 2018.
7	Chen Z, Liu W, Li W, et al. Kinect-Based Behavior Measurement in Group Discussion[C]// 2019 the World Symposium on Software Engineering (WSSE 2019). Wuhan: SCIence and Engineering Institute (SCIEI), 2019: 126-130.
8	刘俊毅. 彩色图像引导的深度图像增强[D]. 杭州: 浙江大学, 2014.
	Liu Junyi.Depth Map Enhancement Under the Guidance of Color Image[D]. Hangzhou: Zhejiang University, 2014.
9	邱望标, 李超. 基于运动捕捉技术的中国少数民族舞蹈艺术保护方法研究[J]. 电子科技大学学报(社科版), 2009, 11(4): 101-104.
	Qiu Wangbiao, Li Chao. Research of Protection Methods about the Dances Artistic of National Minorities Based on Motion Capture Technology[J]. Journal of University of Electronic Science and Technology of China (Social Science Edition), 2009, 11(4): 101-104.
10	王亮, 胡卫明, 谭铁牛. 人运动的视觉分析综述[J]. 计算机学报, 2002(3): 225-237.
	Wang Liang, Hu Weiming, Tan Tieniu. A Survey of Visual Analysis of Human Motion[J]. Chinese Journal of Computers, 2002(3): 225-237.
11	朱洪堃, 殷佳炜, 冯文宇, 等. 一种轻量化实时人体姿势检测模型研究与应用[J/OL]. 系统仿真学报: 1-14[2020-10-11]..
	Zhu Hongkun, Yin Jiawei, Feng Wenyu, et al. Research and Application of a Lightweight Real-Time Human Posture Detection Model[J/OL]. Journal of System Simulation: 1-14 [2020-10-11] ..
12	陈璇, 张明敏, 潘志庚. 数字皮影的交互仿真和动画绘制[J]. 中国图象图形学报, 2014, 19(10): 1490-1499.
	Chen Xuan, Zhang Mingmin, Pan Zhigeng. Interaction and Animation Simulation of Digital Shadow Play[J]. Journal of Image and Graphics, 2014, 19(10): 1490-1499.
13	赵瑛, 王冬晖, 李琦, 等. 仿真假体视觉下基于深度图像的手势识别研究[J]. 现代电子技术, 2019, 42(16): 131-135, 139.
	Zhao Ying, Wang Donghui, Li Qi, et al. Research on Gesture Recognition Based on Depth Image Under Simulated Prosthetic Vision[J]. Modern Electronic Technology, 2019, 42(16): 131-135, 139.
14	Pan Y, Mitchell K. PoseMMR: A Collaborative Mixed Reality Authoring Tool for Character Animation[C]// 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). Atlanta, GA,USA: IEEE, 2020.
15	朱大诚. 皮影风格动画研究[D]. 济南: 山东大学, 2014.
	Zhu Dacheng. Research of Shadow Graph-style Animation [D]. Jinan: Shandong University, 2014.
16	冀南. 涉足三维领域创新民族动画古老皮影幻化新生[J]. 国家人文历史, 2018(2): 12-13.
	Ji Nan. Involved in the Three-Dimensional Field, Innovating National Animation, Ancient Shadow Puppets Turned into a New Life[J]. National Humanities History, 2018(2): 12-13.

Depth data	Bit
Bits of Depth	D15~D3
Bits of Index	D2~D0

[1]	王沐晴, 张磊, 范秀敏, 骆晓萌, 朱文敏. VR外设驱动的虚拟人姿态优化仿真方法[J]. 系统仿真学报, 2022, 34(6): 1296-1303.
[2]	李琪, 王向东, 李华. 基于双Kinect传感器的三维人体姿态跟踪方法[J]. 系统仿真学报, 2020, 32(8): 1446-1454.
[3]	徐连瑞, 张锦明. 基于Kinect的室内环境建模[J]. 系统仿真学报, 2019, 31(12): 2643-2651.
[4]	纪显俐, 丁宇龙, 杨双华. 智能沙盘地理教学演示系统[J]. 系统仿真学报, 2019, 31(12): 2816-2828.
[5]	黄天羽, 郭芸莹. 面向动作捕捉的非线性时间序列预测方法研究[J]. 系统仿真学报, 2018, 30(7): 2808-2815.
[6]	何鹏, 胡少军, 何东健. 基于Kinect的树运动轨迹获取与振动频率分析[J]. 系统仿真学报, 2017, 29(7): 1411-1418.
[7]	方志力, 温维亮, 郭新宇, 吴建伟. 基于Kinect的三维玉米植株骨架提取[J]. 系统仿真学报, 2017, 29(3): 524-530.
[8]	潘志庚, 刘从晋, 葛莹莹, 袁庆曙. 支持自然交互的虚拟跑步机系统的设计和实现[J]. 系统仿真学报, 2017, 29(11): 2753-2759.
[9]	王建国, 鄂群, 徐晓刚, 姚科明, 万新龙. 舰载直升机着舰指挥模拟训练系统[J]. 系统仿真学报, 2016, 28(2): 329-334.
[10]	张晓丽, 姚俊峰, 黄萍. 基于Kinect的实时360度虚拟试衣[J]. 系统仿真学报, 2016, 28(10): 2378-2385.
[11]	毛雁明, 章立亮. 基于Kinect深度信息的手势分割与识别[J]. 系统仿真学报, 2015, 27(4): 830-835.
[12]	蔡晨贾农, 施逸飞, 徐凯, 党岗. 基于KinectFusion的室内场景平面结构重建[J]. 系统仿真学报, 2015, 27(10): 2239-2245.
[13]	石祥滨, 刘拴朋, 张德园. 基于关键帧的人体动作识别方法[J]. 系统仿真学报, 2015, 27(10): 2401-2408.
[14]	张瑒, 许林, 孙广毅. 基于RGB-D摄像机与IMU数据融合的动作捕捉系统[J]. 系统仿真学报, 2015, 27(10): 2582-2588.