Maintenance Operation Control of Virtual Human in Collaborative Virtual Maintenance

doi:10.16182/j.issn1004731x.joss.20-0266

Abstract

Abstract: Aiming at the operators being not able to move freely in the limited capture area of optical motion capture devices, the maintenance operation control of virtual human in Collaborative Virtual Maintenance is studied. The lower limb action of operators is captured by the optical motion capture devices, and the wide range motion of the virtual human is controlled according to the real-time recognition results. In order to control the hand motion process of the virtual human in the maintenance operation, the hand interaction of the virtual human is analyzed and an intelligent assistant method for the maintenance operations based on the dynamic Bayesian network is proposed. A collaborative virtual maintenance operation simulation platform of a hydraulic cylinder is taken to prove the feasibility of the method. The method can control the maintenance operation process of virtual human effectively, and has good performance of real-time wish, interactivity and perceptibility.

Key words: virtual human, action recognition, motion control, maintenance operation

CLC Number:

TP391.9

Liang Feng, Zhang Zhili, Li Xiangyang. Maintenance Operation Control of Virtual Human in Collaborative Virtual Maintenance[J]. Journal of System Simulation, 2020, 32(7): 1201-1210.

References 13

[1]	高翔, 安辉, 陈为, 等. 移动增强现实可视化综述[J]. 计算机辅助设计与图形学学报, 2018, 30(1): 1-8.Gao Xiang, An Hui, Chen Wei, et al.A Survey on Mobile Augmented Reality Visualization[J]. Journal of Computer-Aided Design & Computer Graphics, 2018, 30(1): 1-8.
[2]	文国军, 夏雨, 王玉丹, 等. 基于Unity3D的水平定向钻机虚拟实训系统设计[J]. 系统仿真学报, 2020, 32(5): 801-807.Wen Guojun, Xia Yu, Wang Yudan, et al.Design of Virtual Training System for Horizontally Oriented Drill Based on Unity3D[J]. Journal of System Simulation, 2020, 32(5): 801-807.
[3]	胡凌燕, 史康柏, 张碧纯, 等. 在线修正虚拟仿真预测的遥操作机器人系统[J]. 系统仿真学报, 2020, 32(2): 247-254.Hu Lingyan, Shi Kangbai, Zhang Bichun, et al.Tele-robot Control System with Online Virtual Simulation Prediction[J]. Journal of System Simulation, 2020, 32(2): 247-254.
[4]	高杰, 刘鹏, 李蔚清. 基于质量评价的运动跟踪数据融合方法[J]. 合肥工业大学学报(自然科学版), 2019, 42(5): 646-650, 720.Gao Jie, Liu Peng, Li Weiqing.Data Fusion Method of Motion Tracking Based on Quality Evaluation[J]. Journal of Hefei University of Technology (Natural Science), 2019, 42(5): 646-650, 720.
[5]	杨华, 苏势林, 闫雨奇, 等. 基于关键姿态约束的人体运动序列插值生成[J]. 沈阳航空航天大学学报, 2019, 36(3): 59-65.Yang Hua, Su Shilin, Yan Yuqi, et al.Human Motion Sequence Interpolation and Generation Based on Key-pose Constraints[J]. Journal of Shenyang Aerospace University, 2019, 36(3): 59-65.
[6]	张青, 郑岩, 郭庆, 等. 航空发动机设备拆装快速装配仿真研究[J]. 计算机仿真, 2018, 35(3): 257-262.Zhang Qing, Zheng Yan, Guo Qing, et al.Simulation Research on Rapid Assembly and Disassembly of Aeroengine Based on Motion Capture Equipment[J]. Computer Simulation, 2018, 35(3): 257-262.
[7]	Hyun D J, Lim H, Park S, et al.Walking propulsion generation in double stance by powered exoskeleton for paraplegics[J]. Robotics and Autonomous Systems, 2019, 116: 24-37.
[8]	梁丰, 张志利, 李向阳, 等. 人体运动捕捉过程中的人体下肢动作识别[J]. 计算机辅助设计与图形学学报, 2015, 27(12): 2419-2427.Liang Feng, Zhang Zhili, Li Xiangyang, et al.Simulation Research on Rapid Assembly and Disassembly of Aeroengine Based on Motion Capture Equipment[J]. Journal of Computer-Aided Design & Computer Graphics, 2015, 27(12): 2419-2427.
[9]	梁丰, 张志利, 李向阳, 等. 协同虚拟维修中体感交互控制技术研究[M]. 西安: 西北工业大学出版社, 2019.Liang Feng, Zhang Zhili, Li Xiangyang, et al.Research on Somatosensory Interactive Control Technology in Collaborative Virtual Maintenance[M]. Xi’an: Northwest Industrial University Press, 2019.
[10]	姜盛乾. 基于虚拟现实技术的装配及人因评价研究[D]. 长春: 吉林大学, 2019.Jiang Shengqian.Research on Assembling and Human Factor Evaluation Based on Virtual Reality Technology[D]. Changchun: Jilin University, 2019.
[11]	杨子辉. 大规模辐射场景三维实时仿真关键技术研究[D]. 合肥: 中国科学技术大学, 2016.Yang Zihui.Study on Key Technologies of Three-dimensional Real-time Simulation for Large-scale Radiation Scenatio[D]. Hefei: University of Science and Technology of China, 2016.
[12]	李彬, 崔汉国, 李正民, 等. 基于共享子网合成Petri网的协同虚拟维修系统[J]. 中国修船, 2015, 28(5): 45-49.Li Bin, Cui Hanguo, Li Zhengmin, et al.Collaborative Virtual Maintenance System Based on Synthesis of Petri Nets Sharing a Kind of Subnet[J]. China Shiprepair, 2015, 28(5): 45-49.
[13]	徐文胜, 武博, 蒋坚鸿. 武器装备虚拟维修训练系统行为树设计与实现[J]. 系统仿真学报, 2018, 30(7): 2722-2728.Xu Wensheng, Wu Bo, Jiang Jianhong.Design and Realization of Behavior Tree in Weapon Equipment Virtual Maintenance Training System[J]. Journal of System Simulation, 2018, 30(7): 2722-2728.

[1]	Muqing Wang, Lei Zhang, Xiumin Fan, Xiaomeng Luo, Wenmin Zhu. Simulation Method of Virtual Human Pose Optimization Based On VR Peripherals [J]. Journal of System Simulation, 2022, 34(6): 1296-1303.
[2]	Li Yihao, Huang Tianyu, Li Peng, Ding Gangyi. Research on Performance Modeling and Simulation Method Based on Dense Crowd [J]. Journal of System Simulation, 2021, 33(7): 1617-1625.
[3]	Zhao Ying, Lu Yao, Zhang Jian, Liang Qidi, Long Wei. Multi-view Human Action Recognition Based on Deep Neural Network [J]. Journal of System Simulation, 2021, 33(5): 1019-1030.
[4]	Shen Yi, Wu Gang, Zhou Rui. Auxiliary Application of Wearable Computing Equipment in Industrial Intelligent Operation and Maintenance [J]. Journal of System Simulation, 2021, 33(5): 1188-1195.
[5]	Yang Weilong, Xu Kai, Xie Xu, Sun Lin. Research on CGF-oriented Virtual Human Perceptual Attention Model [J]. Journal of System Simulation, 2021, 33(2): 262-270.
[6]	Tang Chao, Zhang Miaohui, Li Wei, Cao Feng, Wang Xiaofeng, Tong Xiaohong. Fusing Local and Global Features for Human Action Recognition [J]. Journal of System Simulation, 2018, 30(7): 2497-2506.
[7]	Li Mingxiao, Geng Qichuan, Mo Hong, Wu Wei, Zhou Zhong. Video Action Recognition Based on Key-frame [J]. Journal of System Simulation, 2018, 30(7): 2787-2793.
[8]	Tang Chao, Zhang Miaohui, Li Wei, Cao Feng, Wang Xiaofeng, Tong Xiaohong. Human Action Recognition Based on Depth Image [J]. Journal of System Simulation, 2018, 30(5): 1641-1649.
[9]	Yi Zhuo, Liao Ying, Hu Xiaofeng, Du Xuehui, Zhu Feng. A deep spatio-temporal RNNs based coordinated operational action recognition [J]. Journal of System Simulation, 2018, 30(3): 793-800.
[10]	Yang Yi, Weng Dongdong, Xue Song, Su Mo. Study of Virtual Human Voice Interaction on the Soothing Effects of Anxiety [J]. Journal of System Simulation, 2016, 28(10): 2578-2585.
[11]	Liang Feng, Zhang Zhili, Li Xiangyang, Tang Zhibo. Processing Method of Space Locations of Virtual Human Based on Wavelet Transform and Kalman Filter [J]. Journal of System Simulation, 2016, 28(1): 213-219.
[12]	Lv Desheng, Guo Yan. Design & Implement on Virtual Scheduling System of Great Square Art Performance [J]. Journal of System Simulation, 2015, 27(8): 1895-1901.
[13]	Liang Feng, Zhang Zhili, Li Xiangyang, Tang Zhibo, Ma Chao. Research on Motion Control Technology of Virtual Human’s Lower Limb Based on Optical Motion Capture Data [J]. Journal of System Simulation, 2015, 27(2): 327-335.
[14]	Yang Yunfei, Gu Yan, Fan Xiumin, He Qichang, Qiu Shiguang. A Research of Virtual Human Assembly Intelligent Simulation Technology [J]. Journal of System Simulation, 2015, 27(10): 2320-2327.
[15]	Shi Xiangbin, Liu Shuanpeng, Zhang Deyuan. Human Action Recognition Method Based on Key Frames [J]. Journal of System Simulation, 2015, 27(10): 2401-2408.