基于手势交互的虚实融合实验仿真技术

doi:10.16182/j.issn1004731x.joss.20-0558

系统仿真学报 ›› 2021, Vol. 33 ›› Issue (10): 2488-2498.doi: 10.16182/j.issn1004731x.joss.20-0558

• 仿真支撑平台/系统技术 • 上一篇下一篇

基于手势交互的虚实融合实验仿真技术

邹序焱^1,3, 何汉武^1,2, 吴悦明¹, 邓景威¹

1.广东工业大学机电工程学院,广东广州 510006;
2.广东工贸职业技术学院,广东广州 510510;
3.宜宾学院人工智能与大数据学部,四川宜宾 644007

收稿日期:2020-08-04 修回日期:2020-09-22 出版日期:2021-10-18 发布日期:2021-10-18
作者简介:邹序焱(1983-),男,博士生,讲师,研究方向为增强现实及人工智能。E-mail：xyzou@yibinu.edu.cn
基金资助:
国家重点研发专项(2018YFB1004902); 广东省重点研发项目(2017B010110008)

Virtual and Real Fusion Experiment Simulation Technology Based on Gesture Interaction

Zou Xuyan^1,3, He Hanwu^1,2, Wu Yueming¹, Deng Jingwei¹

1. Guangdong University of Technology, Guangzhou 510006, China;
2. Guangdong Polytechnic of Industry and Commerce, Guangzhou 510510, China;
3. Department of Artificial Intelligence and Big Data, Yibin university, Yibin 644007, China

Received:2020-08-04 Revised:2020-09-22 Online:2021-10-18 Published:2021-10-18

摘要/Abstract

摘要： 在实验教学中,有些实验因实验环境、实验设备、师资等限定而不能开展。为了解决上述问题,探讨了利用深度相机构建虚实融合仿真实验的技术。利用Aruco标记算法实现了对虚拟场景与真实场景一一对应配准,结合深度相机实时采集的彩色图像和深度图像,构建出虚实融合的实验环境。为了达到操作虚拟器材的目的,利用阈值分割和成像原理,提出了一种手势交互的方法。数值实验表明位姿配准和手势交互方法基本满足虚拟实验的应用要求,为虚拟实验的构建提供了一套可行的技术方案。

关键词: 虚拟实验, 虚实配准, 实时融合, 手势交互, 深度相机

Abstract: In experimental teaching, some traditional experiments cannot be carried out due to limitations of experimental environment, experimental equipment, and faculty. In order to solve the above problems, the method of constructing virtual and real fusion simulation experiment using depth cameras is explored.. The Aruco labeling algorithm is used to achieve one-to-one registration of the virtual scene and the real scene, and the experimental environment of virtual and real fusion is constructed combining the colour images and depth images collected by the depth camera in real time. In order to achieve the purpose of operating virtual equipment, a gesture interaction method is proposed based on threshold segmentation and imaging principles. Numerical experiments show that pose registration and gesture interaction methods basically meet the application requirements of virtual experiments, and provide a set of feasible technical solutions for the construction of virtual experiments

Key words: virtual experiment, virtual-real registration, real-time fusion, gesture interaction, depth camera

中图分类号:

TP391.9

邹序焱, 何汉武, 吴悦明, 邓景威. 基于手势交互的虚实融合实验仿真技术[J]. 系统仿真学报, 2021, 33(10): 2488-2498.

Zou Xuyan, He Hanwu, Wu Yueming, Deng Jingwei. Virtual and Real Fusion Experiment Simulation Technology Based on Gesture Interaction[J]. Journal of System Simulation, 2021, 33(10): 2488-2498.

参考文献

[1] 卢宏茜. 高中化学实验教学现状及优化对策研究[D].汉中: 陕西理工大学, 2018.
Lu Hongxi.Study on the Current Situation of Chemical experiment Teaching in High School and the Optimization Countermeasures[D]. Hanzhong: Shaanxi University of Technology, 2018.
[2] 刘智远. 农村初中物理探究式实验教学状况调查研究[D]. 哈尔滨: 哈尔滨师范大学, 2018.
Liu Zhiyuan.Investigation and Research on the Physical Inquiry Teaching in Rural Junior Middle School[D]. Harbin: Harbin Normal University, 2018.
[3] 李祥会, 王国英, 王红菊, 等. 痰液中结核分枝杆菌检测的虚拟仿真实验建设[J]. 中国病原生物学杂志, 2020, 15(5): 616-619.
Li Xinghui, Wang Guoying, Wang Hongju, et al.The Dessign and Implementation of an Experiment Virtually Simulating Detection of Mycobacterium Tuberculosis in Sputum[J]. Journal of Pathogen Biology, 2020, 15(5): 616-619.
[4] 金文, 陈曦, 韩洪洪. 电动机拖动虚拟实验设计及在教学中的应用[J]. 实验室研究与探索, 2020, 39(3): 211-214.
Jin Wen, Chen Xi, Han Honghong.Development of Virtual Experiments of AC Motor Drive and Its Application in the Teaching[J]. Research and Exploration in Laboratory, 2020, 39(3): 211-214.
[5] Wei X, Weng D, Liu Y, et al.Teaching based on Augmented Reality for a Technical Creative Design Course[J]. Computers & Education (S0360-1315), 2015, 81: 221-234.
[6] Dede C.Immersive Interfaces for Engagement and Learning[J]. Science (S0036-8075), 2009, 323(5910): 66-69.
[7] Billinghurst M, Duenser A.Augmented Reality in the Classroom[J]. Computer (S1558-0814), 2012, 45(7): 56-63.
[8] Pang S, Wang M, Zhang X.Research on Virtual Simulation Experiment of Engine Room Resource Management[C]// 2019 IEEE 5th International Conference on Mechatronics System and Robots (ICMSR). Singapore: IEEE, 2019: 125-128.
[9] Li G, Xu B, Zhang Y.Measurement of Core Permeability in "Reservoir Physics" by Virtual Simulation Experiment[C]// 2019 International Conference on Virtual Reality and Intelligent Systems (ICVRIS). Jishou, China: IEEE, 2019: 23-26.
[10] Liu C, Yao H, Hu C, et al.A Cloud-based Virtual Experiment Platform for Computer Hardware Teaching[C]// 2019 IEEE International Conference on Computer Science and Educational Informatization (CSEI). Kunming, China: IEEE, 2019: 68-71.
[11] Yuan J, Feng Z, Dong D, et al.Research on Multimodal Perceptual Navigational Virtual and Real Fusion Intelligent Experiment Equipment and Algorithm[J]. In IEEE Access (S2169-3536), 2020, 8: 43375-43390.
[12] Liu H, Liu D.An Intelligent Recognition Method of Circuit Connection in Three-dimensional Virtual Experiments[C]// 2019 IEEE International Conference on Computer Science and Educational Informatization (CSEI). Kunming, China: IEEE, 2019: 329-333.
[13] Chen C S, Chang W Y.Pose Estimation for Generalized Imaging Device via Solving Non-perspective N Point Problem[C]// 2002 IEEE International Conference on Robotics and Automation. Washington: IEEE, 2002: 2931-2937.
[14] 周克旻, 周蓉, 滕婧, 等. 多标记室内小型无人机定位与姿态估计方法[J]. 计算机工程与应用, 2019, 55(22): 231-238.
Zhou Kemin, Zhou Rong, Teng Jing, et al.Method of Location and Attitude Estimation for Small Indoor UAV with Multiple Markers[J]. Computer Engineering and Applications, 2019, 55(22): 231-238.
[15] 王营华, 宋光明, 刘盛松, 等. 一种视觉引导的作业型飞行机器人设计[J]. 机器人, 2019, 41(3): 353-361.
Wang Yinghua, Song Guangming, Liu Shengsong, et al.Design of a Vision-Guided Aerial Manipulator[J]. Robot, 2019, 41(3): 353-361.
[16] 付长斐, 叶宾, 李会军. 基于HSV颜色空间的运动目标识别[J]. 控制与信息技术, 2020(2): 70-74.
Fu Changfei, Ye Bin, Li Huijun.Moving Target Recognition Based on HSV Color Space[J].Control and Information Technology, 2020(2): 70-74.
[17] 高向东, 罗锡柱. 一种基于图像质心的焊缝跟踪新方法[J]. 焊接学报, 2003, 24(6): 15-18.
Gao Xiangdong, Luo Xizhu.An Image Centroid Method for Seam Tracking in Gas Tungsten Arc Welding[J]. Transactions of the China Welding Institution, 2003, 24(6): 15-18.
[18] 周亢, 闫建国, 屈耀红. 全角度姿态角解算方法研究与仿真[J]. 系统仿真学报, 2009, 21(6): 1697-1700.
Zhou Kang, Yan Jianguo, Qu Yaohong.Research and Simulation on Method of Calculating Full-scale Attitude[J]. Journal of System Simulation, 2009, 21(6): 1697-1700.

基于手势交互的虚实融合实验仿真技术

Virtual and Real Fusion Experiment Simulation Technology Based on Gesture Interaction

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

本文评价

[1]	许世健, 赵丹, 苏铖宇, 王赋攀, 张晓蓉, 王昉, 吴亚东. 面向流场可视化的沉浸式虚拟现实交互系统研究[J]. 系统仿真学报, 2022, 34(5): 1160-1172.
[2]	阴敬方, 朱登明, 石敏, 王兆其. 基于引导对抗网络的人体深度图像修补方法[J]. 系统仿真学报, 2020, 32(7): 1312-1321.
[3]	宋中越, 甄冬, 张浩, 师占群, 马姣姣. 基于联合仿真的风力发电机试验台系统研究[J]. 系统仿真学报, 2018, 30(2): 740-746.