Fast generation Method of Multi-sensing Channel Fusion Virtual Experiment

doi:10.16182/j.issn1004731x.joss.21-0009

Abstract

Abstract:

Aiming at the low development efficiency and single experience in traditional virtual experiments, a rapid generation method of multi-sensing channel fusion virtual experiment visualization is proposed. The experimental steps and sequence of experimental steps of the virtual experiment are defined. A parameterized description method of experimental elements based on Petri net is proposed to describe the sequence of experimental steps, which breaks through the constraints the established procedure steps of traditional virtual experiments and supports the exploratory virtual experiments. The visual expression method of the routing graph and the conversion method between the routing graph and the Petri net are proposed, in which users can quickly edit the sequence of experimental steps by editing the routing graph. The feedbacks of the two channels of smell and temperature are incorporated, to realize the virtual realization of the fusion of multi-sensing channels of sight, hearing, smell and touch. To verify the proposed method, a prototype system is developed, and the thermite reaction experiment is taken as an example to introduce the generation process. The results show that the visual rapid editing method is effective and feasible for the editing of virtual experiments.

Key words: virtual experiment, Petri net, routing graph, multi-channel fusion, generation method

CLC Number:

TP391

Jingwei Deng, Hanwu He, Yueming Wu, Jianhao Su. Fast generation Method of Multi-sensing Channel Fusion Virtual Experiment[J]. Journal of System Simulation, 2022, 34(12): 2639-2648.

Figures/Tables 19

Fig. 1

Table 1

Table 2

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Table 3

When child step is empty, parent step determines the Petri net under the operable condition of the step

关系类型		父步骤式	父步骤式描述
顺序关系		$f 1 p$ 为空 $f 2 p$ 为1 $f 3 p$ 为2	开始实验时 $p 1$ 即可操作 $p 1$ 完成后 $p 2$ 才可操作 $p 2$ 完成后 $p 3$ 才可操作
并发关系		$f 1 p$ 为空 $f 2 p$ 为1 $f 3 p$ 为1 $f 4 p$ 为2 $f 5 p$ 为3	开始实验时 $p 1$ 即可操作 $p 1$ 完成后 $p 2$ 才可操作 $p 1$ 完成后 $p 3$ 才可操作 $p 2$ 完成后 $p 4$ 才可操作 $p 3$ 完成后 $p 5$ 才可操作
选择关系	只需完成任意分支	$f 1 p$ 为空 $f 2 p$ 为1 $f 3 p$ 为1 $f 4 p$ 为2\|3	开始实验时 $p 1$ 即可操作 $p 1$ 完成后 $p 2$ 才可操作 $p 1$ 完成后 $p 3$ 才可操作 $p 2$ 或 $p 3$ 完成后 $p 4$ 才可操作
选择关系	只能完成一条分支	$f 1 p$ 为空 $f 2 p$ 为1&!3 $f 3 p$ 为1&!2 $f 4 p$ 为2\|3	开始实验时 $p 1$ 即可操作 $p 1$ 完成且 $p 3$ 未完成时 $p 2$ 才可操作 $p 1$ 完成且 $p 2$ 未完成时 $p 3$ 才可操作 $p 2$ 或 $p 3$ 完成后 $p 4$ 才可操作
共生关系		$f 1 p$ 和 $f 2 p$ 为空 $f 3 p$ 为1 $f 4 p$ 为2 $f 5 p$ 为3&4	开始实验时 $p 1$ 、 $p 2$ 即可操作 $p 1$ 完成后 $p 3$ 才可操作 $p 2$ 完成后 $p 4$ 才可操作 $p 3$ 和 $p 4$ 完成后 $p 5$ 才可操作

Table 3

Table 4

Parallel relationship after adding child step

Petri网	步骤式	步骤式描述
	$f 1 p$ 为空	开始实验 $p 1$ 即可操作
	$f 2 p$ 为1	$p 1$ 完成后 $p 2$ 才可操作
	$f 3 p$ 为1	$p 1$ 完成后 $p 3$ 才可操作
	$f 2 c$ 为4	$p 2$ 完成后只能先做 $p 4$
	$f 3 c$ 为5	$p 3$ 完成后只能先做 $p 5$

Table 4

Fig. 6

Table 5

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Fig. 11

Table 6

Steps of each node in routing graph

步骤索引	步骤式	步骤索引	步骤式
1	—	9	$f 9 p$ 为8
2	$f 2 p$ 为1， $f 2 c$ 为3	10	$f 10 p$ 为9
3	$f 3 p$ 为2， $f 3 c$ 为4	11	$f 11 p$ 为10
4	$f 4 p$ 为3	12	$f 12 p$ 为11
5	$f 5 p$ 为1， $f 5 c$ 为6	13	$f 13 p$ 为11
6	$f 6 p$ 为5， $f 6 c$ 为7	14	$f 14 p$ 为9
7	$f 7 p$ 为6	15	$f 15 p$ 为14
8	$f 8 p$ 为4&7

Table 6

Fig. 12

Fig. 13

References 12

1	朱敏, 朱焱. 虚拟实验与物理课程教学[M]. 南京: 东南大学出版社, 2008.
	Zhu Min, Zhu Yan. Virtual Experiment and Physics Course Teaching[M]. Nanjing: Southeast University Press, 2008.
2	Sun C, Pan Z, Li Y. SRP Based Natural Interaction between Real and Virtual Worlds in Augmented Reality[C]//International Conference on Cyberworlds. Hangzhou: IEEE Computer Society, 2008: 117-124.
3	Tsuda N, Morikawa A, Nomura Y, et al. Pressure Presentation Strength for Calligraphy Brushwork Instruction[C]//IEEE International Symposium on Robotics&Intelligent Sensors. Ottawa: IEEE, 2017: 198-202.
4	Luo Tianren, Zhang Mingmin, Li Zheng, et al. Dream-Experiment: A MR User Interface with Natural Multi-Channel Interaction for Virtual Experiments[J]. IEEE Transactions on Visualization and Computer Graphics(S1077-2626), 2020, 26(12): 3524-3534.
5	Nicholas H, Pooi-Mun W, Matthew C, et al. Virtual Reality Training for Assembly of Hybrid Medical Devices[J]. Multimedia Tools & Applications(S1380-7501), 2018(77): 1-32.
6	卿烈华. 基于Petri网的民航机电维护优化研究[D]. 兰州: 兰州大学, 2016.
	Liehua Qing. Research on Civil Aviation Maintenance Process Based on Petri Net[D]. Lanzhou: Lanzhou University, 2016.
7	董健康, 杨柳, 耿宏. 基于循环迭代的飞机虚拟拆装过程建模方法[J]. 计算机工程与设计, 2020, 41(2): 494-499.
	Dong Jiankang, Yang Liu, Geng Hong. Aircraft Virtual Disassembly Process Modeling Method Based on Cyclic Iteration[J]. Computer Engineering and Design, 2020, 41(2): 494-499.
8	Al-Khalifa Hend S. CHEMOTION: A Gesture Based Chemistry Virtual Laboratory with Leap Motion[J]. John Wiley&Sons, Ltd(S1180-4009), 2017, 25(6): 1-16.
9	Coelho H, Melo M, Martins José, et al. Collaborative Immersive Authoring Tool for Real-Time Creation of Multisensory VR Experiences[J]. Multimedia Tools and Applications(S1380-7501), 2019, 78(14): 19473-19493.
10	Mujika A, Oyarzun D, Iparragirre I, et al. MACHS: An Authoring Tool to Create Serious Games for Machine-Tool Operator Training[C]//International Conference on Emerging Elearning Technologies& Applications. Stara Lesna: IEEE, 2011: 147-152.
11	黎永斌, 吴悦明, 何汉武, 等. 一种气味发生器: 201922340576.3[P]. 2020-08-28.
	Li Yongbin, Wu Yueming, He Hanwu, et al. An Odor Generator: 201922340576.3[P]. 2020-08-28.
12	吴悦明, 许明西, 何汉武, 等. 一种虚拟实验平台的热触觉再现装置: 201910667640.0[P]. 2019-07-23.
	Wu Yueming, Xu Mingxi, He Hanwu, et al. Thermal Tactile Reproduction Device of Virtual Experiment Platform: 201910667640.0[P]. 2019-07-23.

操作动作		参数指标
需要操作	移动物体	对象物体、目标位姿、工具、提示信息
	物体移离
	物体移近
	倾倒物体	对象物体、目标位姿、工具、倾倒角、提示信息
无需操作	显示物体	对象物体、显示位姿、提示信息
	隐藏物体	对象物体、提示信息
	出现特效	特效对象、显示位姿、持续时长、特效尺寸
	出现文字	文字内容、显示位姿、持续时长、文字尺寸
	物体动画	对象物体、目标位姿、动画速度
	播放视频	视频对象、播放时段、持续时长
	发出声音	声音对象、目标位姿、持续时长、声音大小
	散发气味	气味种类、持续时长、气味浓度、风扇功率
	释放热量	温度、持续时长、风扇功率

操作动作		实验现象
需要操作	移动物体	-
	物体移离	-
	物体移近	-
	倾倒物体	-
无需操作	显示物体	对象物体从不可见状态变为可见状态
	隐藏物体	对象物体从可见状态变为不可见状态
	出现特效	特效对象出现在目标位姿，并持续一定时长
	出现文字	文字对象出现在目标位姿，并持续一定时长
	物体动画	对象物体朝目标位姿以一定速度移动和旋转
	播放视频	在目标位姿播放一段视频
	发出声音	在目标位姿出现一段声音，支持立体声效果
	散发气味	嗅觉呈现装置散发指定气味
	释放热量	温感呈现装置释放指定温度的热风

操作动作	其他可选参数
移动物体	索引号、步骤式、对象物体、目标位姿、工具、提示信息
物体移离
物体移近
倾倒物体	索引号、步骤式、对象物体、目标位姿、工具、倾倒角、提示信息
显示物体	索引号、步骤式、对象物体、显示位姿、提示信息
隐藏物体	索引号、步骤式、对象物体、提示信息
出现特效	索引号、步骤式、特效对象、显示位姿、持续时长、特效尺寸
出现文字	索引号、步骤式、文字内容、显示位姿、持续时长、文字尺寸
物体动画	索引号、步骤式、对象物体、目标位姿、动画速度
播放视频	索引号、步骤式、视频对象、播放时段、持续时长
发出声音	索引号、步骤式、声音对象、目标位姿、持续时长、声音大小
散发气味	索引号、步骤式、气味种类、持续时长、气味浓度、风扇功率
释放热量	索引号、步骤式、温度、持续时长、风扇功率

[1]	Fan Zeng, He Mingjun, Xing Xiangyu. Research on Digital Simulation Method for Cognitive Load Evaluation of pilots [J]. Journal of System Simulation, 2025, 37(8): 1921-1932.
[2]	Pan Chunrong, Cui Yu, Xiong Wenqing, Zhou Hao, Luo Jiliang. Research on Adaptive Scheduling of Single-arm Cluster Tools for Throughput Ratio of Multiple Wafer Types with Concurrent Processing [J]. Journal of System Simulation, 2024, 36(12): 2906-2916.
[3]	Yin Yanchao, Feng Jiasheng, Yi Bin, Li Wang, Yin Qingwen. Visual Monitoring System of Digital Twin Workshop for Process Manufacturing [J]. Journal of System Simulation, 2024, 36(1): 120-130.
[4]	Zhe Yan, Wei Liu, Yuyue Du. Modeling and Analysis of Metro Emergency Decision Based on Logical Game Probability Petri Net [J]. Journal of System Simulation, 2023, 35(7): 1602-1618.
[5]	Chuanliang Xia, Maibo Guo, Zhuangzhuang Wang, Yan Sun. Shared Subnet Synthesis and Application of Object-oriented PRES Net [J]. Journal of System Simulation, 2023, 35(4): 833-842.
[6]	Mingbao Pang, Zhen Liu. Simulation on Cooperative Control of Connected and Automated Vehicles at Interchange Based on Petri Net [J]. Journal of System Simulation, 2023, 35(3): 484-493.
[7]	Guangqiu Huang, He Wang. Dynamic Risk Assessment of VOCs Cross Regional Flow Based on Petri Nets [J]. Journal of System Simulation, 2023, 35(1): 57-68.
[8]	Yali Wu, Shuting He, Yanxi Yang, Lianqiang Feng, Fuqiang Wang, Yulu Chen. Scheduling Optimization of Aluminum Extrusion Production Line Based on Timed Petri Net and BSO Algorithm [J]. Journal of System Simulation, 2023, 35(1): 178-189.
[9]	Hong Zheng, Zerun Liu, Jianhua Huang, Shihui Qian. Verification of Transaction Ordering Dependence Vulnerability of Smart Contract Based on CPN [J]. Journal of System Simulation, 2022, 34(7): 1629-1638.
[10]	Guangqiu Huang, Tiantian Wu. Cause Analysis of VOCs Hazards in Related Areas Based on Object Function Petri Net [J]. Journal of System Simulation, 2022, 34(3): 470-481.
[11]	Zou Xuyan, He Hanwu, Wu Yueming, Deng Jingwei. Virtual and Real Fusion Experiment Simulation Technology Based on Gesture Interaction [J]. Journal of System Simulation, 2021, 33(10): 2488-2498.
[12]	Zheng Hong, Ye Cheng, Deng Wenxuan, Pan Li. A Game Cheating Detection System Based on Petri Nets [J]. Journal of System Simulation, 2020, 32(3): 455-463.
[13]	Huang Guangqiu, He Tong, Lu Qiuqin. Cascade Transmission Evaluation of Ecosystem Damage Caused by Dust Migration [J]. Journal of System Simulation, 2020, 32(2): 278-288.
[14]	Zhang Zhili, Zhang Yunrong, Ma Shixin. Concurrent Conflict Control Method of "Multi-rights+Operation Queues" Based on CPN Tools [J]. Journal of System Simulation, 2019, 31(6): 1092-1100.
[15]	Zhou Xueguang, Lü Weidong, Yuan Zhimin. Colored Petri Net Based Modeling on Warship Command and Control Information Flow [J]. Journal of System Simulation, 2019, 31(5): 828-842.