Research on Simulation Training Technology for Special Vehicle Command Based on Gesture Behavior Cognition Interaction

doi:10.16182/j.issn1004731x.joss.25-0600

Abstract

Abstract:

To overcome the practical shortcomings in the driving command training of special vehicles, a simulation training technology framework based on gesture behavior cognition and command interaction was proposed. A somatosensory interactive motion capture device and human skeleton model were used to conduct the real-time dynamic recognition and spatial coordinate transformation of the gesture actions of the command training personnel. The median filtering method was applied to eliminate the abrupt data and random noise. The spatial coordinates were processed consistently by using the human skeleton centralization and normalization method. Based on the command gesture action behavior model library and the spatial posture angle calculation of arm skeletons, the posture features of different command gesture actions were determined, and the gesture behavior cognition based on action features was realized. The intelligent vehicle motion command information and control parameters were formed through the association and mapping of gesture commands and sent to the motion control module through the wireless communication protocol. After parsing and processing, they were converted into voltage and current signals to drive the intelligent vehicle to move forward/backward and turn left/right, ultimately achieving real-time interactive control of its motion process by different command gestures. Test verification by multiple groups of command training personnel shows that the proposed technical means and method approaches have good versatility, consistency, and accuracy, which can provide an economical and efficient solution for the driving command training of special vehicles.

Key words: gesture behavior cognition, human-computer interaction, special vehicle, command simulation training, action feature, motion control

CLC Number:

TP391.9

Li Xiangyang, Zhang Zhili, Wang Rui, Wang Xiao, Chi Cheng. Research on Simulation Training Technology for Special Vehicle Command Based on Gesture Behavior Cognition Interaction[J]. Journal of System Simulation, 2025, 37(7): 1804-1822.

Figures/Tables 24

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Fig. 5

Fig. 6

Fig. 7

Table 2

Posture features of command gesture actions

序号	指挥命令	手势动作姿态特征
序号	指挥命令	$θ E L, t m$	$θ E R, t m$	$θ W L, t m$	$θ W R, t m$	$φ E L, t m$	$φ E R, t m$	$φ W L, t m$	$φ W R, t m$	$n H L$	$n H R$
1	车辆启动	85±5	/	90±5	/	80±10	/	85±5	/	+	/
2	车辆前进	-30±10	-30±10	90±5	90±5	-30±5	-30±5	110±10	110±10	-	-
3	车辆后退	-30±10	-30±10	90±5	90±5	-30±5	-30±5	110±10	110±10	+	+
4	车辆停止	85±10	85±10	90±5	90±5	85±5	85±5	90±5	90±5	+	+
5	车辆前进左转	-40±5	0±10	90±5	0±5	0±10	/	90±5	/	-	+
6	车辆前进右转	0±10	-40±5	0±5	90±5	/	0±10	/	90±5	+	-
7	车辆后退左转	-40±5	0±10	90±5	0±5	0±10	/	90±5	/	+	+
8	车辆后退右转	0±10	-40±5	0±5	90±5	/	0±10	/	90±5	+	+

Table 2

Fig. 8

Table 3

Table 4

Table 5

Fig. 9

Fig. 10

Fig. 11

Fig. 12

Table 6

Fig. 13

Table 7

Fig. 14

Fig. 15

Fig. 16

Table 8

References 20

[1]	李向阳, 汪潇, 张志利, 等. 基于虚实结合的特装车辆驾驶训练模拟系统[J]. 系统仿真学报, 2021, 33(12): 2919-2934.
	Li Xiangyang, Wang Xiao, Zhang Zhili, et al. Special Vehicle Driving Training Simulation System Based on Integration of Virtuality and Reality[J]. Journal of System Simulation, 2021, 33(12): 2919-2934.
[2]	梁丰, 张志利, 李向阳, 等. 力反馈特种车辆驾驶舱半实物仿真模型研究[J]. 系统仿真学报, 2021, 33(12): 2935-2943.
	Liang Feng, Zhang Zhili, Li Xiangyang, et al. Research on Semi-physical Simulation Model of Special Vehicle Cockpit with Force Feedback[J]. Journal of System Simulation, 2021, 33(12): 2935-2943.
[3]	李亦豪, 张志利, 李向阳, 等. 特种车辆模拟驾驶训练中六自由度平台控制研究[J]. 系统仿真学报, 2021, 33(11): 2681-2690.
	Li Yihao, Zhang Zhili, Li Xiangyang, et al. Research on Six Degrees of Freedom Platform Control in Special Vehicle Simulated Driving Training[J]. Journal of System Simulation, 2021, 33(11): 2681-2690.
[4]	唐传谦, 刘志强, 苏毅俊, 等. 基于Kinect的皮影动作实时匹配研究[J]. 系统仿真学报, 2022, 34(3): 633-639.
	Tang Chuanqian, Liu Zhiqiang, Su Yijun, et al. Research on Real-time Motion Matching of Shadow Play Based on Kinect[J]. Journal of System Simulation, 2022, 34(3): 633-639.
[5]	王沐晴, 张磊, 范秀敏, 等. VR外设驱动的虚拟人姿态优化仿真方法[J]. 系统仿真学报, 2022, 34(6): 1296-1303.
	Wang Muqing, Zhang Lei, Fan Xiumin, et al. Simulation Method of Virtual Human Pose Optimization Based on VR Peripherals[J]. Journal of System Simulation, 2022, 34(6): 1296-1303.
[6]	詹燕, 陈志慧, 朱宝昌, 等. 基于自适应颜色快速点特征直方图的托盘识别方法[J]. 上海交通大学学报, 2023, 57(3): 297-308.
	Zhan Yan, Chen Zhihui, Zhu Baochang, et al. A Pallet Recognition Method Based on Adaptive Color Fast Point Feature Histogram[J]. Journal of Shanghai Jiao Tong University, 2023, 57(3): 297-308.
[7]	肖志刚, 周猛祥, 袁洪波, 等. 光照强度对Kinect v2深度数据测量精度的影响[J]. 农业机械学报, 2021, 52(增1): 108-117.
	Xiao Zhigang, Zhou Mengxiang, Yuan Hongbo, et al. Influence Analysis of Light Intensity on Kinect v2 Depth Measurement Accuracy[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(S1): 108-117.
[8]	边存灵, 吕伟刚, 冯伟. 骨架人体行为识别研究回顾、现状及展望[J]. 计算机工程与应用, 2024, 60(20): 1-29.
	Bian Cunling, Weigang Lü, Feng Wei. Skeleton-based Human Action Recognition: History, Status and Prospects[J]. Computer Engineering and Applications, 2024, 60(20): 1-29.
[9]	甘建旺, 沙芸, 张国英. 基于图像中值灰度相似度函数加权曲率滤波算法[J]. 电子学报, 2021, 49(6): 1187-1194.
	Gan Jianwang, Sha Yun, Zhang Guoying. Weighted Curvature Filtering Algorithm Based on Image Median Gray Similarity Function[J]. Acta Electronica Sinica, 2021, 49(6): 1187-1194.
[10]	叶瑞乾, 何浩, 郑鹏, 等. 一种基于中值滤波和统计学的拉曼光谱spike剔除算法[J]. 光谱学与光谱分析, 2022, 42(10): 3174-3179.
	Ye Ruiqian, He Hao, Zheng Peng, et al. A Spike Removal Algorithm Based on Median Filter and Statistic for Raman Spectra[J]. Spectroscopy and Spectral Analysis, 2022, 42(10): 3174-3179.
[11]	戴江涛. 基于深度学习的人体动作识别方法研究[D]. 沈阳: 沈阳工业大学, 2022.
	Dai Jiangtao. Research on Human Action Recognition Method Based on Deep Learning[D]. Shenyang: Shenyang University of Technology, 2022.
[12]	全燕鸣, 覃镇波, 李维诗, 等. 基于归一化算法的一维标定物多相机标定[J]. 光学学报, 2019, 39(4): 247-254.
	Quan Yanming, Qin Zhenbo, Li Weishi, et al. Multi-camera Calibration of One-dimensional Calibration Objects Based on Normalization Algorithm[J]. Acta Optica Sinica, 2019, 39(4): 247-254.
[13]	宋玉琴, 曾贺东, 高师杰, 等. 基于改进高分辨表征的人体姿态估计算法[J]. 计算机工程与设计, 2022, 43(4): 1045-1051.
	Song Yuqin, Zeng Hedong, Gao Shijie, et al. Human Pose Estimation Algorithm Based on Improved High Resolution Representation[J]. Computer Engineering and Design, 2022, 43(4): 1045-1051.
[14]	邓平, 吴明辉. 基于机器学习的人体运动姿态识别方法[J]. 中国惯性技术学报, 2022, 30(1): 37-43.
	Deng Ping, Wu Minghui. Human Motion Attitude Recognition Method Based on Machine Learning[J]. Journal of Chinese Inertial Technology, 2022, 30(1): 37-43.
[15]	曹恩国, 徐祺, 沈峰岑, 等. 多运动复合型被动式下肢外骨骼设计及其智能交互评估[J]. 机械工程学报, 2023, 59(11): 43-53.
	Cao Enguo, Xu Qi, Shen Fengcen, et al. Multi-motion Compound Passive Lower Limb Exoskeleton Design and Its Intelligent Interaction Evaluation[J]. Journal of Mechanical Engineering, 2023, 59(11): 43-53.
[16]	袁小庆, 邹缓, 吴涛, 等. 基于运动意图识别的上肢助力外骨骼复合控制方法研究[J]. 机械工程学报, 2023, 59(15): 73-82.
	Yuan Xiaoqing, Zou Huan, Wu Tao, et al. Compound Control Method for Human-powered Augmentation Upper Exoskeleton Based on Motion Intent Recognition[J]. Journal of Mechanical Engineering, 2023, 59(15): 73-82.
[17]	姚栋伟, 吴锋, 杨志家, 等. 基于增量式数字PID的汽油机怠速控制研究[J]. 浙江大学学报(工学版), 2010, 44(6): 1122-1126.
	Yao Dongwei, Wu Feng, Yang Zhijia, et al. Design of Idle Speed Controller for an SI Engine Based on Incremental Digital PID[J]. Journal of Zhejiang University(Engineering Science), 2010, 44(6): 1122-1126.
[18]	赵远, 许云鹏, 汪子楷, 等. 原子重力仪激光系统数字PID闭环控制设计及实现[J]. 中国惯性技术学报, 2025, 33(5): 495-501.
	Zhao Yuan, Xu Yunpeng, Wang Zikai, et al. Design and Implementation of Digital PID Closed-loop Control for Laser System of Atomic Gravimeter[J]. Journal of Chinese Inertial Technology, 2025, 33(5): 495-501.
[19]	柴天佑, 周正, 郑锐, 等. 端边云协同的PID整定智能系统[J]. 自动化学报, 2023, 49(3): 514-527.
	Chai Tianyou, Zhou Zheng, Zheng Rui, et al. PID Tuning Intelligent System Based on End-edge-cloud Collaboration[J]. Acta Automatica Sinica, 2023, 49(3): 514-527.
[20]	周志勇, 莫非, 赵凯, 等. 基于PPO的自适应PID控制算法研究[J]. 系统仿真学报, 2024, 36(6): 1425-1432.
	Zhou Zhiyong, Mo Fei, Zhao Kai, et al. Adaptive PID Control Algorithm Based on PPO[J]. Journal of System Simulation, 2024, 36(6): 1425-1432.

序号	指挥命令	头部信息	对象类型	指令代码	控制参数	尾部信息
1	车辆启动	0xFF	0x00	0x00	CarStart	0xFF
2	车辆前进	0xFF	0x00	0x01	MoveSpeed	0xFF
3	车辆后退	0xFF	0x00	0x02	BackSpeed	0xFF
4	车辆停止	0xFF	0x00	0x03	CarStop	0xFF
5	车辆前进左转	0xFF	0x01	0x04	MoveLeftAngle	0xFF
6	车辆前进右转	0xFF	0x01	0x05	MoveRightAngle	0xFF
7	车辆后退左转	0xFF	0x01	0x06	BackLeftAngle	0xFF
8	车辆后退右转	0xFF	0x01	0x07	BackRightAngle	0xFF

人员序号	性别	身高/cm	体重/kg
1	男	190	73
2	男	177	67
3	男	182	80
4	女	160	46
5	男	173	75

序号	手势指令	最长识别时长/ms	最短识别时长/ms	平均识别时长/ms	成功次数	识别成功率/%
1	车辆启动	362	198	236	500	100
2	车辆前进	437	315	338	492	98.40
3	车辆后退	529	322	346	493	98.60
4	车辆停止	235	204	243	500	100
5	车辆前进左转	464	321	370	495	99.00
6	车辆前进右转	458	337	362	496	99.20

人员序号	测试次数	失败次数	最长时长/s	最短时长/s	最小偏差/mm	最大偏差/mm
1	30	3	113	57	1	6
2	30	5	168	69	2	8