航母甲板上舰载机的混合避障和仿真

doi:10.16182/j.issn1004731x.joss.21-1145

系统仿真学报 ›› 2023, Vol. 35 ›› Issue (3): 592-603.doi: 10.16182/j.issn1004731x.joss.21-1145

航母甲板上舰载机的混合避障和仿真

薛均晓¹(), 孔祥燕¹(), 董博威², 陶浩³, 管海洋², 石磊¹, 徐明亮²

^1.郑州大学网络空间安全学院, 河南郑州 450002
^2.郑州大学计算机与人工智能学院, 河南郑州 450001
^3.中国船舰研究设计中心, 湖北武汉 430064

收稿日期:2021-11-09 修回日期:2022-01-18 出版日期:2023-03-30 发布日期:2023-03-22
通讯作者: 孔祥燕 E-mail:xuejx@zzu.edu.cn;m15537229290@163.com
作者简介:薛均晓(1982-)，男，副教授，博士，研究方向为人工智能、虚拟现实。E-mail：xuejx@zzu.edu.cn
基金资助:
国家自然科学基金(62036010);河南省自然科学基金(202300410378);河南省高等学校青年骨干教师培养计划(22020GGJS014)

Obstacle Avoidance and Simulation of Carrier-Based Aircraft on the Deck of Aircraft Carrier

Junxiao Xue¹(), Xiangyan Kong¹(), Bowei Dong², Hao Tao³, Haiyang Guan², Lei Shi¹, Mingliang Xu²

^1.School of Cyber Science and Engineering, Zhengzhou University, Zhengzhou 450002, China
^2.School of Computer and Artificial Intelligence, Zhengzhou University, Zhengzhou 450001, China
^3.China Ship Research and Design Center, Wuhan 430064, China

Received:2021-11-09 Revised:2022-01-18 Online:2023-03-30 Published:2023-03-22
Contact: Xiangyan Kong E-mail:xuejx@zzu.edu.cn;m15537229290@163.com

摘要/Abstract

摘要：

针对航母甲板上舰载机混合避障随机性强、实时性差、规划速度慢等问题，结合最小二乘法与DDPG(deep deterministic policy gradient)算法提出一种PDDPG(predictive depth deterministic policy gradient)算法。该方法利用最小二乘法预测航母甲板上动态障碍物的短期轨迹。DDPG根据动态障碍物的短期轨迹为智能体提供在连续空间里学习和决策行为的能力。基于人工势场设置奖励函数，提高混合避障算法的收敛速度和准确率。使用Unity 3D构建了航母甲板高动态复杂场景，进行舰载机混合避障仿真实验。实验结果表明，PDDPG能较准确地实现航母甲板上舰载机的混合避障，与其他方法相比，在精度上提高了7%~30%。与DQN(deep Q network)相比，路径长度和转弯角度上分别减少了100个单位和400o~450o。

关键词: 路径规划, 混合避障, 最小二乘法, 人工势场, DDPG

Abstract:

A predictive depth deterministic policy gradient (PDDPG) algorithm is proposed by combining the least squares method with deep deterministic policy gradient(DDPG) for the problems of strong randomness, poor real-time performance, and slow planning speed by obstacle avoidance on aircraft carrier deck. The short-term trajectory of dynamic obstacles on the deck is predicted by the least square method. DDPG is used to provide agents with the ability to learn and make decisions in continuous space by the short-term trajectory of dynamic obstacles. The reward function is set based on the artificial potential field to improve the convergence speed and accuracy of the algorithm. A high dynamic complex scene of aircraft carrier deck is constructed using unity 3D to simulate experiments of obstacle avoidance method. The experimental results show that the method can accurately realize the hybrid obstacle avoidance of carrier aircraft on the aircraft carrier deck, and the accuracy is improved by 7% ~ 30% compared with other methods. Compared with deep Q network (DQN), the path length and turning angle are reduced by 100 units and 400o~450o respectively.

Key words: path planning, obstacle avoidance, least square method, artificial potential field, DDPG(deep deterministic policy gradient)

中图分类号:

TP 391.9

薛均晓, 孔祥燕, 董博威, 陶浩, 管海洋, 石磊, 徐明亮. 航母甲板上舰载机的混合避障和仿真[J]. 系统仿真学报, 2023, 35(3): 592-603.

Junxiao Xue, Xiangyan Kong, Bowei Dong, Hao Tao, Haiyang Guan, Lei Shi, Mingliang Xu. Obstacle Avoidance and Simulation of Carrier-Based Aircraft on the Deck of Aircraft Carrier[J]. Journal of System Simulation, 2023, 35(3): 592-603.

图/表 13

图1

图2

图3

图 4

图 5

图 6

表1

图7

图8

图9

图10

表2

表3

参考文献 21

1	薛均晓, 徐明亮, 李亚飞, 等. 面向航空母舰电子显灵板的多智能体建模技术研究进展[J]. 计算机辅助设计与图形学学报, 2021, 33(10): 1475-1485.
	Xue Junxiao, Xu Mingliang, Li Yafei, et al. Research Progress of Multi-agent Technology for Aircraft Carrier Electronic Display Panel[J]. Journal of Computer Aided Design and Graphics, 2021, 33(10): 1475-1485.
2	Wang J, Meng M Q H. Optimal Path Planning Using Generalized Voronoi Graph and Multiple Potential Functions[J]. IEEE Transactions on Industrial Electronics (S0278-0046), 2020, 67(12): 10621-10630.
3	Dang T, Mascarich F, Khattak S, et al. Graph-Based Path Planning for Autonomous Robotic Exploration in Subterranean Environments[C]// 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). China: IEEE, 2019: 3105-3112.
4	DIJKSTRA E W. A Note on Two Problems in Connexion with Graphs[J]. Numerische Mathematik (S0299-599X), 1959, 1(1): 2569-271.
5	Hart P E, Nilsson N J, Raphael B. A Formal Basis for The Heuristic Determination of Minimum Cost Paths[J]. Acm Sigart Bulletin (S0029-599X), 1972, 4(2): 28-29.
6	Hwang Y K, Ahuja N. A Potential Field Approach to Path Planning[J]. IEEE Transactions on Robotics and Automation (S1070-9932) 1992, 8(1): 23-32.
7	Bruce J, Veloso M M. Real-Time Randomized Path Planning for Robot Navigation[C]// Robot Soccer World Cup. Lausanne, Switzerland: Springer; 2002: 288-295.
8	Stentz A. Intelligent Unmanned Ground Vehicles[M]. Germany: Springer, 1997: 203-220.
9	Dorigo M, Gambardella L M. Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem[J]. IEEE Trans on Evolutionary Computation (S1089-778X), 1997, 1(1): 53-66.
10	Li G, Chou W. Path Planning for Mobile Robot Using Self-adaptive Learning Particle Swarm Optimization[J]. Science China Information Sciences (S1674-733X), 2018, 61(5): 052204.
11	Katoch S, Chauhan S S, Kumar V. A Review on Genetic Algorithm: Past, Present, and Future[J]. Multimedia Tools and Applications (S1380-7501), 2021, 80(5): 8091-8126.
12	Zhang H, Lin W, Chen A. Path Planning for the Mobile Robot: A Review[J]. Symmetry (S2073-8994), 2018, 10(10): 450.
13	Sutton R S, Barto A G. Reinforcement Learning: An Introduction[M]. England: MIT press, 2018.
14	Yan C, Xiang X. A Path Planning Algorithm for UAV Based on Improved Q-learning[C]// 2nd International Conference on Robotics and Automation Sciences (ICRAS).Wuhan, China: IEEE, 2018: 1-5.
15	Bouhamed O, Ghazzai H, Besbes H, et al. Q-learning Based Routing Scheduling for a Multi-task Autonomous Agent[C]// 2019 IEEE 62nd International Midwest Symposium on Circuits and Systems (MWSCAS). USA: IEEE, 2019: 634-637.
16	Sichkar V N. Reinforcement Learning Algorithms in Global Path Planning for Mobile Robot[C]// 2019 International Conference on Industrial Engineering, Applications and Manufacturing (ICIEAM). Sochi, Russia: IEEE, 2019: 1-5.
17	Watkins C J, Dayan P. Q-learning[J]. Machine Learning (S0885-6125), 1992 (3/4): 279-292.
18	Sewak Mohit. Deep Reinforcement Learning[M]. Germany: Springer, 2019: 95-108.
19	Sun Yushan, Ran Xiangrui, Zhang Guocheng, et al. AUV 3D Path Planning Based on the Improved Hierarchical Deep Q Network[J]. Journal of Marine Science and Engineering (S2077-1312), 2020, 8(2): 145.
20	薛均晓, 孔祥燕, 郭毅博, 等. 基于深度强化学习的舰载机动态避障方法[J]. 计算机辅助设计与图形学学报(S1003-9775), 2021, 33(7): 1102-1112.
	Xue Junxiao, Kong Xiangyan, Guo Yibo, et al. Dynamic Obstacle Avoidance Method for Carrier Aircraft Based on Deep Reinforcement Learning[J]. Journal of Computer Aided Design and Graphics (S1003-9775), 2021, 33(7): 1102-1112.
21	Zhang Xi, Liu Youbo, Duan Jiajun, et al. DDPG-based Multi-agent Framework for SVC Tuning in Urban Power Grid with Renewable Energy Resources[J]. IEEE Transactions on Power Systems (S1558-0679), 2021, 36(6): 5465-5475.

算法	场景1	场景2
DQN	0.90	0.90
A2C	0.67	0.64
DDPG	0.92	0.82
PDDPG	0.98	0.91

算法	场景1	场景2
DQN	438.22	465.45
A2C	352.60	324.16
DDPG	328.97	320.61
PDDPG	328.95	315.68

算法	场景1	场景2
DQN	586.73	522.62
A2C	142.03	158.00
DDPG	139.96	134.28
PDDPG	139.80	132.58

[1]	张国辉, 王璇, 张雅楠, 高昂. 实际环境中多无人车协同路径规划模型研究[J]. 系统仿真学报, 2023, 35(2): 408-422.
[2]	张玉超, 蒋沅, 代冀阳. 三阶多机器人协同编队动态避障控制[J]. 系统仿真学报, 2022, 34(8): 1762-1774.
[3]	李腾, 丁佩佩, 刘金芳. 货到人拣选系统多阶段可穿行多AGV路径规划[J]. 系统仿真学报, 2022, 34(7): 1512-1523.
[4]	陆淼嘉, 黄承媛, 滕靖. 基于多智能体的网购生鲜无人车配送调度仿真[J]. 系统仿真学报, 2022, 34(6): 1185-1195.
[5]	张森, 张孟炎, 邵敬平, 普杰信. 基于随机策略搜索的多机三维路径规划方法[J]. 系统仿真学报, 2022, 34(6): 1286-1295.
[6]	蒙盾, 胡卓, 张华军. 基于改进A^*算法的多层邮轮疏散系统仿真[J]. 系统仿真学报, 2022, 34(6): 1375-1382.
[7]	梁江涛, 王慧琴. 基于改进蚁群算法的建筑火灾疏散路径规划研究[J]. 系统仿真学报, 2022, 34(5): 1044-1053.
[8]	邓向阳, 张立民, 方伟, 汤淼. 基于双向汇聚引导蚁群算法的机器人路径规划[J]. 系统仿真学报, 2022, 34(5): 1101-1108.
[9]	李兆强, 张时雨. 基于快速RRT算法的三维路径规划算法研究[J]. 系统仿真学报, 2022, 34(3): 503-511.
[10]	王启明, 宗高强, 胥津铭. 多段式自动泊车最优路径规划与仿真分析[J]. 系统仿真学报, 2022, 34(2): 385-395.
[11]	乔乔, 王艳, 纪志成. 基于冲突搜索算法的多机器人路径规划[J]. 系统仿真学报, 2022, 34(12): 2659-2669.
[12]	田广键, 代冀阳, 应进, 王宁. 基于自适应分段势场法的多无人机航迹规划[J]. 系统仿真学报, 2022, 34(11): 2368-2376.
[13]	张莉, 张惠珍, 刘冬, 陆雨欣. 考虑紧迫度的应急物资调度及粒子群算法求解[J]. 系统仿真学报, 2022, 34(09): 1988-1998.
[14]	陈志梅, 李敏, 邵雪卷, 赵志诚. 基于改进RRT算法的桥式起重机避障路径规划[J]. 系统仿真学报, 2021, 33(8): 1832-1838.
[15]	向兵, 杨刚, 董晓红. 多层次地形编辑工具的设计与实现[J]. 系统仿真学报, 2021, 33(4): 900-909.

航母甲板上舰载机的混合避障和仿真

Obstacle Avoidance and Simulation of Carrier-Based Aircraft on the Deck of Aircraft Carrier

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 21

相关文章 15

编辑推荐

Metrics

本文评价