Obstacle Avoidance and Simulation of Carrier-Based Aircraft on the Deck of Aircraft Carrier

doi:10.16182/j.issn1004731x.joss.21-1145

Abstract

Abstract:

A predictive depth deterministic policy gradient (PDDPG) algorithm is proposed by combining the least squares method with deep deterministic policy gradient(DDPG) for the problems of strong randomness, poor real-time performance, and slow planning speed by obstacle avoidance on aircraft carrier deck. The short-term trajectory of dynamic obstacles on the deck is predicted by the least square method. DDPG is used to provide agents with the ability to learn and make decisions in continuous space by the short-term trajectory of dynamic obstacles. The reward function is set based on the artificial potential field to improve the convergence speed and accuracy of the algorithm. A high dynamic complex scene of aircraft carrier deck is constructed using unity 3D to simulate experiments of obstacle avoidance method. The experimental results show that the method can accurately realize the hybrid obstacle avoidance of carrier aircraft on the aircraft carrier deck, and the accuracy is improved by 7% ~ 30% compared with other methods. Compared with deep Q network (DQN), the path length and turning angle are reduced by 100 units and 400o~450o respectively.

Key words: path planning, obstacle avoidance, least square method, artificial potential field, DDPG(deep deterministic policy gradient)

CLC Number:

TP 391.9

Junxiao Xue, Xiangyan Kong, Bowei Dong, Hao Tao, Haiyang Guan, Lei Shi, Mingliang Xu. Obstacle Avoidance and Simulation of Carrier-Based Aircraft on the Deck of Aircraft Carrier[J]. Journal of System Simulation, 2023, 35(3): 592-603.

Figures/Tables 13

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Table 1

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Table 2

Table 3

References 21

1	薛均晓, 徐明亮, 李亚飞, 等. 面向航空母舰电子显灵板的多智能体建模技术研究进展[J]. 计算机辅助设计与图形学学报, 2021, 33(10): 1475-1485.
	Xue Junxiao, Xu Mingliang, Li Yafei, et al. Research Progress of Multi-agent Technology for Aircraft Carrier Electronic Display Panel[J]. Journal of Computer Aided Design and Graphics, 2021, 33(10): 1475-1485.
2	Wang J, Meng M Q H. Optimal Path Planning Using Generalized Voronoi Graph and Multiple Potential Functions[J]. IEEE Transactions on Industrial Electronics (S0278-0046), 2020, 67(12): 10621-10630.
3	Dang T, Mascarich F, Khattak S, et al. Graph-Based Path Planning for Autonomous Robotic Exploration in Subterranean Environments[C]// 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). China: IEEE, 2019: 3105-3112.
4	DIJKSTRA E W. A Note on Two Problems in Connexion with Graphs[J]. Numerische Mathematik (S0299-599X), 1959, 1(1): 2569-271.
5	Hart P E, Nilsson N J, Raphael B. A Formal Basis for The Heuristic Determination of Minimum Cost Paths[J]. Acm Sigart Bulletin (S0029-599X), 1972, 4(2): 28-29.
6	Hwang Y K, Ahuja N. A Potential Field Approach to Path Planning[J]. IEEE Transactions on Robotics and Automation (S1070-9932) 1992, 8(1): 23-32.
7	Bruce J, Veloso M M. Real-Time Randomized Path Planning for Robot Navigation[C]// Robot Soccer World Cup. Lausanne, Switzerland: Springer; 2002: 288-295.
8	Stentz A. Intelligent Unmanned Ground Vehicles[M]. Germany: Springer, 1997: 203-220.
9	Dorigo M, Gambardella L M. Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem[J]. IEEE Trans on Evolutionary Computation (S1089-778X), 1997, 1(1): 53-66.
10	Li G, Chou W. Path Planning for Mobile Robot Using Self-adaptive Learning Particle Swarm Optimization[J]. Science China Information Sciences (S1674-733X), 2018, 61(5): 052204.
11	Katoch S, Chauhan S S, Kumar V. A Review on Genetic Algorithm: Past, Present, and Future[J]. Multimedia Tools and Applications (S1380-7501), 2021, 80(5): 8091-8126.
12	Zhang H, Lin W, Chen A. Path Planning for the Mobile Robot: A Review[J]. Symmetry (S2073-8994), 2018, 10(10): 450.
13	Sutton R S, Barto A G. Reinforcement Learning: An Introduction[M]. England: MIT press, 2018.
14	Yan C, Xiang X. A Path Planning Algorithm for UAV Based on Improved Q-learning[C]// 2nd International Conference on Robotics and Automation Sciences (ICRAS).Wuhan, China: IEEE, 2018: 1-5.
15	Bouhamed O, Ghazzai H, Besbes H, et al. Q-learning Based Routing Scheduling for a Multi-task Autonomous Agent[C]// 2019 IEEE 62nd International Midwest Symposium on Circuits and Systems (MWSCAS). USA: IEEE, 2019: 634-637.
16	Sichkar V N. Reinforcement Learning Algorithms in Global Path Planning for Mobile Robot[C]// 2019 International Conference on Industrial Engineering, Applications and Manufacturing (ICIEAM). Sochi, Russia: IEEE, 2019: 1-5.
17	Watkins C J, Dayan P. Q-learning[J]. Machine Learning (S0885-6125), 1992 (3/4): 279-292.
18	Sewak Mohit. Deep Reinforcement Learning[M]. Germany: Springer, 2019: 95-108.
19	Sun Yushan, Ran Xiangrui, Zhang Guocheng, et al. AUV 3D Path Planning Based on the Improved Hierarchical Deep Q Network[J]. Journal of Marine Science and Engineering (S2077-1312), 2020, 8(2): 145.
20	薛均晓, 孔祥燕, 郭毅博, 等. 基于深度强化学习的舰载机动态避障方法[J]. 计算机辅助设计与图形学学报(S1003-9775), 2021, 33(7): 1102-1112.
	Xue Junxiao, Kong Xiangyan, Guo Yibo, et al. Dynamic Obstacle Avoidance Method for Carrier Aircraft Based on Deep Reinforcement Learning[J]. Journal of Computer Aided Design and Graphics (S1003-9775), 2021, 33(7): 1102-1112.
21	Zhang Xi, Liu Youbo, Duan Jiajun, et al. DDPG-based Multi-agent Framework for SVC Tuning in Urban Power Grid with Renewable Energy Resources[J]. IEEE Transactions on Power Systems (S1558-0679), 2021, 36(6): 5465-5475.

算法	场景1	场景2
DQN	0.90	0.90
A2C	0.67	0.64
DDPG	0.92	0.82
PDDPG	0.98	0.91

算法	场景1	场景2
DQN	438.22	465.45
A2C	352.60	324.16
DDPG	328.97	320.61
PDDPG	328.95	315.68

算法	场景1	场景2
DQN	586.73	522.62
A2C	142.03	158.00
DDPG	139.96	134.28
PDDPG	139.80	132.58

[1]	Guohui Zhang, Xuan Wang, Yanan Zhang, Ang Gao. Research on Cooperative Path Planning Model of Multiple Unmanned Vehicles in Real Environment [J]. Journal of System Simulation, 2023, 35(2): 408-422.
[2]	Yuchao Zhang, Yuan Jiang, Jiyang Dai. Dynamic Obstacle Avoidance Control of Three-order Multi-robot Cooperative Formation [J]. Journal of System Simulation, 2022, 34(8): 1762-1774.
[3]	Teng Li, Peipei Ding, Jinfang Liu. Multi-Stage Multi-AGV Path Planning with Walk under Shelves for Robotic Mobile Fulfillment Systems [J]. Journal of System Simulation, 2022, 34(7): 1512-1523.
[4]	Sen Zhang, Mengyan Zhang, Jingping Shao, Jiexin Pu. Multi-UAVs 3D Path Planning Method Based on Random Strategy Search [J]. Journal of System Simulation, 2022, 34(6): 1286-1295.
[5]	Dun Meng, Zhuo Hu, Huajun Zhang. Simulation of Multi-layer Ship Evacuation System Based on Improved A* Algorithm [J]. Journal of System Simulation, 2022, 34(6): 1375-1382.
[6]	Xiangyang Deng, Limin Zhang, Wei Fang, Miao Tang. Robot Path Planning Based on Bidirectional Aggregation Ant Colony Optimization [J]. Journal of System Simulation, 2022, 34(5): 1101-1108.
[7]	Zhaoqiang Li, Shiyu Zhang. Research on 3D Path Planning Algorithm Based on Fast RRT Algorithm [J]. Journal of System Simulation, 2022, 34(3): 503-511.
[8]	Qiming Wang, Gaoqiang Zong, Jinming Xu. Optimal Path Planning for Multi-stage Automatic Parking and Simulation Analysis [J]. Journal of System Simulation, 2022, 34(2): 385-395.
[9]	Qiao Qiao, Yan Wang, Zhicheng Ji. Multi-robot Path Planning Based on CBS Algorithm [J]. Journal of System Simulation, 2022, 34(12): 2659-2669.
[10]	Guangjian Tian, Jiyang Dai, Jin Ying, Ning Wang. Multi-UAV Trajectory Planning Based on Adaptive Segmented Potential Field Method [J]. Journal of System Simulation, 2022, 34(11): 2368-2376.
[11]	Li Zhang, Huizhen Zhang, Dong Liu, Yuxin Lu. Particle Swarm Algorithm for Solving Emergency Material Dispatch Considering Urgency [J]. Journal of System Simulation, 2022, 34(09): 1988-1998.
[12]	Chen Zhimei, Li Min, Shao Xuejuan, Zhao Zhicheng. Obstacle Avoidance Path Planning of Bridge Crane Based on Improved RRT Algorithm [J]. Journal of System Simulation, 2021, 33(8): 1832-1838.
[13]	Xiang Bing, Yang Gang, Dong Xiaohong. Design and Implementation of Multi Level Terrain Editing Tool [J]. Journal of System Simulation, 2021, 33(4): 900-909.
[14]	Li Qi, Wang Wei. Path Designing of Multi-omnidirectional Wheel Collaborative Sorting Platform [J]. Journal of System Simulation, 2021, 33(3): 698-709.
[15]	Huang Xiaodong, Yuan Haitao, Bi Jing, Liu Tao. DQN-based Path Planning Method and Simulation for Submarine and Warship in Naval Battlefield [J]. Journal of System Simulation, 2021, 33(10): 2440-2448.