Multi-UAVs 3D Path Planning Method Based on Random Strategy Search

doi:10.16182/j.issn1004731x.joss.21-0112

Abstract

Abstract:

In view of the difficulty of the traditional path planning method without energy consumption constraints to meet the emergency rescue requirements in the complex mountain operation environment, a three-dimensional path planning algorithm for multi-UAVs is proposed based on LSTM-DPPO(long short-term memory-distributed proximal policy optimization) framework. The LSTM long and short-term memory neural network is used to extract the important characteristic state information sequence of the multiple unmanned aerial vehicles in their respective flight process. After repeated iteration and updating, an optimal network parameter model is obtained. Combined with the energy consumption, the optimal 3D detection path is generated. Simulation experiments verify that the proposed method is more effective than the traditional path planning method and can plan the optimal detection path with the minimum energy consumption.

Key words: multi-UAVs, deep reinforcement learning algorithms, neural network, 3D path planning, energy consumption

CLC Number:

TP183

Sen Zhang, Mengyan Zhang, Jingping Shao, Jiexin Pu. Multi-UAVs 3D Path Planning Method Based on Random Strategy Search[J]. Journal of System Simulation, 2022, 34(6): 1286-1295.

Figures/Tables 16

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 1

Comparison of indicators with and without energy consumption constraints

指标		第300次	第1 000次
所有无人机总路径/km	无能耗约束	76.59	64.12
所有无人机总路径/km	有能耗约束	65.23	59.14
已检测目标区域量	无能耗约束	8	10
已检测目标区域量	有能耗约束	7	10
所有无人机总能耗× $10 - 3$ /kJ	无能耗约束	42.486	36.128
所有无人机总能耗× $10 - 3$ /kJ	有能耗约束	35.000	27.416

Table 1

Fig. 9

Fig. 10

Fig. 11

Table 2

Local data comparison between LSTM-DPO and A* algorithm

算法	路径/km	飞行时长/min	能耗 $× 10 - 3$ /kJ
LSTM-DPPO	5.613	11.25	5.283
A*	5.552	11.38	6.192

Table 2

Fig. 12

Fig. 13

Table 3

Comparison of indicators under a variety of algorithms

算法	所有无人机总路径/km	已检测目标区域量	所有飞行时间总长/min	所有无人机总能耗 $× 10 - 3$ /kJ
LSTM-DPPO	59.14	10	18.75	27.416
A*	58.12	10	19.22	33.286
蚁群	63.85	10	21.27	35.822
Dijkstra	64.67	10	21.87	32.468

Table 3

References 26

1	Mohammadreza Radmanesh, Manish Kumar, Sarim Moha Mmad. Grey Wolf Optimization Based Sense and Avoid Algorithm in a Bayesian Framework for Multi-UAVs Path Planning in an Uncertain Environment[J]. Aerospace Science and Technology(S1270-9638), 2018, 77: 168-179.
2	阚平, 姜兆亮, 刘玉浩, 等. 多植保无人机协同路径规划[J]. 航空学报, 2020, 41(4): 255-265.
	Kan Ping, Jiang Zhaoliang, Liu Yuhao, et al. Collaborative Path Planning for Multi-Plant Protection UAV[J]. Journal of Aviation, 2020, 41(4): 255-265.
3	戴健, 许菲, 陈琪锋. 多无人机协同搜索区域划分与路径规划[J]. 航空学报, 2020, 41(增1): 146-153.
	Dai Jian, Xu Fei, Chen Qifeng. Multi-UAVs Cooperative Search Area Division and Path Planning[J]. Journal of Aviation, 2020, 41(S1): 146-153.
4	Yoon Y, Kim M, Kim Y. Three-Dimensional Path Planning for Aerial Refueling Between One Tanker and Multi-UAVs[J]. International Journal of Aeronautical and Space(S2093-274X), 2018, 19(4): 1027-1040.
5	Yang X, Zhou W, Zhang Y. On Collaborative Path Planning for Multi-UAVs Based on Pythagorean Hodograph curve[C]// Guidance, Navigation & Control Conference. Nanjing: IEEE, 2016: 12-14.
6	黄东晋, 蒋晨凤, 韩凯丽. 基于深度强化学习的三维路径规划算法[J]. 计算机工程与应用, 2020, 56(15): 30-36.
	Huang Dongjin, Jiang Chenfeng, Han Kaili. 3D Path Planning Algorithm Based on Deep Reinforcement Learning[J]. Computer Engineering and Applications, 2020, 56(15): 30-36.
7	Sun P, Shan R. Predictive Control with Velocity Observer for Cushion Robot Based on PSO for Path Planning[J]. Journal of Systems Science & Complexity(S1009-6124), 2020, 33(4): 988-1011.
8	Johnathan Votion, Cao Yongcan. Diversity-Based Cooperative Multivehicle Path Planning for Risk Management in Costmap Environments[J]. IEEE Transactions on Industrial Electronics(S0278-0046), 2019, 66(8): 6117-6127.
9	Yu W, Low Kin Huat, Chen Lü. Cooperative Path Planning for Heterogeneous Unmanned Vehicles in a Search-and-Track Mission Aiming at an Underwater Target[J]. IEEE Transactions on Vehicular Technology(S0018-9545), 2020, 69(6): 6782-6787.
10	Chnjiang W, Shijie Z, Licai X. Dynamic Path Planning Based on Improved Ant Colony Algorithm in Traffic Congestion[J]. IEEE Access(S2169-3536), 2020, 8: 180773-180783.
11	Yi Jun, Bai Junren, He Haibo. A Multifactorial Evolutionary Algorithm for Multitasking Under Interval Uncertainties[J]. IEEE Transactions on Evolutionary Computation(S1089-778X), 2020, 24(5): 908-922.
12	陈海, 何开锋, 钱炜祺. 多无人机协同覆盖路径规划[J]. 航空学报, 2016, 37(3): 928-935.
	Chen Hai, He Kaifeng, Qian Weiqi. Multi-UAV Collaborative Coverage Path Planning [J]. Journal of Aviation, 2016, 37(3): 928-935.
13	Yao X, Wang X, Zhang L. Model Predictive and Adaptive Neural Sliding Mode Control for 3D Path Following of Autonomous Underwater Vehicle with Input Saturation[J]. Neural Computing and Applications (S0941-0643), 2020, 32(22): 16875-16889.
14	Delin G, Lan T, Xinggan Z. Joint Optimization of Handover Control and Power Allocation Based on Multi-Agent Deep Reinforcement Learning[J]. IEEE Transactions on Vehicular Technology(S0018-9545), 2020, 69(11): ‏13124-13138.
15	Jonggyu J, Hyun Jong Y. Deep Reinforcement Learning-Based Resource Allocation and Power Control in Small Cells with Limited Information Exchange[J]. IEEE Transactions on Vehicular Technology(S0018-9545), 2020, 69(11): 13768-13783.
16	Fengxiao T, Yibo Z, Neia Kato. Deep Reinforcement Learning for Dynamic Uplink/DowNlink Resource Allocation in High Mobility 5G HetNet [J]. IEEE Journal on Selected Areas in Communications(S0733-8716),2020, 38(12): ‏2773-2782.
17	Guan Y, Yangang R, Shengbo Eben L. Centralized Cooperation for Connected and Automated Vehicles at Intersections by Proximal Policy Optimization [J]. IEEE Transactions on Vehicular Technology(S0018-9545), 2020, 69(11): 12597-12608.
18	Wenhan Z, Chunbo L, Jin W. Deep-Reinfo Rcement-Learning-Based Offloading Scheduling for Vehicular Edge Computing[J]. IEEE Internet of Things Journal(S2327-4662), 2020, 7(6): 5449-5465.
19	Guangda C, Shunyi Y, Jun M. Distributed Non-Communicating Multi-Robot Collision Avoidance Via Map-Based Deep Reinforcement Learning[J]. International Journal of Electrical Power & Energy Systems(S0142-0615), 2020, 20(17): 4836.
20	Da L, Zhaosheng Z, Peng L. Battery Fault Diagnosis for Electric Vehicles Based on Voltage Abnormality by Combining the Long Short-Term Memory Neural Network and the Equivalent Circuit Model[J]. IEEE Transactions on Power Electronics(S0885-8993), 2020, 36(12):‏ 1303-131.
21	Tao L, Yongjin H, Ankang J. Adversarial Active Learning for Named Entity Recognition in Cybersecurity[J]. CMC-Computers Materials & Continua(S1546-2218), 2020, 66(1): 407-420.
22	Yang Yuan L, Do Tien Van, Nguyen Hai T. A Comparison of Forecasting Models for the Resource Usage of MapReduce Applications[J]. Neurocomputing(S0925-2312), 2020, 418: 36-55.
23	Ziming Y, Yan X. A Multi-Agent Deep Reinforcement Learning Method for Cooperative Load Frequency Control of a Multi-Area Power System[J]. IEEE Transactions on Power Systems(S0885-8950), 2020, 35(6): ‏4180-4192.
24	Fouad Y, Nassim R, Laid D, et al.Trajectory Optimisation for a Quadrotor Helicopter Considering Energy Consumption[C]// 2017 4th International Conference on Control, Decision and Information Technologies. Barcelona, Spain: IEEE, 2017: 5-7.
25	Goeke D, Schneider M. Routing a Mixed Fleet of Electric and Conventional Vehicles[J]. European Journal of Operational Research(S0377-2217), 2015, 245(1): 81-99.
26	孙长银, 穆朝絮. 多智能体深度强化学习的若干关键科学问题[J]. 自动化学报, 2020, 46(7): 1301-1312.
	Sun Changyin, Mu Zhaoxu. Some Key Scientific Problems of Deep Reinforcement Learning for Multi-Agent[J]. Acta Automatica Sinica, 2020, 46(7): 1301-1312.

[1]	Hongliang Zhang, Renman Ding, Gongjie Xu. Energy-Efficient Scheduling of Multi-objective Flexible Job Shop Considering Interval Processing Time [J]. Journal of System Simulation, 2022, 34(9): 1976-1987.
[2]	Yecai Guo, Qingwei Wang. Modulation Recognition Algorithm Based on Truncated Migration and Parallel ResNet [J]. Journal of System Simulation, 2022, 34(9): 2009-2018.
[3]	Weiguo Tong, Shichao Zeng, Lifeng Zhang, Zhe Hou, Jiayue Guo. Electrical Resistance Tomography and Flow Pattern Identification Method Based on Deep Residual Neural Network [J]. Journal of System Simulation, 2022, 34(9): 2028-2036.
[4]	Lifeng Zhang, Yu Miao. A High Resolution Reconstruction Method of Temperature Distribution in Acoustic Tomography [J]. Journal of System Simulation, 2022, 34(9): 2065-2073.
[5]	Xinghua Liu, Chen Geng, Shenghan Xie, Jiaqiang Tian, Hui Cao. Day Ahead Thermal-photovoltaic Economic Dispatch Considering Uncertainty of Photovoltaic Power Generation [J]. Journal of System Simulation, 2022, 34(8): 1874-1884.
[6]	Xiaoqing Zhang, Wanfang Xiao, Yingjie Guo, Bowen Liu, Xuesen Han, Jingwei Ma, Gao Gao, He Huang, Shihong Xia. Identification of Switching Operation Based on LSTM and MoE [J]. Journal of System Simulation, 2022, 34(8): 1899-1907.
[7]	Lifeng Zhang, Huiren Wang. Image Reconstruction of Electrical Capacitance Tomography Based on Convolutional Neural Network and Finite Element Simulation [J]. Journal of System Simulation, 2022, 34(4): 712-718.
[8]	Yanqin Long, Guifang Qiao, Guangming Song, Ying Zhang, Linlin Cheng. Motion Simulation and Performance Analysis of 2D Variable Stiffness Snake-like Robot [J]. Journal of System Simulation, 2022, 34(4): 759-767.
[9]	Sijin Zhou, Dicheng Chen, Geng Tu, Dazhi Jiang. Multi-modality Affective Computing Model Based on Personality and Memory Mechanism [J]. Journal of System Simulation, 2022, 34(4): 745-758.
[10]	Xiaoan Sun, Xiaoli Luan, Fei Liu. Electronic Solid Waste Prediction Based on Intelligent Optimization Grey Model [J]. Journal of System Simulation, 2022, 34(3): 536-542.
[11]	Zi'an Zhao, Hong Zhou, Yingjian Lei. Modeling and Optimization for Manufacturing Cell Scheduling Based on Improved Wolf Pack Algorithm and Simulation [J]. Journal of System Simulation, 2022, 34(2): 201-211.
[12]	Juan Wei, Lei You, Yangyong Guo, Zhihai Tang. Multi-floor Evacuation Model Based on Wavelet Neural Network [J]. Journal of System Simulation, 2022, 34(2): 269-277.
[13]	Bangqian Ao, Sha Yang, Jinqing Linghu, zhenhuan Ye. Design of Fatigue Driving Detection System Based on Cascaded Neural Network [J]. Journal of System Simulation, 2022, 34(2): 323-333.
[14]	Yan Bai, Lulu Wu, Yin'e He, Yuying Wang. Energy Consumption Prediction for Air-conditioning System Based on Dynamic Temperature Control [J]. Journal of System Simulation, 2022, 34(2): 366-375.
[15]	Wei Tengfei, Pan Tinglong. Short-term Power Load Forecasting Based on LSTM Neural Network Optimized by Improved PSO [J]. Journal of System Simulation, 2021, 33(8): 1866-1874.