Multi-UAVs 3D Path Planning Method Based on Random Strategy Search

doi:10.16182/j.issn1004731x.joss.21-0112

Abstract

Abstract:

In view of the difficulty of the traditional path planning method without energy consumption constraints to meet the emergency rescue requirements in the complex mountain operation environment, a three-dimensional path planning algorithm for multi-UAVs is proposed based on LSTM-DPPO(long short-term memory-distributed proximal policy optimization) framework. The LSTM long and short-term memory neural network is used to extract the important characteristic state information sequence of the multiple unmanned aerial vehicles in their respective flight process. After repeated iteration and updating, an optimal network parameter model is obtained. Combined with the energy consumption, the optimal 3D detection path is generated. Simulation experiments verify that the proposed method is more effective than the traditional path planning method and can plan the optimal detection path with the minimum energy consumption.

Key words: multi-UAVs, deep reinforcement learning algorithms, neural network, 3D path planning, energy consumption

CLC Number:

TP183

Sen Zhang, Mengyan Zhang, Jingping Shao, Jiexin Pu. Multi-UAVs 3D Path Planning Method Based on Random Strategy Search[J]. Journal of System Simulation, 2022, 34(6): 1286-1295.

Figures/Tables 16

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 1

Comparison of indicators with and without energy consumption constraints

指标		第300次	第1 000次
所有无人机总路径/km	无能耗约束	76.59	64.12
所有无人机总路径/km	有能耗约束	65.23	59.14
已检测目标区域量	无能耗约束	8	10
已检测目标区域量	有能耗约束	7	10
所有无人机总能耗× $10 - 3$ /kJ	无能耗约束	42.486	36.128
所有无人机总能耗× $10 - 3$ /kJ	有能耗约束	35.000	27.416

Table 1

Fig. 9

Fig. 10

Fig. 11

Table 2

Local data comparison between LSTM-DPO and A* algorithm

算法	路径/km	飞行时长/min	能耗 $× 10 - 3$ /kJ
LSTM-DPPO	5.613	11.25	5.283
A*	5.552	11.38	6.192

Table 2

Fig. 12

Fig. 13

Table 3

Comparison of indicators under a variety of algorithms

算法	所有无人机总路径/km	已检测目标区域量	所有飞行时间总长/min	所有无人机总能耗 $× 10 - 3$ /kJ
LSTM-DPPO	59.14	10	18.75	27.416
A*	58.12	10	19.22	33.286
蚁群	63.85	10	21.27	35.822
Dijkstra	64.67	10	21.87	32.468

Table 3

References 26

1	Mohammadreza Radmanesh, Manish Kumar, Sarim Moha Mmad. Grey Wolf Optimization Based Sense and Avoid Algorithm in a Bayesian Framework for Multi-UAVs Path Planning in an Uncertain Environment[J]. Aerospace Science and Technology(S1270-9638), 2018, 77: 168-179.
2	阚平, 姜兆亮, 刘玉浩, 等. 多植保无人机协同路径规划[J]. 航空学报, 2020, 41(4): 255-265.
	Kan Ping, Jiang Zhaoliang, Liu Yuhao, et al. Collaborative Path Planning for Multi-Plant Protection UAV[J]. Journal of Aviation, 2020, 41(4): 255-265.
3	戴健, 许菲, 陈琪锋. 多无人机协同搜索区域划分与路径规划[J]. 航空学报, 2020, 41(增1): 146-153.
	Dai Jian, Xu Fei, Chen Qifeng. Multi-UAVs Cooperative Search Area Division and Path Planning[J]. Journal of Aviation, 2020, 41(S1): 146-153.
4	Yoon Y, Kim M, Kim Y. Three-Dimensional Path Planning for Aerial Refueling Between One Tanker and Multi-UAVs[J]. International Journal of Aeronautical and Space(S2093-274X), 2018, 19(4): 1027-1040.
5	Yang X, Zhou W, Zhang Y. On Collaborative Path Planning for Multi-UAVs Based on Pythagorean Hodograph curve[C]// Guidance, Navigation & Control Conference. Nanjing: IEEE, 2016: 12-14.
6	黄东晋, 蒋晨凤, 韩凯丽. 基于深度强化学习的三维路径规划算法[J]. 计算机工程与应用, 2020, 56(15): 30-36.
	Huang Dongjin, Jiang Chenfeng, Han Kaili. 3D Path Planning Algorithm Based on Deep Reinforcement Learning[J]. Computer Engineering and Applications, 2020, 56(15): 30-36.
7	Sun P, Shan R. Predictive Control with Velocity Observer for Cushion Robot Based on PSO for Path Planning[J]. Journal of Systems Science & Complexity(S1009-6124), 2020, 33(4): 988-1011.
8	Johnathan Votion, Cao Yongcan. Diversity-Based Cooperative Multivehicle Path Planning for Risk Management in Costmap Environments[J]. IEEE Transactions on Industrial Electronics(S0278-0046), 2019, 66(8): 6117-6127.
9	Yu W, Low Kin Huat, Chen Lü. Cooperative Path Planning for Heterogeneous Unmanned Vehicles in a Search-and-Track Mission Aiming at an Underwater Target[J]. IEEE Transactions on Vehicular Technology(S0018-9545), 2020, 69(6): 6782-6787.
10	Chnjiang W, Shijie Z, Licai X. Dynamic Path Planning Based on Improved Ant Colony Algorithm in Traffic Congestion[J]. IEEE Access(S2169-3536), 2020, 8: 180773-180783.
11	Yi Jun, Bai Junren, He Haibo. A Multifactorial Evolutionary Algorithm for Multitasking Under Interval Uncertainties[J]. IEEE Transactions on Evolutionary Computation(S1089-778X), 2020, 24(5): 908-922.
12	陈海, 何开锋, 钱炜祺. 多无人机协同覆盖路径规划[J]. 航空学报, 2016, 37(3): 928-935.
	Chen Hai, He Kaifeng, Qian Weiqi. Multi-UAV Collaborative Coverage Path Planning [J]. Journal of Aviation, 2016, 37(3): 928-935.
13	Yao X, Wang X, Zhang L. Model Predictive and Adaptive Neural Sliding Mode Control for 3D Path Following of Autonomous Underwater Vehicle with Input Saturation[J]. Neural Computing and Applications (S0941-0643), 2020, 32(22): 16875-16889.
14	Delin G, Lan T, Xinggan Z. Joint Optimization of Handover Control and Power Allocation Based on Multi-Agent Deep Reinforcement Learning[J]. IEEE Transactions on Vehicular Technology(S0018-9545), 2020, 69(11): ‏13124-13138.
15	Jonggyu J, Hyun Jong Y. Deep Reinforcement Learning-Based Resource Allocation and Power Control in Small Cells with Limited Information Exchange[J]. IEEE Transactions on Vehicular Technology(S0018-9545), 2020, 69(11): 13768-13783.
16	Fengxiao T, Yibo Z, Neia Kato. Deep Reinforcement Learning for Dynamic Uplink/DowNlink Resource Allocation in High Mobility 5G HetNet [J]. IEEE Journal on Selected Areas in Communications(S0733-8716),2020, 38(12): ‏2773-2782.
17	Guan Y, Yangang R, Shengbo Eben L. Centralized Cooperation for Connected and Automated Vehicles at Intersections by Proximal Policy Optimization [J]. IEEE Transactions on Vehicular Technology(S0018-9545), 2020, 69(11): 12597-12608.
18	Wenhan Z, Chunbo L, Jin W. Deep-Reinfo Rcement-Learning-Based Offloading Scheduling for Vehicular Edge Computing[J]. IEEE Internet of Things Journal(S2327-4662), 2020, 7(6): 5449-5465.
19	Guangda C, Shunyi Y, Jun M. Distributed Non-Communicating Multi-Robot Collision Avoidance Via Map-Based Deep Reinforcement Learning[J]. International Journal of Electrical Power & Energy Systems(S0142-0615), 2020, 20(17): 4836.
20	Da L, Zhaosheng Z, Peng L. Battery Fault Diagnosis for Electric Vehicles Based on Voltage Abnormality by Combining the Long Short-Term Memory Neural Network and the Equivalent Circuit Model[J]. IEEE Transactions on Power Electronics(S0885-8993), 2020, 36(12):‏ 1303-131.
21	Tao L, Yongjin H, Ankang J. Adversarial Active Learning for Named Entity Recognition in Cybersecurity[J]. CMC-Computers Materials & Continua(S1546-2218), 2020, 66(1): 407-420.
22	Yang Yuan L, Do Tien Van, Nguyen Hai T. A Comparison of Forecasting Models for the Resource Usage of MapReduce Applications[J]. Neurocomputing(S0925-2312), 2020, 418: 36-55.
23	Ziming Y, Yan X. A Multi-Agent Deep Reinforcement Learning Method for Cooperative Load Frequency Control of a Multi-Area Power System[J]. IEEE Transactions on Power Systems(S0885-8950), 2020, 35(6): ‏4180-4192.
24	Fouad Y, Nassim R, Laid D, et al.Trajectory Optimisation for a Quadrotor Helicopter Considering Energy Consumption[C]// 2017 4th International Conference on Control, Decision and Information Technologies. Barcelona, Spain: IEEE, 2017: 5-7.
25	Goeke D, Schneider M. Routing a Mixed Fleet of Electric and Conventional Vehicles[J]. European Journal of Operational Research(S0377-2217), 2015, 245(1): 81-99.
26	孙长银, 穆朝絮. 多智能体深度强化学习的若干关键科学问题[J]. 自动化学报, 2020, 46(7): 1301-1312.
	Sun Changyin, Mu Zhaoxu. Some Key Scientific Problems of Deep Reinforcement Learning for Multi-Agent[J]. Acta Automatica Sinica, 2020, 46(7): 1301-1312.

[1]	Wei Chengbiao, Zhao Taoyan, Cao Jiangtao, Li Ping. Design and Prediction of Deep Fuzzy Neural Network [J]. Journal of System Simulation, 2025, 37(9): 2200-2210.
[2]	Lü Jinhu, Jiang Hongyi, Liu Deyuan, Tan Shaolin. Modeling and Simulation of Complex Systems Based on Graph Neural Networks [J]. Journal of System Simulation, 2025, 37(7): 1624-1638.
[3]	Wang Ziyi, Zhang Kai, Qian Dianwei, Liu Yuzhen. A DRL⁃based Approach for Distributed Equipment Nodes Selection [J]. Journal of System Simulation, 2025, 37(6): 1565-1573.
[4]	Li Min, Zhang Sen, Zeng Xiangguang, Wang Gang, Zhang Tongwei, Xie Dijie, Ren Wenzhe, Zhang Tao. Trajectory Planning of Quadruped Robot Over Obstacle with Single Leg Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(4): 895-909.
[5]	Zhang Peng, Feng Ke, Gong Jiancheng, Yang Xiaoqiang, Shen Jinxing. Combat Effectiveness Evaluation of Air Defense Missile Weapon System Based on RBF Neural Network [J]. Journal of System Simulation, 2025, 37(2): 529-540.
[6]	Shen Jiawei, Cai Daye, Yang Guoqing, Lü Pan, Li Hong. Dynamic Loading Simulation Method for Large-scale Spiking Neural Network [J]. Journal of System Simulation, 2025, 37(2): 541-550.
[7]	Li Xiaobin, Hu Bing, Yin Chao, Li Bo, Ma Jun. Spatiotemporal Graph Convolution-based Demand Forecasting and Simulation Analysis for Automotive Parts Supply Chain [J]. Journal of System Simulation, 2025, 37(12): 3060-3074.
[8]	Zhang Wenkang, Sun Xiaofeng, Zhong Yiping, Yin Yong. Numerical Simulations of Ship Liquid Tank Sloshing Based on Graph Neural Networks [J]. Journal of System Simulation, 2025, 37(12): 3087-3099.
[9]	Xu Jinglin, Chen Qianru, Peng Yang, Yu Fangqiang. Self-calibrating Passenger Flow Simulation and Spatial Optimization for Public Building Based on GRU-SA [J]. Journal of System Simulation, 2025, 37(11): 2826-2838.
[10]	Tian Yuanxing, Han Zeyin, Wang Ning, Su Baoding, Xiang Weilin. Twin Modeling of Gearbox Fault Early Warning System Based on Spatio-Temporal Characteristics [J]. Journal of System Simulation, 2025, 37(11): 2867-2876.
[11]	Zhang Zhili, Liu Jin, Zhou Zhaofa, Liang Zhe, Zhang Yunhao. Research on Temperature Compensation Technology of Fiber Optic Gyroscope based on ISCSO-BP Neural Network Model [J]. Journal of System Simulation, 2025, 37(11): 2904-2917.
[12]	Li Boning, Chen Ming, Qi Shuchang, Meng Haoran, Wang Lei. Attitude Control of Quadrotor UAV Based on Disturbance Observer and Command Filtering [J]. Journal of System Simulation, 2025, 37(11): 2793-2803.
[13]	Huang Deqi, Tu Yating, Zhang Zhenhua, Guo Xin. Modeling and Simulation of Traffic Signal Control Based on MLP with Improved GCN-TD3 [J]. Journal of System Simulation, 2025, 37(10): 2568-2577.
[14]	Gu Hao, Wang Jiayu, Xiong Weili. Soft Sensor Modeling Based on Improved Transformer in Dual-stream Framework [J]. Journal of System Simulation, 2025, 37(10): 2594-2604.
[15]	Jiang Wenbin, Cao Yuqing, Xie Li, Yang Huizhong. Robust Identification of Dual-rate Sampled Nonlinear Systems Based on SALR Network [J]. Journal of System Simulation, 2025, 37(10): 2652-2661.