Journal of System Simulation ›› 2021, Vol. 33 ›› Issue (10): 2440-2448.doi: 10.16182/j.issn1004731x.joss.21-0229

Previous Articles     Next Articles

DQN-based Path Planning Method and Simulation for Submarine and Warship in Naval Battlefield

Huang Xiaodong1, Yuan Haitao2, Bi Jing3,*, Liu Tao4   

  1. 1. Naval Aeronautical University, Shandong 264001, China;
    2. School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China;
    3. Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China;
    4. School of Software Engineering, Beijing Jiaotong University, Beijing 100044, China
  • Received:2021-03-19 Revised:2021-04-15 Online:2021-10-18 Published:2021-10-18

Abstract: To realize multi-agent intelligent planning and target tracking in complex naval battlefield environment, the work focuses on agents (submarine or warship), and proposes a simulation method based on reinforcement learning algorithm called Deep Q Network (DQN). Two neural networks with the same structure and different parameters are designed to update real and predicted Q values for the convergence of value functions. An ε-greedy algorithm is proposed to design an action selection mechanism, and a reward function is designed for the naval battlefield environment to increase the update velocity and generalization ability of Learning with Experience Replay (LER). Simulation results show that compared with existing path routing algorithms and multi-agent path routing algorithms, each agent can effectively avoid obstacles in unfamiliar environments and achieve more intelligent path planning and target tracking through a certain number of steps of learning.

Key words: Deep Q network, reinforcement learning, multiple agents, path planning, target tracking

CLC Number: