关键词: 机器人, 路径规划, 深度强化学习, 人工势场, 优先经验回放

Abstract:

An improved A-DDQN algorithm is proposed to address the challenges of reward sparsity and the inability to distinguish sample importance in traditional DQN algorithms during robot path planning. Building on the original DQN, an enhancement is made by incorporating the Double-DQN approach, which updates the predictive Q-value network based on actions selected by the Q network, rather than directly using the predicted Q-values for action selection, thereby mitigating overestimation issues. Secondly, the concept of artificial potential field (APF) is introduced to design specific rewards for each step of the robot's movement, guiding the robot and addressing the problem of sparse rewards. Lastly, the prioritized experience replay (PER) mechanism is integrated, adjusting the sampling probability of experiences through priority ranking to accelerate the learning process and enhance performance. Comparative analysis of path planning before and after the algorithm improvements in a two-dimensional grid map shows that in small-scale maps, the improved A-DDQN algorithm reduces path length, iteration times, and the number of turning points by 11.5%, 23.1%, and 61.5% respectively; in large-scale maps with sparse obstacles, these reductions are 19.4%, 50.0%, and 52.9% respectively; and in large-scale maps with dense obstacles, reductions are 29.7%, 48.1%, and 64.3%. These simulation results prove that the improved algorithm achieves faster convergence and superior path planning performance.

Key words: robots, path planning, deep reinforcement learning, artificial potential field(APF), priority experience replay(PER)