关键词: 强化学习, 课程学习, Transformer, 机械臂, 接球控制

Abstract:

method integrating the PPO algorithm with Transformer network architecture is proposed, and curriculum learning strategy is introduced to solve the difficult training convergence and low efficiency of traditional RL methods in complex and dynamic high-degree-of-freedom tasks such as robotic arm ball catching. The Transformer is employed to effectively capture the complex high-dimensional dependency between the robotic arm's state space, ball trajectory, and environmental physical parameters. Curriculum learning progressively increases catching difficulty by designing training tasks from simple to complex objectives. The experimental results show this method increases the ball-catching success rate by over 60% compared to the traditional PPO and features excellent accuracy at tracking balls with real-world disturbance characteristics. This method not only enhances the performance and efficiency of dynamic catching for robotic arms in both simulated and real-world disturbance conditions, but also provides a novel solution for complex task control in real-world scenarios.

Key words: RL, curriculum learning, Transformer, robotic arm, ball-catching control