Obstacle Avoidance Path Planning and Simulation of Mobile Picking Robot Based on DPPO

doi:10.16182/j.issn1004731x.joss.22-0487

Abstract

Junqiang Lin, Hongjun Wang, Xiangjun Zou, Po Zhang, Chengen Li, Yipeng Zhou, Shujie Yao. Obstacle Avoidance Path Planning and Simulation of Mobile Picking Robot Based on DPPO[J]. Journal of System Simulation, 2023, 35(8): 1692-1704.

Figures/Tables 16

Fig. 1

Table 1

List of state space sets

观测变量	符号	状态描述
位置变量	$p r o b o t$	机器人空间位置
	$p o b s$	障碍物空间位置
	$p t r e e$	果树空间位置
机器人变量	$R v$	线速度
	$R ω$	角速度
	$R t$	移动时间
距离变量	$d R O$	机器人与障碍物的距离
距离变量	$d R G$	机器人与目标点的距离

Table 1

Table 2

Action space collection list

控制方式

符号

动作描述

取值范围

前进控制

a v

$a v > 0$ 加速

$a v = 0$ 静止

[0, 1]

转向控制

a ω

$- 1 < a ω < 0$ 左转

$a ω = 0$ 直行

$0 < a ω < 1$ 右转

[

-

1, 1]

Table 2

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Table 3

Fig. 6

Fig. 7

Table 4

Fig. 8

Table 5

Fig. 9

Fig. 10

Fig. 11

References 25

1	胡广锐, 孔微雨, 齐闯, 等. 果园环境下移动采摘机器人导航路径优化[J]. 农业工程学报, 2021, 37(9): 175-184.
	Hu Guangrui, Kong Weiyu, Qi Chuang, et al. Optimization of the Navigation Path for a Mobile Harvesting Robot in Orchard Environment[J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(9): 175-184.
2	闫皎洁, 张锲石, 胡希平. 基于强化学习的路径规划技术综述[J]. 计算机工程, 2021, 47(10): 16-25.
	Yan Jiaojie, Zhang Qieshi, Hu Xiping. Review of Path Planning Techniques Based on Reinforcement Learning[J]. Computer Engineering, 2021, 47(10): 16-25.
3	Lai Xin, Li Jiahe, Chambers J. Enhanced Center Constraint Weighted A^* Algorithm for Path Planning of Petrochemical Inspection Robot[J]. Journal of Intelligent & Robotic Systems, 2021, 102(4): 78.
4	Orozco-Rosas U, Montiel O, Sepúlveda Roberto. Mobile Robot Path Planning Using Membrane Evolutionary Artificial Potential Field[J]. Applied Soft Computing, 2019, 77: 236-251.
5	陈志梅, 李敏, 邵雪卷, 等. 基于改进RRT算法的桥式起重机避障路径规划[J]. 系统仿真学报, 2021, 33(8): 1832-1838.
	Chen Zhimei, Li Min, Shao Xuejuan, et al. Obstacle Avoidance Path Planning of Bridge Crane Based on Improved RRT Algorithm[J]. Journal of System Simulation, 2021, 33(8): 1832-1838.
6	Ajeil H F, Ibraheem I K, Sahib M A, et al. Multi-objective Path Planning of an Autonomous Mobile Robot Using Hybrid PSO-MFB Optimization Algorithm[J]. Applied Soft Computing, 2020, 89: 106076.
7	AL-Taharwa I, Sheta A, Al-Weshah M. A Mobile Robot Path Planning Using Genetic Algorithm in Static Environment[J]. Journal of Computer Science, 2008, 4(4): 341-344.
8	邓向阳, 张立民, 方伟, 等. 基于双向汇聚引导蚁群算法的机器人路径规划[J]. 系统仿真学报, 2022, 34(5): 1101-1108.
	Deng Xiangyang, Zhang Limin, Fang Wei, et al. Robot Path Planning Based on Bidirectional Aggregation Ant Colony Optimization[J]. Journal of System Simulation, 2022, 34(5): 1101-1108.
9	张文, 刘勇, 张超凡, 等. 基于方向A^*算法的温室机器人实时路径规划[J]. 农业机械学报, 2017, 48(7): 22-28.
	Zhang Wen, Liu Yong, Zhang Chaofan, et al. Real-time Path Planning of Greenhouse Robot Based on Directional A^* Algorithm[J]. Transactions of the Chinese Society for Agricultural Machinery, 2017, 48(7): 22-28.
10	李腾, 丁佩佩, 刘金芳. 货到人拣选系统多阶段可穿行多AGV路径规划[J]. 系统仿真学报, 2022, 34(7): 1512-1523.
	Li Teng, Ding Peipei, Liu Jinfang. Multi-stage Multi-AGV Path Planning With Walk Under Shelves for Robotic Mobile Fulfillment Systems[J]. Journal of System Simulation, 2022, 34(7): 1512-1523.
11	殷建军, 董文龙, 梁利华, 等. 复杂环境下农业机器人路径规划优化方法[J]. 农业机械学报, 2019, 50(5): 17-22.
	Yin Jianjun, Dong Wenlong, Liang Lihua, et al. Optimization Method of Agricultural Robot Path Planning in Complex Environment[J]. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(5): 17-22.
12	魏武, 韩进, 李艳杰, 等. 基于双树Quick-RRT^*算法的移动机器人路径规划[J]. 华南理工大学学报(自然科学版), 2021, 49(7): 51-58.
	Wei Wu, Han Jin, Li Yanjie, et al. Path Planning of Mobile Robots Based on Dual-tree Quick-RRT^* Algorithm[J]. Journal of South China University of Technology(Natural Science Edition), 2021, 49(7): 51-58.
13	孙辉辉, 胡春鹤, 张军国. 移动机器人运动规划中的深度强化学习方法[J]. 控制与决策, 2021, 36(6): 1281-1292.
	Sun Huihui, Hu Chunhe, Zhang Junguo. Deep Reinforcement Learning for Motion Planning of Mobile Robots[J]. Control and Decision, 2021, 36(6): 1281-1292.
14	张荣霞, 武长旭, 孙同超, 等. 深度强化学习及在路径规划中的研究进展[J]. 计算机工程与应用, 2021, 57(19): 44-56.
	Zhang Rongxia, Wu Changxu, Sun Tongchao, et al. Progress on Deep Reinforcement Learning in Path Planning[J]. Computer Engineering and Applications, 2021, 57(19): 44-56.
15	刘全, 翟建伟, 章宗长, 等. 深度强化学习综述[J]. 计算机学报, 2018, 41(1): 1-27.
	Liu Quan, Zhai Jianwei, Zhang Zongchang, et al. A Survey on Deep Reinforcement Learning[J]. Chinese Journal of Computers, 2018, 41(1): 1-27.
16	Lin Junqiang, Zhang Po, Li Chengen, et al. APF-DPPO: An Automatic Driving Policy Learning Method Based on the Artificial Potential Field Method to Optimize the Reward Function[J]. Machines, 2022, 10(7): 533.
17	徐晓苏, 袁杰. 基于改进强化学习的移动机器人路径规划方法[J]. 中国惯性技术学报, 2019, 27(3): 314-320.
	Xu Xiaosu, Yuan Jie. Path Planning for Mobile Robot Based on Improved Reinforcement Learning Algorithm[J]. Journal of Chinese Inertial Technology, 2019, 27(3): 314-320.
18	王珂, 卜祥津, 李瑞峰, 等. 景深约束下的深度强化学习机器人路径规划[J]. 华中科技大学学报(自然科学版), 2018, 46(12): 77-82.
	Wang Ke, Bu Xiangjin, Li Ruifeng, et al. Path Planning for Robots Based on Deep Reinforcement Learning by Depth Constraint[J]. Journal of Huazhong University of Science and Technology(Natural Science Edition), 2018, 46(12): 77-82.
19	成怡, 郝密密. 改进深度强化学习的室内移动机器人路径规划[J]. 计算机工程与应用, 2021, 57(21): 256-262.
	Cheng Yi, Hao Mimi. Path Planning for Indoor Mobile Robot With Improved Deep Reinforcement Learning[J]. Computer Engineering and Applications, 2021, 57(21): 256-262.
20	Zheng Jianfeng, Mao Shuren, Wu Zhenyu, et al. Improved Path Planning for Indoor Patrol Robot Based on Deep Reinforcement Learning[J]. Symmetry, 2022, 14(1): 132.
21	邹湘军, 罗锡文, 卢俊, 等. 虚拟环境下农业移动机器人行为及其仿真建模[J]. 系统仿真学报, 2006, 18(增2): 551-553, 562.
	Zou Xiangjun, Luo Xiwen, Lu Jun, et al. Modeling for Behavior and Simulation of Agriculture Mobile Robot in Virtual Environment[J]. Journal of System Simulation, 2006, 18(S2): 551-553, 562.
22	Schulman J, Wolski F, Dhariwal P, et al. Proximal Policy Optimization Algorithms[EB/OL]. (2017-08-28)[2022-05-06].
23	Heess N, TB D, Sriram S, et al. Emergence of Locomotion Behaviours in Rich Environments[EB/OL]. (2017-07-10)[2022-05-06].
24	熊俊涛, 李中行, 陈淑绵, 等. 基于深度强化学习的虚拟机器人采摘路径避障规划[J]. 农业机械学报, 2020, 51(增2): 1-10.
	Xiong Juntao, Li Zhongxing, Chen Shumian, et al. Obstacle Avoidance Planning of Virtual Robot Picking Path Based on Deep Reinforcement Learning[J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(S2): 1-10.
25	熊超, 解武杰, 董文瀚. 基于碰撞锥改进人工势场的无人机避障路径规划[J]. 计算机工程, 2018, 44(9): 314-320.
	Xiong Chao, Xie Wujie, Dong Wenhan. Obstacle Avoidance Path Planning for UAV Based on Artificial Potential Field Improved by Collision Cone[J]. Computer Engineering, 2018, 44(9): 314-320.

奖惩机制类型	实验数	成功数	移动采摘机器人每次完成任务所使用时间/s				成功率/%
奖惩机制类型	实验数	成功数	最小值	平均值	最大值	标准差	成功率/%
接触碰撞惩罚	120	104	9.41	9.66	10.17	0.41	86.7
基于人工势场法	120	111	10.03	10.48	11.23	0.67	92.5
改进人工势场法	120	117	8.96	9.15	9.47	0.25	97.5

[1]	Jiang Ming, He Tao. Solving the Vehicle Routing Problem Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(9): 2177-2187.
[2]	Yu Yiran, Lai Huicheng, Gao Guxue, Zhang Guo, Peng Wangyinan, Yang Longfei, Huang Junhao. Optimization Method for Multi Agricultural Machinery Collaborative Operation Based on Genetic Algorithm and A * Algorithm [J]. Journal of System Simulation, 2025, 37(9): 2397-2408.
[3]	Ni Peilong, Mao Pengjun, Wang Ning, Yang Mengjie. Robot Path Planning Based on Improved A-DDQN Algorithm [J]. Journal of System Simulation, 2025, 37(9): 2420-2430.
[4]	Zhang Kaixiang, Mao Jianlin, Wang Niya, Xu Zhihao. Multi-robot Hierarchical Collaborative k-robust Path Planning for Path Interference [J]. Journal of System Simulation, 2025, 37(8): 2074-2088.
[5]	Chen Zhen, Wu Zhuoyi, Zhang Lin. Research on Policy Representation in Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(7): 1753-1769.
[6]	Wan Yuhang, Zhu Zilu, Zhong Chunfu, Liu Yongkui, Lin Tingyu, Zhang Lin. Dynamic Path Planning for Robotic Arms Based on an Improved PPO Algorithm [J]. Journal of System Simulation, 2025, 37(6): 1462-1473.
[7]	Ye Chen, Shao Peng, Zhang Shaoping, Li Wenting, Zhou Tengming. Enhanced Artificial Gorilla Algorithm for Mobile Robot Path Planning [J]. Journal of System Simulation, 2025, 37(6): 1474-1485.
[8]	Zhang Yan, Li Binghua, Huo Tao, Liu Rong. Research on Robot Dynamic Obstacle Avoidance Method Based on Improved A* and Dynamic Window Algorithm [J]. Journal of System Simulation, 2025, 37(6): 1555-1564.
[9]	Wu Guohua, Zeng Jiaheng, Wang Dezhi, Zheng Long, Zou Wei. A Quadrotor Trajectory Tracking Control Method Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(5): 1169-1187.
[10]	Zhou Xiaohui, Li Yanqiang, Wang Yong, Zhao Decai, Yang Xiaoyao. Robot Path Planning Based on Ant Colony Algorithm with Dual Heuristic Information [J]. Journal of System Simulation, 2025, 37(5): 1280-1289.
[11]	Yu Die, Bao Baizhong, Si Yan, Duan Jian, Zhan Xiaobin, Shi Tielin. Mobile Robot Path Planning Based on Search-step Optimized A* Algorithm [J]. Journal of System Simulation, 2025, 37(4): 1041-1050.
[12]	Zhang Sen, Dai Qiangqiang. UAV Path Planning Based on Improved Deep Deterministic Policy Gradients [J]. Journal of System Simulation, 2025, 37(4): 875-881.
[13]	He Zhigang, Li Dayan, Wang Niya, Mao Jianlin, Wang Ning. A Multi-robot Collaborative Path Planning Algorithm with Chain Working Mode [J]. Journal of System Simulation, 2025, 37(4): 953-967.
[14]	Wang He, Xu Jianing, Yan Guangyu. Research on Pedestrian Avoidance Strategy for AGV Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(3): 595-606.
[15]	Lin Guijuan, Li Zihan, Wang Yu. Research on Improved A* Algorithm Path Planning Based on Global Key Point Extraction [J]. Journal of System Simulation, 2025, 37(3): 667-678.