Research on Pedestrian Avoidance Strategy for AGV Based on Deep Reinforcement Learning

doi:10.16182/j.issn1004731x.joss.24-0088

Abstract

Abstract:

To ensure the safety and comfort of pedestrians during Automated Guided Vehicle (AGV) obstacle avoidance in smart factory environments, a deep reinforcement learning-based end-to-end obstacle avoidance method is proposed.The YOLOv8 module is introduced to extract pedestrian pose information, and a visual-based state space is designed. A reinforcement learning mechanism is formulated based on personal space theory, penalizing AGV behaviors such as entering pedestrian comfort space and collisions. A virtual simulation system is constructed, utilizing PPO algorithm along with LSTM network layer for obstacle avoidance strategy training and simulation experiments. Simulation results indicate that this obstacle avoidance strategy, under conditions of no environmental map establishment and visual input, can control the AGV to maintain a comfortable social distance from pedestrians during obstacle avoidance..

Key words: DRL, AGV, YOLOv8, PPO, obstacle avoidance, personal space theory, end to end

CLC Number:

TP242

Wang He, Xu Jianing, Yan Guangyu. Research on Pedestrian Avoidance Strategy for AGV Based on Deep Reinforcement Learning[J]. Journal of System Simulation, 2025, 37(3): 595-606.

Figures/Tables 14

Fig. 1

Fig. 2

Table 1

Action space collection list

控制方式

动作描述

取值范围

线速度

v l

$v l > 0$ 前进

$v l = 0$ 静止

0,1

角速度

v a

$- π / 2 ≤ v a < 0$ 左转

$v a = 0$ 直行

$0 < v a ≤ π / 2$ 右转

- π / 2, π / 2

Table 1

Table 2

Personal space distance list

距离区域	距离范围/m	区域描述
亲密距离	$0 ≤ D i n t i m a t e ≤ 0.45$	身体接触或私人互动
私人距离	$0.45 < D p e r s o n a l ≤ 1.2$	亲朋互动或高度有组织的活动
社交距离	$1.2 < D s o c i a l ≤ 3.5$	正式场合或公共场所的互动
公共距离	$D p u b l i c > 3.5$	没有互动或单向互动

Table 2

Fig. 3

Fig. 4

Table 3

hyperparameter settings list

超参数	设置
batch_size	256
经验缓存池容量	2 560
折扣系数 $γ$	0.99
裁剪系数 $ε$	0.2
学习率 $l r$	$2 × 10 - 4$
优化器	Adam

Table 3

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 4

Simulation experimental results list

轮次	行人刷新位置	任务完成时间/s	最小距离值/m
1	$(10, - 2)$	13.17	1.05
2	$(10, - 1)$	13.24	1.44
3	$(10,1)$	13.13	1.82
4	$(10,2)$	14.02	1.70

Table 4

Fig. 9

Table 5

References 21

1	Durrant-Whyte H, Bailey T. Simultaneous Localization and Mapping: Part I[J]. IEEE Robotics & Automation Magazine, 2006, 13(2): 99-110.
2	LaValle S M. Planning Algorithms[M]. Cambridge: Cambridge University Press, 2006.
3	Yi Guohong, Feng Zhili, Mei Tiancan, et al. Multi-AGVs Path Planning Based on Improved Ant Colony Algorithm[J]. The Journal of Supercomputing, 2019, 75(9): 5898-5913.
4	Fransen Karlijn, van Eekelena Joost. Efficient Path Planning for Automated Guided Vehicles Using A* (Astar) Algorithm Incorporating Turning Costs in Search Heuristic[J]. International Journal of Production Research, 2023, 61(3): 707-725.
5	Nwaonumah E, Samanta B. Deep Reinforcement Learning for Visual Navigation of Wheeled Mobile Robots[C]//2020 SoutheastCon. Piscataway: IEEE, 2020: 1-8.
6	Zhu Yuke, Mottaghi R, Kolve E, et al. Target-driven Visual Navigation in Indoor Scenes Using Deep Reinforcement Learning[C]//2017 IEEE International Conference on Robotics and Automation (ICRA). Piscataway: IEEE, 2017: 3357-3364.
7	Li Keyu, Xu Yangxin, Wang Jiankun, et al. SARL: Deep Reinforcement Learning Based Human-aware Navigation for Mobile Robot in Indoor Environments[C]//2019 IEEE International Conference on Robotics and Biomimetics (ROBIO). Piscataway: IEEE, 2019: 688-694.
8	Lu Xiaojun, Woo Hanwool, Faragasso Angela, et al. Socially Aware Robot Navigation in Crowds via Deep Reinforcement Learning with Resilient Reward Functions[J]. Advanced Robotics, 2022, 36(8): 388-403.
9	Wang S H, Wu Y H, Li T H S. Deep Reinforcement Learning with Pedestrian Trajectory Prediction Model for Service Robot Navigation in Crowded Environments[C]//2023 International Conference on Advanced Robotics and Intelligent Systems (ARIS). Piscataway: IEEE, 2023: 1-6.
10	Cimurs Reinis, Jin Han Lee, Il Hong Suh. Goal-oriented Obstacle Avoidance with Deep Reinforcement Learning in Continuous Action Space[J]. Electronics, 2020, 9(3): 411.
11	Lian Pengfei, Yuan Liang, Sun Lihui. Visual Navigation for Mobile Robots Based on Deep Reinforcement Learning[C]//2023 IEEE International Conference on Real-time Computing and Robotics (RCAR). Piscataway: IEEE, 2023: 650-656.
12	Xue Honghu, Hein Benedikt, Bakr Mohamed, et al. Using Deep Reinforcement Learning with Automatic Curriculum Learning for Mapless Navigation in Intralogistics[J]. Applied Sciences, 2022, 12(6): 3153.
13	Sunil Srivatsav Samsani, Mutahira Husna, Mannan Saeed Muhammad. Memory-based Crowd-aware Robot Navigation Using Deep Reinforcement Learning[J]. Complex & Intelligent Systems, 2023, 9(2): 2147-2158.
14	林俊强, 王红军, 邹湘军, 等. 基于DPPO的移动采摘机器人避障路径规划及仿真[J]. 系统仿真学报, 2023, 35(8): 1692-1704.
	Lin Junqiang, Wang Hongjun, Zou Xiangjun, et al. Obstacle Avoidance Path Planning and Simulation of Mobile Picking Robot Based on DPPO[J]. Journal of System Simulation, 2023, 35(8): 1692-1704.
15	Schulman J, Wolski F, Dhariwal P, et al. Proximal Policy Optimization Algorithms[EB/OL]. (2017-08-28) [2023-12-20]. .
16	Schulman J, Levine S, Abbeel P, et al. Trust Region Policy Optimization[C]//Proceedings of the 32nd International Conference on Machine Learning. Chia Laguna Resort: PMLR, 2015: 1889-1897.
17	Mnih V, Adrià Puigdomènech Badia, Mirza M, et al. Asynchronous Methods for Deep Reinforcement Learning[C]//Proceedings of the 33rd International Conference on Machine Learning. Chia Laguna Resort: PMLR, 2016: 1928-1937.
18	夏家伟, 朱旭芳, 罗亚松, 等. 基于深度强化学习的无人艇轨迹跟踪算法研究[J]. 华中科技大学学报(自然科学版), 2023, 51(5): 74-80.
	Xia Jiawei, Zhu Xufang, Luo Yasong, et al. Study on Trajectory Tracking Algorithm of Unmanned Surface Vehicle Based on Deep Reinforcement Learning[J]. Journal of Huazhong University of Science and Technology(Natural Science Edition), 2023, 51(5): 74-80.
19	Xiao Xin, Feng Xinlong. Multi-object Pedestrian Tracking Using Improved YOLOv8 and OC-SORT[J]. Sensors, 2023, 23(20): 8439.
20	Pacchierotti Elena, Christensen Henrik I, Jensfelt Patric. Evaluation of Passing Distance for Social Robots[C]//ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication. Piscataway: IEEE, 2006: 315-320.
21	孙立香, 孙晓娴, 刘成菊, 等. 人群环境中基于深度强化学习的移动机器人避障算法[J]. 信息与控制, 2022, 51(1): 107-118.
	Sun Lixiang, Sun Xiaoxian, Liu Chengju, et al. Obstacle Avoidance Algorithm for Mobile Robot Based on Deep Reinforcement Learning in Crowd Environment[J]. Information and Control, 2022, 51(1): 107-118.

轮次	距离	轮次	距离	轮次	距离
1	1.87	11	1.95	21	0
2	1.68	12	1.96	22	1.63
3	1.92	13	2.45	23	0.85
4	0.74	14	1.25	24	1.31
5	1.28	15	2.09	25	1.29
6	2.15	16	1.61	26	2.15
7	0.56	17	2.01	27	0.95
8	2.35	18	1.55	28	0.59
9	1.80	19	1.30	29	0.79
10	1.35	20	1.35	30	0.72

[1]	Huang Sijin, Wen Jia, Chen Zheyi. Intelligent Service Migration towards MEC-based IoV Systems [J]. Journal of System Simulation, 2025, 37(2): 379-391.
[2]	Fei Shuaidi, Cai Changlong, Liu Fei, Chen Minghui, Liu Xiaoming. Research on the Target Allocation Method for Air Defense and Anti-missile Defense of Naval Ships [J]. Journal of System Simulation, 2025, 37(2): 508-516.
[3]	Li Chao, Li Jiabao, Ding Caichang, Ye Zhiwei, Zuo Fangwei. Edge Surveillance Task Offloading and Resource Allocation Algorithm Based on DRL [J]. Journal of System Simulation, 2024, 36(9): 2113-2126.
[4]	Huo Hanlin, Zou Xiangjun, Chen Yan, Zhou Xinzhao, Chen Mingyou, Li Chengen, Pan Yaoqiang, Tang Yunchao. Visual Robot Obstacle Avoidance Planning and Simulation Using Mapped Point Clouds [J]. Journal of System Simulation, 2024, 36(9): 2149-2158.
[5]	Ji Peng, Zhang Xinyuan, Gao Shuaixuan, Wei Shuorang. Path Planning Based on Improved A* and Dynamic Window Approach [J]. Journal of System Simulation, 2024, 36(9): 2171-2180.
[6]	Liu Bin, Lan Ying, Huang Wentao, Fan Qinqin. UAV Dynamic Path Planning Algorithm Combined with Dynamic Window Approach [J]. Journal of System Simulation, 2024, 36(8): 1843-1853.
[7]	Luo Tianyu, Xing Lining, Wang Rui, Wang Ling, Shi Jianmai, Sun Xin. Dynamic Air Defense Resource Allocation Optimization Based on Improved Differential Evolution Algorithm [J]. Journal of System Simulation, 2024, 36(6): 1285-1297.
[8]	Zhu Zilu, Liu Yongkui, Zhang Lin, Wang Lihui, Lin Tingyu. Simulation of Robotic Peg-in-hole Assembly Strategy Based on DRL [J]. Journal of System Simulation, 2024, 36(6): 1414-1424.
[9]	Zhou Zhiyong, Mo Fei, Zhao Kai, Hao Yunbo, Qian Yufeng. Adaptive PID Control Algorithm Based on PPO [J]. Journal of System Simulation, 2024, 36(6): 1425-1432.
[10]	Zhu Jingyu, Zhang Hongli, Kuang Minchi, Shi Heng, Zhu Jihong, Qiao zhi, Zhou Wenqing. Curriculum Learning-based Simulation of UAV Air Combat Under Sparse Rewards [J]. Journal of System Simulation, 2024, 36(6): 1452-1467.
[11]	Wang Hongjun, Lin Junqiang, Zou Xiangjun, Zhang Po, Zhou Mingxuan, Zou Weirui, Tang Yunchao, Luo Lufeng. Construction of a Virtual Interactive System for Orchards Based on Digital Twin [J]. Journal of System Simulation, 2024, 36(6): 1493-1508.
[12]	Wang Yuan, Xu Lin, Gong Xiaoze, Zhang Yongliang, Wang Yongli. Gradient-based Deep Reinforcement Learning Interpretation Methods [J]. Journal of System Simulation, 2024, 36(5): 1130-1140.
[13]	Liu Zesen, Bi Sheng, Guo Chuanhong, Wang Yankui, Dong Min. Deep Learning Based Local Path Planning Method for Moving Robots [J]. Journal of System Simulation, 2024, 36(5): 1199-1210.
[14]	Zhang Rui, Zhou Li, Liu Zhengyang. Dynamic Path Planning for Mobile Robot Based on RRT* and Dynamic Window Approach [J]. Journal of System Simulation, 2024, 36(4): 957-968.
[15]	Liu Fulin, Li Qingxin. Formation Strategy of Hybrid Obstacle Avoidance Algorithm for Multiple Mobile Robots [J]. Journal of System Simulation, 2024, 36(3): 726-734.