Research on Pedestrian Avoidance Strategy for AGV Based on Deep Reinforcement Learning

doi:10.16182/j.issn1004731x.joss.24-0088

Abstract

Abstract:

To ensure the safety and comfort of pedestrians during Automated Guided Vehicle (AGV) obstacle avoidance in smart factory environments, a deep reinforcement learning-based end-to-end obstacle avoidance method is proposed.The YOLOv8 module is introduced to extract pedestrian pose information, and a visual-based state space is designed. A reinforcement learning mechanism is formulated based on personal space theory, penalizing AGV behaviors such as entering pedestrian comfort space and collisions. A virtual simulation system is constructed, utilizing PPO algorithm along with LSTM network layer for obstacle avoidance strategy training and simulation experiments. Simulation results indicate that this obstacle avoidance strategy, under conditions of no environmental map establishment and visual input, can control the AGV to maintain a comfortable social distance from pedestrians during obstacle avoidance..

Key words: DRL, AGV, YOLOv8, PPO, obstacle avoidance, personal space theory, end to end

CLC Number:

TP242

Wang He, Xu Jianing, Yan Guangyu. Research on Pedestrian Avoidance Strategy for AGV Based on Deep Reinforcement Learning[J]. Journal of System Simulation, 2025, 37(3): 595-606.

Figures/Tables 14

Fig. 1

Fig. 2

Table 1

Action space collection list

控制方式

动作描述

取值范围

线速度

v l

$v l > 0$ 前进

$v l = 0$ 静止

0,1

角速度

v a

$- π / 2 ≤ v a < 0$ 左转

$v a = 0$ 直行

$0 < v a ≤ π / 2$ 右转

- π / 2, π / 2

Table 1

Table 2

Personal space distance list

距离区域	距离范围/m	区域描述
亲密距离	$0 ≤ D i n t i m a t e ≤ 0.45$	身体接触或私人互动
私人距离	$0.45 < D p e r s o n a l ≤ 1.2$	亲朋互动或高度有组织的活动
社交距离	$1.2 < D s o c i a l ≤ 3.5$	正式场合或公共场所的互动
公共距离	$D p u b l i c > 3.5$	没有互动或单向互动

Table 2

Fig. 3

Fig. 4

Table 3

hyperparameter settings list

超参数	设置
batch_size	256
经验缓存池容量	2 560
折扣系数 $γ$	0.99
裁剪系数 $ε$	0.2
学习率 $l r$	$2 × 10 - 4$
优化器	Adam

Table 3

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 4

Simulation experimental results list

轮次	行人刷新位置	任务完成时间/s	最小距离值/m
1	$(10, - 2)$	13.17	1.05
2	$(10, - 1)$	13.24	1.44
3	$(10,1)$	13.13	1.82
4	$(10,2)$	14.02	1.70

Table 4

Fig. 9

Table 5

References 21

1	Durrant-Whyte H, Bailey T. Simultaneous Localization and Mapping: Part I[J]. IEEE Robotics & Automation Magazine, 2006, 13(2): 99-110.
2	LaValle S M. Planning Algorithms[M]. Cambridge: Cambridge University Press, 2006.
3	Yi Guohong, Feng Zhili, Mei Tiancan, et al. Multi-AGVs Path Planning Based on Improved Ant Colony Algorithm[J]. The Journal of Supercomputing, 2019, 75(9): 5898-5913.
4	Fransen Karlijn, van Eekelena Joost. Efficient Path Planning for Automated Guided Vehicles Using A* (Astar) Algorithm Incorporating Turning Costs in Search Heuristic[J]. International Journal of Production Research, 2023, 61(3): 707-725.
5	Nwaonumah E, Samanta B. Deep Reinforcement Learning for Visual Navigation of Wheeled Mobile Robots[C]//2020 SoutheastCon. Piscataway: IEEE, 2020: 1-8.
6	Zhu Yuke, Mottaghi R, Kolve E, et al. Target-driven Visual Navigation in Indoor Scenes Using Deep Reinforcement Learning[C]//2017 IEEE International Conference on Robotics and Automation (ICRA). Piscataway: IEEE, 2017: 3357-3364.
7	Li Keyu, Xu Yangxin, Wang Jiankun, et al. SARL: Deep Reinforcement Learning Based Human-aware Navigation for Mobile Robot in Indoor Environments[C]//2019 IEEE International Conference on Robotics and Biomimetics (ROBIO). Piscataway: IEEE, 2019: 688-694.
8	Lu Xiaojun, Woo Hanwool, Faragasso Angela, et al. Socially Aware Robot Navigation in Crowds via Deep Reinforcement Learning with Resilient Reward Functions[J]. Advanced Robotics, 2022, 36(8): 388-403.
9	Wang S H, Wu Y H, Li T H S. Deep Reinforcement Learning with Pedestrian Trajectory Prediction Model for Service Robot Navigation in Crowded Environments[C]//2023 International Conference on Advanced Robotics and Intelligent Systems (ARIS). Piscataway: IEEE, 2023: 1-6.
10	Cimurs Reinis, Jin Han Lee, Il Hong Suh. Goal-oriented Obstacle Avoidance with Deep Reinforcement Learning in Continuous Action Space[J]. Electronics, 2020, 9(3): 411.
11	Lian Pengfei, Yuan Liang, Sun Lihui. Visual Navigation for Mobile Robots Based on Deep Reinforcement Learning[C]//2023 IEEE International Conference on Real-time Computing and Robotics (RCAR). Piscataway: IEEE, 2023: 650-656.
12	Xue Honghu, Hein Benedikt, Bakr Mohamed, et al. Using Deep Reinforcement Learning with Automatic Curriculum Learning for Mapless Navigation in Intralogistics[J]. Applied Sciences, 2022, 12(6): 3153.
13	Sunil Srivatsav Samsani, Mutahira Husna, Mannan Saeed Muhammad. Memory-based Crowd-aware Robot Navigation Using Deep Reinforcement Learning[J]. Complex & Intelligent Systems, 2023, 9(2): 2147-2158.
14	林俊强, 王红军, 邹湘军, 等. 基于DPPO的移动采摘机器人避障路径规划及仿真[J]. 系统仿真学报, 2023, 35(8): 1692-1704.
	Lin Junqiang, Wang Hongjun, Zou Xiangjun, et al. Obstacle Avoidance Path Planning and Simulation of Mobile Picking Robot Based on DPPO[J]. Journal of System Simulation, 2023, 35(8): 1692-1704.
15	Schulman J, Wolski F, Dhariwal P, et al. Proximal Policy Optimization Algorithms[EB/OL]. (2017-08-28) [2023-12-20]. .
16	Schulman J, Levine S, Abbeel P, et al. Trust Region Policy Optimization[C]//Proceedings of the 32nd International Conference on Machine Learning. Chia Laguna Resort: PMLR, 2015: 1889-1897.
17	Mnih V, Adrià Puigdomènech Badia, Mirza M, et al. Asynchronous Methods for Deep Reinforcement Learning[C]//Proceedings of the 33rd International Conference on Machine Learning. Chia Laguna Resort: PMLR, 2016: 1928-1937.
18	夏家伟, 朱旭芳, 罗亚松, 等. 基于深度强化学习的无人艇轨迹跟踪算法研究[J]. 华中科技大学学报(自然科学版), 2023, 51(5): 74-80.
	Xia Jiawei, Zhu Xufang, Luo Yasong, et al. Study on Trajectory Tracking Algorithm of Unmanned Surface Vehicle Based on Deep Reinforcement Learning[J]. Journal of Huazhong University of Science and Technology(Natural Science Edition), 2023, 51(5): 74-80.
19	Xiao Xin, Feng Xinlong. Multi-object Pedestrian Tracking Using Improved YOLOv8 and OC-SORT[J]. Sensors, 2023, 23(20): 8439.
20	Pacchierotti Elena, Christensen Henrik I, Jensfelt Patric. Evaluation of Passing Distance for Social Robots[C]//ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication. Piscataway: IEEE, 2006: 315-320.
21	孙立香, 孙晓娴, 刘成菊, 等. 人群环境中基于深度强化学习的移动机器人避障算法[J]. 信息与控制, 2022, 51(1): 107-118.
	Sun Lixiang, Sun Xiaoxian, Liu Chengju, et al. Obstacle Avoidance Algorithm for Mobile Robot Based on Deep Reinforcement Learning in Crowd Environment[J]. Information and Control, 2022, 51(1): 107-118.

轮次	距离	轮次	距离	轮次	距离
1	1.87	11	1.95	21	0
2	1.68	12	1.96	22	1.63
3	1.92	13	2.45	23	0.85
4	0.74	14	1.25	24	1.31
5	1.28	15	2.09	25	1.29
6	2.15	16	1.61	26	2.15
7	0.56	17	2.01	27	0.95
8	2.35	18	1.55	28	0.59
9	1.80	19	1.30	29	0.79
10	1.35	20	1.35	30	0.72

[1]	Gong Feng, Jiang Tao, Zhang Qin, Liu Yu. Simulation and Optimization of Support Processes for Aircraft Fleet Launch Under Limited Resources [J]. Journal of System Simulation, 2025, 37(8): 1965-1977.
[2]	Lu Bin, Yang Xuan, Yang Zhenyu, Gao Xiaotian. Adaptive Sampling and Ghost Multi-scale Fusion for Lightweight Weld Defect Detection [J]. Journal of System Simulation, 2025, 37(8): 1978-1990.
[3]	Zhang Yue, Zhang Wenliang, Feng Qiang, Guo Xing, Ren Yi, Wang Zili. Combat-oriented Comprehensive Simulation and Verification Technology for Equipment System RMS [J]. Journal of System Simulation, 2025, 37(7): 1823-1835.
[4]	Wan Yuhang, Zhu Zilu, Zhong Chunfu, Liu Yongkui, Lin Tingyu, Zhang Lin. Dynamic Path Planning for Robotic Arms Based on an Improved PPO Algorithm [J]. Journal of System Simulation, 2025, 37(6): 1462-1473.
[5]	Zhang Yan, Li Binghua, Huo Tao, Liu Rong. Research on Robot Dynamic Obstacle Avoidance Method Based on Improved A* and Dynamic Window Algorithm [J]. Journal of System Simulation, 2025, 37(6): 1555-1564.
[6]	Wang Ziyi, Zhang Kai, Qian Dianwei, Liu Yuzhen. A DRL⁃based Approach for Distributed Equipment Nodes Selection [J]. Journal of System Simulation, 2025, 37(6): 1565-1573.
[7]	Wu Guohua, Zeng Jiaheng, Wang Dezhi, Zheng Long, Zou Wei. A Quadrotor Trajectory Tracking Control Method Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(5): 1169-1187.
[8]	Li Jie, Liu Yang, Li Liang, Su Bengan, Wei Jialong, Zhou Guangda, Shi Yanmin, Zhao Zhen. Remote Sensing Small Object Detection Based on Cross-stage Two-branch Feature Aggregation [J]. Journal of System Simulation, 2025, 37(4): 1025-1040.
[9]	Zhang Sen, Dai Qiangqiang. UAV Path Planning Based on Improved Deep Deterministic Policy Gradients [J]. Journal of System Simulation, 2025, 37(4): 875-881.
[10]	Li Min, Zhang Sen, Zeng Xiangguang, Wang Gang, Zhang Tongwei, Xie Dijie, Ren Wenzhe, Zhang Tao. Trajectory Planning of Quadruped Robot Over Obstacle with Single Leg Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(4): 895-909.
[11]	Zhang Bin, Lei Yonglin, Li Qun, Gao Yuan, Chen Yong, Zhu Jiajun, Bao Chenlong. Reinforcement Learning Modeling of Missile Penetration Decision Based on Combat Simulation [J]. Journal of System Simulation, 2025, 37(3): 763-774.
[12]	Huang Sijin, Wen Jia, Chen Zheyi. Intelligent Service Migration towards MEC-based IoV Systems [J]. Journal of System Simulation, 2025, 37(2): 379-391.
[13]	Fei Shuaidi, Cai Changlong, Liu Fei, Chen Minghui, Liu Xiaoming. Research on the Target Allocation Method for Air Defense and Anti-missile Defense of Naval Ships [J]. Journal of System Simulation, 2025, 37(2): 508-516.
[14]	Peng Yi, Lei Yunkui, Yang Qingqing, Li Hui, Wang Jianming. Improved PID Search Algorithm for UAV Path Planning in Mountainous Environments [J]. Journal of System Simulation, 2025, 37(12): 3075-3086.
[15]	Wu Shuheng, Liu Yongkui, Zhang Lin, Xiao Yingying, Wang Lihui. Lightweight Assembly Workpiece Detection Algorithm Based on Improved YOLOv8 [J]. Journal of System Simulation, 2025, 37(12): 3099-3111.