基于深度强化学习的AGV行人避让策略研究

doi:10.16182/j.issn1004731x.joss.24-0088

系统仿真学报 ›› 2025, Vol. 37 ›› Issue (3): 595-606.doi: 10.16182/j.issn1004731x.joss.24-0088

• 论文 • 上一篇

基于深度强化学习的AGV行人避让策略研究

王贺¹^,², 许佳宁¹, 闫广宇¹^,²

^1.沈阳建筑大学机械工程学院，辽宁沈阳 110000
^2.沈阳建筑大学现代建筑工程装备与技术国际合作联合实验室，辽宁沈阳 110000

收稿日期:2024-01-22 修回日期:2024-03-07 出版日期:2025-03-17 发布日期:2025-03-21
通讯作者: 闫广宇
第一作者简介:王贺(1981-)，男，副教授，博士，研究方向为陶瓷零件制备及加工技术。
基金资助:
国家自然科学基金(51942507);学科创新引智项目(D18017);中国科协青年人才托举工程项目(2023QNRC001);辽宁省教育厅项目(Infw202017)

Research on Pedestrian Avoidance Strategy for AGV Based on Deep Reinforcement Learning

Wang He¹^,², Xu Jianing¹, Yan Guangyu¹^,²

^1.School of Mechanical Engineering, Shenyang Jianzhu University, Shenyang 110000, China
^2.Joint International Research Laboratory of Modern Construction Engineering Equipment and Technology, Shenyang Jianzhu University, Shenyang 110000, China

Received:2024-01-22 Revised:2024-03-07 Online:2025-03-17 Published:2025-03-21
Contact: Yan Guangyu

摘要/Abstract

摘要：

为控制自动导引车（AGV）在智能工厂环境中避障时能够保障行人的安全舒适，提出一种基于深度强化学习的AGV端到端避障方法。引入YOLOv8模块提取行人位姿信息，并设计了基于视觉的状态空间；根据个人空间理论设计强化学习的奖惩机制，对AGV进入行人舒适空间和发生碰撞等行为进行惩罚；搭建了虚拟仿真系统，使用PPO并结合LSTM网络层完成了避障策略的训练并进行仿真实验验证。仿真结果表明：该避障策略在不建立环境地图、视觉输入的条件下，能够控制AGV在避障过程中与行人保持舒适的社交距离。

关键词: 深度强化学习, 自动导引车, YOLOv8, 近端策略优化, 避障, 个人空间理论, 端到端

Abstract:

To ensure the safety and comfort of pedestrians during Automated Guided Vehicle (AGV) obstacle avoidance in smart factory environments, a deep reinforcement learning-based end-to-end obstacle avoidance method is proposed.The YOLOv8 module is introduced to extract pedestrian pose information, and a visual-based state space is designed. A reinforcement learning mechanism is formulated based on personal space theory, penalizing AGV behaviors such as entering pedestrian comfort space and collisions. A virtual simulation system is constructed, utilizing PPO algorithm along with LSTM network layer for obstacle avoidance strategy training and simulation experiments. Simulation results indicate that this obstacle avoidance strategy, under conditions of no environmental map establishment and visual input, can control the AGV to maintain a comfortable social distance from pedestrians during obstacle avoidance..

Key words: DRL, AGV, YOLOv8, PPO, obstacle avoidance, personal space theory, end to end

中图分类号:

TP242

王贺,许佳宁,闫广宇 . 基于深度强化学习的AGV行人避让策略研究[J]. 系统仿真学报, 2025, 37(3): 595-606.

Wang He,Xu Jianing,Yan Guangyu . Research on Pedestrian Avoidance Strategy for AGV Based on Deep Reinforcement Learning[J]. Journal of System Simulation, 2025, 37(3): 595-606.

图/表 14

图1

图2

表1

动作空间集合列表

控制方式

动作描述

取值范围

线速度

v l

$v l > 0$ 前进

$v l = 0$ 静止

0,1

角速度

v a

$- π / 2 ≤ v a < 0$ 左转

$v a = 0$ 直行

$0 < v a ≤ π / 2$ 右转

- π / 2, π / 2

表1

表2

个人空间距离定义列表

距离区域	距离范围/m	区域描述
亲密距离	$0 ≤ D i n t i m a t e ≤ 0.45$	身体接触或私人互动
私人距离	$0.45 < D p e r s o n a l ≤ 1.2$	亲朋互动或高度有组织的活动
社交距离	$1.2 < D s o c i a l ≤ 3.5$	正式场合或公共场所的互动
公共距离	$D p u b l i c > 3.5$	没有互动或单向互动

表2

图3

图4

表3

超参数设置列表

超参数	设置
batch_size	256
经验缓存池容量	2 560
折扣系数 $γ$	0.99
裁剪系数 $ε$	0.2
学习率 $l r$	$2 × 10 - 4$
优化器	Adam

表3

图5

图6

图7

图8

表4

仿真实验结果列表

轮次	行人刷新位置	任务完成时间/s	最小距离值/m
1	$(10, - 2)$	13.17	1.05
2	$(10, - 1)$	13.24	1.44
3	$(10,1)$	13.13	1.82
4	$(10,2)$	14.02	1.70

表4

图9

表5

参考文献 21

1	Durrant-Whyte H, Bailey T. Simultaneous Localization and Mapping: Part I[J]. IEEE Robotics & Automation Magazine, 2006, 13(2): 99-110.
2	LaValle S M. Planning Algorithms[M]. Cambridge: Cambridge University Press, 2006.
3	Yi Guohong, Feng Zhili, Mei Tiancan, et al. Multi-AGVs Path Planning Based on Improved Ant Colony Algorithm[J]. The Journal of Supercomputing, 2019, 75(9): 5898-5913.
4	Fransen Karlijn, van Eekelena Joost. Efficient Path Planning for Automated Guided Vehicles Using A* (Astar) Algorithm Incorporating Turning Costs in Search Heuristic[J]. International Journal of Production Research, 2023, 61(3): 707-725.
5	Nwaonumah E, Samanta B. Deep Reinforcement Learning for Visual Navigation of Wheeled Mobile Robots[C]//2020 SoutheastCon. Piscataway: IEEE, 2020: 1-8.
6	Zhu Yuke, Mottaghi R, Kolve E, et al. Target-driven Visual Navigation in Indoor Scenes Using Deep Reinforcement Learning[C]//2017 IEEE International Conference on Robotics and Automation (ICRA). Piscataway: IEEE, 2017: 3357-3364.
7	Li Keyu, Xu Yangxin, Wang Jiankun, et al. SARL: Deep Reinforcement Learning Based Human-aware Navigation for Mobile Robot in Indoor Environments[C]//2019 IEEE International Conference on Robotics and Biomimetics (ROBIO). Piscataway: IEEE, 2019: 688-694.
8	Lu Xiaojun, Woo Hanwool, Faragasso Angela, et al. Socially Aware Robot Navigation in Crowds via Deep Reinforcement Learning with Resilient Reward Functions[J]. Advanced Robotics, 2022, 36(8): 388-403.
9	Wang S H, Wu Y H, Li T H S. Deep Reinforcement Learning with Pedestrian Trajectory Prediction Model for Service Robot Navigation in Crowded Environments[C]//2023 International Conference on Advanced Robotics and Intelligent Systems (ARIS). Piscataway: IEEE, 2023: 1-6.
10	Cimurs Reinis, Jin Han Lee, Il Hong Suh. Goal-oriented Obstacle Avoidance with Deep Reinforcement Learning in Continuous Action Space[J]. Electronics, 2020, 9(3): 411.
11	Lian Pengfei, Yuan Liang, Sun Lihui. Visual Navigation for Mobile Robots Based on Deep Reinforcement Learning[C]//2023 IEEE International Conference on Real-time Computing and Robotics (RCAR). Piscataway: IEEE, 2023: 650-656.
12	Xue Honghu, Hein Benedikt, Bakr Mohamed, et al. Using Deep Reinforcement Learning with Automatic Curriculum Learning for Mapless Navigation in Intralogistics[J]. Applied Sciences, 2022, 12(6): 3153.
13	Sunil Srivatsav Samsani, Mutahira Husna, Mannan Saeed Muhammad. Memory-based Crowd-aware Robot Navigation Using Deep Reinforcement Learning[J]. Complex & Intelligent Systems, 2023, 9(2): 2147-2158.
14	林俊强, 王红军, 邹湘军, 等. 基于DPPO的移动采摘机器人避障路径规划及仿真[J]. 系统仿真学报, 2023, 35(8): 1692-1704.
	Lin Junqiang, Wang Hongjun, Zou Xiangjun, et al. Obstacle Avoidance Path Planning and Simulation of Mobile Picking Robot Based on DPPO[J]. Journal of System Simulation, 2023, 35(8): 1692-1704.
15	Schulman J, Wolski F, Dhariwal P, et al. Proximal Policy Optimization Algorithms[EB/OL]. (2017-08-28) [2023-12-20]. .
16	Schulman J, Levine S, Abbeel P, et al. Trust Region Policy Optimization[C]//Proceedings of the 32nd International Conference on Machine Learning. Chia Laguna Resort: PMLR, 2015: 1889-1897.
17	Mnih V, Adrià Puigdomènech Badia, Mirza M, et al. Asynchronous Methods for Deep Reinforcement Learning[C]//Proceedings of the 33rd International Conference on Machine Learning. Chia Laguna Resort: PMLR, 2016: 1928-1937.
18	夏家伟, 朱旭芳, 罗亚松, 等. 基于深度强化学习的无人艇轨迹跟踪算法研究[J]. 华中科技大学学报(自然科学版), 2023, 51(5): 74-80.
	Xia Jiawei, Zhu Xufang, Luo Yasong, et al. Study on Trajectory Tracking Algorithm of Unmanned Surface Vehicle Based on Deep Reinforcement Learning[J]. Journal of Huazhong University of Science and Technology(Natural Science Edition), 2023, 51(5): 74-80.
19	Xiao Xin, Feng Xinlong. Multi-object Pedestrian Tracking Using Improved YOLOv8 and OC-SORT[J]. Sensors, 2023, 23(20): 8439.
20	Pacchierotti Elena, Christensen Henrik I, Jensfelt Patric. Evaluation of Passing Distance for Social Robots[C]//ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication. Piscataway: IEEE, 2006: 315-320.
21	孙立香, 孙晓娴, 刘成菊, 等. 人群环境中基于深度强化学习的移动机器人避障算法[J]. 信息与控制, 2022, 51(1): 107-118.
	Sun Lixiang, Sun Xiaoxian, Liu Chengju, et al. Obstacle Avoidance Algorithm for Mobile Robot Based on Deep Reinforcement Learning in Crowd Environment[J]. Information and Control, 2022, 51(1): 107-118.

轮次	距离	轮次	距离	轮次	距离
1	1.87	11	1.95	21	0
2	1.68	12	1.96	22	1.63
3	1.92	13	2.45	23	0.85
4	0.74	14	1.25	24	1.31
5	1.28	15	2.09	25	1.29
6	2.15	16	1.61	26	2.15
7	0.56	17	2.01	27	0.95
8	2.35	18	1.55	28	0.59
9	1.80	19	1.30	29	0.79
10	1.35	20	1.35	30	0.72

[1]	黄思进, 文佳, 陈哲毅. 面向边缘车联网系统的智能服务迁移方法[J]. 系统仿真学报, 2025, 37(2): 379-391.
[2]	费帅迪, 蔡长龙, 刘飞, 陈明晖, 刘晓明. 舰船防空反导的目标分配方法研究[J]. 系统仿真学报, 2025, 37(2): 508-516.
[3]	姜嘉成, 贾政轩, 徐钊, 林廷宇, 赵芃芃, 欧一鸣. 基于博弈对抗复杂系统的决策建模与求解[J]. 系统仿真学报, 2025, 37(1): 66-78.
[4]	李超, 李贾宝, 丁才昌, 叶志伟, 左方威. 基于DRL的边缘监控任务卸载与资源分配算法[J]. 系统仿真学报, 2024, 36(9): 2113-2126.
[5]	霍韩淋, 邹湘军, 陈燕, 周馨曌, 陈明猷, 李承恩, 潘耀强, 唐昀超. 基于视觉机器人障碍点云映射避障规划及仿真[J]. 系统仿真学报, 2024, 36(9): 2149-2158.
[6]	姬鹏, 张新元, 高帅轩, 魏铄让. 融合改进A*算法与动态窗口法的路径规划研究[J]. 系统仿真学报, 2024, 36(9): 2171-2180.
[7]	刘斌, 兰莹, 黄文焘, 范勤勤. 融合动态窗口法的无人机动态路径规划算法[J]. 系统仿真学报, 2024, 36(8): 1843-1853.
[8]	祝靖宇, 张宏立, 匡敏驰, 史恒, 朱纪洪, 乔直, 周文卿. 稀疏奖励下基于课程学习的无人机空战仿真[J]. 系统仿真学报, 2024, 36(6): 1452-1467.
[9]	王红军, 林俊强, 邹湘军, 张坡, 周铭轩, 邹伟锐, 唐昀超, 罗陆锋. 基于数字孪生的果园虚拟交互系统构建[J]. 系统仿真学报, 2024, 36(6): 1493-1508.
[10]	王远, 徐琳, 宫小泽, 张永亮, 王永利. 基于梯度的深度强化学习解释方法[J]. 系统仿真学报, 2024, 36(5): 1130-1140.
[11]	刘泽森, 毕盛, 郭传鈜, 王延葵, 董敏. 基于深度学习的机器人局部路径规划方法[J]. 系统仿真学报, 2024, 36(5): 1199-1210.
[12]	张瑞, 周丽, 刘正洋. 融合RRT*与DWA算法的移动机器人动态路径规划[J]. 系统仿真学报, 2024, 36(4): 957-968.
[13]	刘福琳, 李庆鑫. 多移动机器人混合避障算法的编队策略[J]. 系统仿真学报, 2024, 36(3): 726-734.
[14]	秦保新, 张羽霄, 吴思锐, 曹卫冲, 李湛. 基于改进D3QN的煤炭码头卸车排产智能优化方法[J]. 系统仿真学报, 2024, 36(3): 770-781.
[15]	赵莹莹, 董普森, 朱天晨, 李凡, 苏运, 邰振赢, 孙庆赟, 凡航. 面向电网拓扑调度仿真的采样效率优化方法研究[J]. 系统仿真学报, 2024, 36(2): 283-295.

基于深度强化学习的AGV行人避让策略研究

Research on Pedestrian Avoidance Strategy for AGV Based on Deep Reinforcement Learning

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 21

相关文章 15

编辑推荐

Metrics

本文评价