Intelligent Path Planning for Mobile Robots Based on SAC Algorithm

doi:10.16182/j.issn1004731x.joss.22-0412

Abstract

Laiyi Yang, Jing Bi, Haitao Yuan. Intelligent Path Planning for Mobile Robots Based on SAC Algorithm[J]. Journal of System Simulation, 2023, 35(8): 1726-1736.

Figures/Tables 11

Fig. 1

Fig. 2

Table 1

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 2

Fig. 9

References 35

1	朱大奇, 颜明重. 移动机器人路径规划技术综述[J]. 控制与决策, 2010, 25(7): 961-967.
	Zhu Daqi, Yan Mingzhong. Survey on Technology of Mobile Robot Path Planning[J]. Control and Decision, 2010, 25(7): 961-967.
2	黄晓冬, 苑海涛, 毕敬, 等. 基于DQN的海战场舰船路径规划及仿真[J]. 系统仿真学报, 2021, 33(10): 2440-2448.
	Huang Xiaodong, Yuan Haitao, Bi Jing, et al. DQN-Based Path Planning Method and Simulation for Submarine and Warship in Naval Battlefield[J]. Journal of System Simulation, 2021, 33(10): 2440-2448.
3	Gasparetto A, Boscariol P, Lanzutti A, et al. Path Planning and Trajectory Planning Algorithms: A General Overview[M]//Carbone G, Gomez-Bravo F. Motion and Operation Planning of Robotic Systems: Background and Practical Approaches. Cham: Springer International Publishing, 2015: 3-27.
4	张捍东, 郑睿, 岑豫皖. 移动机器人路径规划技术的现状与展望[J]. 系统仿真学报, 2005, 17(2): 439-443.
	Zhang Handong, Zheng Rui, Cen Yuwan. Present Situation and Future Development of Mobile Robot Path Planning Technology[J]. Journal of System Simulation, 2005, 17(2): 439-443.
5	Tavares R S, Martins T C, Tsuzuki M S G. Simulated Annealing With Adaptive Neighborhood: A Case Study in Off-line Robot Path Planning[J]. Expert Systems With Applications, 2011, 38(4): 2951-2965.
6	Liu Yuecheng, Zhao Yongjia. A Virtual-waypoint Based Artificial Potential Field Method for UAV Path Planning[C]//2016 IEEE Chinese Guidance, Navigation and Control Conference(CGNCC). Piscataway, NJ, USA: IEEE, 2016: 949-953.
7	Le A V, Prabakaran V, Sivanantham V, et al. Modified A-star Algorithm for Efficient Coverage Path Planning in Tetris Inspired Self-reconfigurable Robot With Integrated Laser Sensor[J]. Sensors, 2018, 18(8): 2585.
8	邓学强. 基于改进人工势场法的移动机器人路径规划[J]. 山东理工大学学报(自然科学版), 2014, 28(1): 38-41.
	Deng Xueqiang. Path Planning of Mobile Robot Based on Modified Artificial Potential Field Method[J]. Journal of Shandong University of Technology(Natural Science Edition), 2014, 28(1): 38-41.
9	Rashid R, Perumal N, Elamvazuthi I, et al. Mobile Robot Path Planning Using Ant Colony Optimization[C]//2016 2nd IEEE International Symposium on Robotics and Manufacturing Automation (ROMA). Piscataway, NJ, USA: IEEE, 2016: 1-6.
10	Roberge V, Tarbouchi M, Labonté Gilles. Comparison of Parallel Genetic Algorithm and Particle Swarm Optimization for Real-time UAV Path Planning[J]. IEEE Transactions on Industrial Informatics, 2013, 9(1): 132-141.
11	乔俊飞, 侯占军, 阮晓钢. 基于神经网络的强化学习在避障中的应用[J]. 清华大学学报(自然科学版), 2008, 48(增2): 1747-1750.
	Qiao Junfei, Hou Zhanjun, Ruan Xiaogang. Neural Network-based Reinforcement Learning Applied to Obstacle Avoidance[J]. Journal of Tsinghua University(Science and Technology), 2008, 48(S2): 1747-1750.
12	刘建伟, 高峰, 罗雄麟. 基于值函数和策略梯度的深度强化学习综述[J]. 计算机学报, 2019, 42(6): 1406-1438.
	Liu Jianwei, Gao Feng, Luo Xionglin. Survey of Deep Reinforcement Learning Based on Value Function and Policy Gradient[J]. Chinese Journal of Computers, 2019, 42(6): 1406-1438.
13	Low E S, Ong P, Cheah K C. Solving the Optimal Path Planning of a Mobile Robot Using Improved Q-learning[J]. Robotics and Autonomous Systems, 2019, 115: 143-161.
14	Harwin L, Supriya P. Comparison of SARSA Algorithm and Temporal Difference Learning Algorithm for Robotic Path Planning for Static Obstacles[C]//2019 Third International Conference on Inventive Systems and Control(ICISC). Piscataway, NJ, USA: IEEE, 2019: 472-476.
15	Li Jianxi, Chen Yiting, Zhao Xiuniao, et al. An Improved DQN Path Planning Algorithm[J]. The Journal of Supercomputing, 2022, 78(1): 616-639.
16	陶重犇, 雷祝兵, 李春光, 等. 基于改进模拟退火算法的搬运机器人路径规划[J]. 计算机测量与控制, 2018, 26(7): 182-185.
	Tao Zhongben, Lei Zhubing, Li Chunguang, et al. Path Planning of Handling Robot Based on Improved Simulated Annealing Algorithm[J]. Computer Measurement & Control, 2018, 26(7): 182-185.
17	Liu Zhixian, Yuan Xiaofang, Huang Guoming, et al. Two Potential Fields Fused Adaptive Path Planning System for Autonomous Vehicle Under Different Velocities[J]. ISA Transactions, 2021, 112: 176-185.
18	蒙盾, 胡卓, 张华军. 基于改进A^*算法的多层邮轮疏散系统仿真[J]. 系统仿真学报, 2022, 34(6): 1375-1382.
	Meng Dun, Hu Zhuo, Zhang Huajun. Simulation of Multi-layer Ship Evacuation System Based on Improved A^* Algorithm[J]. Journal of System Simulation, 2022, 34(6): 1375-1382.
19	徐力, 刘云华, 王启富. 自适应遗传算法在机器人路径规划的应用[J]. 计算机工程与应用, 2020, 56(18): 36-41.
	Xu Li, Liu Yunhua, Wang Qifu. Application of Adaptive Genetic Algorithm in Robot Path Planning[J]. Computer Engineering and Applications, 2020, 56(18): 36-41.
20	Qu Hong, Xing Ke, Alexander T. An Improved Genetic Algorithm With Co-evolutionary Strategy for Global Path Planning of Multiple Mobile Robots[J]. Neurocomputing, 2013, 120: 509-517.
21	Liu Yiyang, Hou Zheng, Tan Yuanyuan, et al. Research on Multi-AGVs Path Planning and Coordination Mechanism[J]. IEEE Access, 2020, 8: 213345-213356.
22	Ajeil F H, Ibraheem I K, Azar A T, et al. Grid-based Mobile Robot Path Planning Using Aging-based Ant Colony Optimization Algorithm in Static and Dynamic Environments[J]. Sensors, 2020, 20(7): 1880.
23	罗阳阳, 彭晓燕. 基于改进PSO的四轮移动机器人全局路径规划[J]. 计算机仿真, 2020, 37(7): 373-379.
	Luo Yangyang, Peng Xiaoyan. Global Path Planning of Four-wheel Mobile Robot Based on Improved PSO[J]. Computer Simulation, 2020, 37(7): 373-379.
24	Zou Qijie, Zhang Yue, Liu Shihui. A Path Planning Algorithm Based on RRT and SARSA (λ) in Unknown and Complex Conditions[C]//2020 Chinese Control And Decision Conference(CCDC). Piscataway, NJ, USA: IEEE, 2020: 2035-2040.
25	Liao Xiaofei, Wang Yang, Xuan Yiliang, et al. AGV Path Planning Model Based on Reinforcement Learning[C]//2020 Chinese Automation Congress(CAC). Piscataway, NJ, USA: IEEE, 2020: 6722-6726.
26	Meerza S I A, Islam M, Uzzal M M. Q-learning Based Particle Swarm Optimization Algorithm for Optimal Path Planning of Swarm of Mobile Robots[C]//2019 1st International Conference on Advances in Science, Engineering and Robotics Technology(ICASERT). Piscataway, NJ, USA: IEEE, 2019: 1-5.
27	Liu Zhiyong, Lan Fei, Yang Haibo. Partition Heuristic RRT Algorithm of Path Planning Based on Q-learning[C]//2019 IEEE 4th Advanced Information Technology, Electronic and Automation Control Conference(IAEAC). Piscataway, NJ, USA: IEEE, 2019: 386-392.
28	Yao Qingfeng, Zheng Zeyu, Qi Liang, et al. Path Planning Method With Improved Artificial Potential Field-a Reinforcement Learning Perspective[J]. IEEE Access, 2020, 8: 135513-135523.
29	Zhou Siyu, Liu Xin, Xu Yingfu, et al. A Deep Q-network (DQN) Based Path Planning Method for Mobile Robots[C]//2018 IEEE International Conference on Information and Automation(ICIA). Piscataway, NJ, USA: IEEE, 2018: 366-371.
30	Dong Yuansheng, Zou Xingjie. Mobile Robot Path Planning Based on Improved DDPG Reinforcement Learning Algorithm[C]//2020 IEEE 11th International Conference on Software Engineering and Service Science(ICSESS). Piscataway, NJ, USA: IEEE, 2020: 52-56.
31	Park S G, Kim D H. Autonomous Flying of Drone Based on PPO Reinforcement Learning Algorithm[J]. Journal of Institute of Control, Robotics and Systems, 2020, 26(11): 955-963.
32	Haarnoja T, Zhou A, Hartikainen K, et al. Soft Actor-critic Algorithms and Applications[EB/OL]. (2019-01-29) [2021-06-13].
33	Ng A Y, Harada D, Russell S J. Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping[C]//Proceedings of the Sixteenth International Conference on Machine Learning. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1999: 278-287.
34	Laud A D. Theory and Application of Reward Shaping in Reinforcement Learning[D]. Urbana, IL, USA: University of Illinois at Urbana-Champaign, 2004.
35	何柳柳, 杨羊, 李征, 等. 面向持续集成测试优化的强化学习奖励机制[J]. 软件学报, 2019, 30(5): 1438-1449.
	He Liuliu, Yang Yang, Li Zheng, et al. Reward of Reinforcement Learning of Test Optimization for Continuous Integration[J]. Journal of Software, 2019, 30(5): 1438-1449.

参数	取值
折扣因子	0.95
学习率	0.000 3
训练批次大小	256
每次迭代的步长	1 000
经验池大小	10⁷
温度系数	0.005
α初始值	1
梯度步数	1
探索函数	随机采样

指标	SAC	PPO
到达率/%	94	71
平均步数	342.63	301.29
平均路径长度/m	726.38	1072.93

[1]	Jiang Ming, He Tao. Solving the Vehicle Routing Problem Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(9): 2177-2187.
[2]	Yu Yiran, Lai Huicheng, Gao Guxue, Zhang Guo, Peng Wangyinan, Yang Longfei, Huang Junhao. Optimization Method for Multi Agricultural Machinery Collaborative Operation Based on Genetic Algorithm and A * Algorithm [J]. Journal of System Simulation, 2025, 37(9): 2397-2408.
[3]	Ni Peilong, Mao Pengjun, Wang Ning, Yang Mengjie. Robot Path Planning Based on Improved A-DDQN Algorithm [J]. Journal of System Simulation, 2025, 37(9): 2420-2430.
[4]	Zhang Kaixiang, Mao Jianlin, Wang Niya, Xu Zhihao. Multi-robot Hierarchical Collaborative k-robust Path Planning for Path Interference [J]. Journal of System Simulation, 2025, 37(8): 2074-2088.
[5]	Chen Zhen, Wu Zhuoyi, Zhang Lin. Research on Policy Representation in Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(7): 1753-1769.
[6]	Wan Yuhang, Zhu Zilu, Zhong Chunfu, Liu Yongkui, Lin Tingyu, Zhang Lin. Dynamic Path Planning for Robotic Arms Based on an Improved PPO Algorithm [J]. Journal of System Simulation, 2025, 37(6): 1462-1473.
[7]	Ye Chen, Shao Peng, Zhang Shaoping, Li Wenting, Zhou Tengming. Enhanced Artificial Gorilla Algorithm for Mobile Robot Path Planning [J]. Journal of System Simulation, 2025, 37(6): 1474-1485.
[8]	Zhang Yan, Li Binghua, Huo Tao, Liu Rong. Research on Robot Dynamic Obstacle Avoidance Method Based on Improved A* and Dynamic Window Algorithm [J]. Journal of System Simulation, 2025, 37(6): 1555-1564.
[9]	Wu Guohua, Zeng Jiaheng, Wang Dezhi, Zheng Long, Zou Wei. A Quadrotor Trajectory Tracking Control Method Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(5): 1169-1187.
[10]	Zhou Xiaohui, Li Yanqiang, Wang Yong, Zhao Decai, Yang Xiaoyao. Robot Path Planning Based on Ant Colony Algorithm with Dual Heuristic Information [J]. Journal of System Simulation, 2025, 37(5): 1280-1289.
[11]	Yu Die, Bao Baizhong, Si Yan, Duan Jian, Zhan Xiaobin, Shi Tielin. Mobile Robot Path Planning Based on Search-step Optimized A* Algorithm [J]. Journal of System Simulation, 2025, 37(4): 1041-1050.
[12]	Zhang Sen, Dai Qiangqiang. UAV Path Planning Based on Improved Deep Deterministic Policy Gradients [J]. Journal of System Simulation, 2025, 37(4): 875-881.
[13]	He Zhigang, Li Dayan, Wang Niya, Mao Jianlin, Wang Ning. A Multi-robot Collaborative Path Planning Algorithm with Chain Working Mode [J]. Journal of System Simulation, 2025, 37(4): 953-967.
[14]	Lin Guijuan, Li Zihan, Wang Yu. Research on Improved A* Algorithm Path Planning Based on Global Key Point Extraction [J]. Journal of System Simulation, 2025, 37(3): 667-678.
[15]	Bai Yuxin, Chen Zhenya, Shi Ruitao, Su Weitao, Ma Zhuoqiang, Yang Shangjin. Research on Robot Path Planning Based on Improved Harris Hawks Algorithm [J]. Journal of System Simulation, 2025, 37(3): 742-752.

Intelligent Path Planning for Mobile Robots Based on SAC Algorithm

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 35

Related Articles 15

Recommended Articles

Metrics

Comments