基于DRL和自由步态的六足机器人运动规划研究

doi:10.16182/j.issn1004731x.joss.22-1220

系统仿真学报 ›› 2024, Vol. 36 ›› Issue (2): 373-384.doi: 10.16182/j.issn1004731x.joss.22-1220

基于DRL和自由步态的六足机器人运动规划研究

王鑫鹏¹^,³(), 傅汇乔², 邓归洲¹^,³, 唐开强²^,³(), 陈春林², 留沧海¹^,³

^1.西南科技大学制造科学与工程学院, 四川绵阳 621010
^2.南京大学工程管理学院, 江苏南京 210046
^3.制造过程测试技术省部共建教育部重点实验室, 四川绵阳 621000

收稿日期:2022-10-14 修回日期:2022-11-07 出版日期:2024-02-15 发布日期:2024-02-04
通讯作者: 唐开强 E-mail:xpwang@mails.swust.edu.cn;kqtang@smail.nju.edu.cn
第一作者简介:王鑫鹏(1995-)，男，硕士生，研究方向为深度强化学习和机器人技术。E-mail：xpwang@mails.swust.edu.cn
基金资助:
国家自然科学基金(62073160);国家重点研发计划(72018AAA0101100)

Research on Motion Planning of Hexapod Robot Based on DRL and Free Gait

Wang Xinpeng¹^,³(), Fu Huiqiao², Deng Guizhou¹^,³, Tang Kaiqiang²^,³(), Chen Chunlin², Liu Canghai¹^,³

^1.School of Manufacturing Science and Engineering, Southwest University of Science and Technology, Mianyang 621010, China
^2.School of Management and Engineering, Nanjing university, Nanjing 210046, China
^3.Manufacturing Process Testing Technology Key Laboratory of the Ministry of Education, Mianyang 621000, China

Received:2022-10-14 Revised:2022-11-07 Online:2024-02-15 Published:2024-02-04
Contact: Tang Kaiqiang E-mail:xpwang@mails.swust.edu.cn;kqtang@smail.nju.edu.cn

摘要/Abstract

摘要：

为提高六足机器人在非结构环境下的通过率和运动性能，提出一种基于DRL和自由步态规划器的多接触运动规划算法。自由步态规划器获取目标状态下可达落足点从而输出最优步态序列；利用DRL训练得到六足机器人在随机生成的梅花桩环境中的质心运动策略。为了保证机器人在运动过程中相邻状态之间的可达性，利用状态转移可行性模型对状态转移可行性进行判定，实现六足机器人在不同宽度沟壑梅花桩环境下的落脚点规划。仿真与样机实验表明：多接触运动规划算法能够让机器人快速平稳地从起点到达目标区域，并自动调整步态模式以应对不同环境下随机分布的梅花桩。

关键词: 六足机器人, 自由步态, 深度强化学习, 多接触运动规划, 非结构环境

Abstract:

To improve the passability and the motion performance of the hexapod robot in the unstructured environment, a multi-contact motion planning algorithm based on DRL and free gait planner is proposed. Firstly, the free gait planner obtains the reachable footholds under the target state and outputs the optimal gait sequence. The center of mass motion policy of the hexapod robot in the randomly generated plum blossom pile environment is obtained by using deep reinforcement learning training. To ensure the reachability between adjacent states of the robot in motion, the state transition feasibility model is used to judge the state transition feasibility. Finally, the foothold planning of the hexapod robot in the plum blossom pile environment with gullies of different widths is realized. Simulation and physical experiments show that the multi-contact motion planning algorithm can make the robot reach the target area quickly and smoothly from the starting point, and automatically adjust the gait pattern to deal with the randomly distributed plum blossom piles in different environments.

Key words: hexapod robot, free gait, DRL, multi-contact motion planning, unstructured environment

中图分类号:

TP242.6

王鑫鹏,傅汇乔,邓归洲等 . 基于DRL和自由步态的六足机器人运动规划研究[J]. 系统仿真学报, 2024, 36(2): 373-384.

Wang Xinpeng,Fu Huiqiao,Deng Guizhou,et al . Research on Motion Planning of Hexapod Robot Based on DRL and Free Gait[J]. Journal of System Simulation, 2024, 36(2): 373-384.

图/表 18

图1

表1

表2

图2

图3

图4

图5

图6

图7

表3

图8

图9

表4

图10

图11

图12

图13

图14

参考文献 23

1	Wu Jianxu, Yang Hui, Li Ruiming, et al. Design and Analysis of a Novel Octopod Platform with a Reconfigurable Trunk[J]. Mechanism and Machine Theory, 2021, 156: 104134.
2	石琳, 罗庆生, 韩宝玲, 等. 仿生六足机器人稳定性的仿生实验研究[J]. 系统仿真学报, 2008, 20(19): 5384-5387.
	Shi Lin, Luo Qingsheng, Han Baoling, et al. Research on Biomimetic Experiment of Stability of Hexapod Robot[J]. Journal of System Simulation, 2008, 20(19): 5384-5387.
3	Hwangbo Jemin, Lee Joonho, Dosovitskiy Alexey, et al. Learning Agile and Dynamic Motor Skills for Legged Robots[J]. Science Robotics, 2019, 4(26): 5872.
4	Stelzer Annett, Hirschmüller Heiko, Görner Martin. Stereo-vision-based Navigation of a Six-legged Walking Robot in Unknown Rough Terrain[J]. The International Journal of Robotics Research, 2012, 31(4): 381-402.
5	李满宏, 张明路, 张建华, 等. 六足机器人关键技术综述[J]. 机械设计, 2015, 32(10): 1-8.
	Li Manhong, Zhang Minglu, Zhang Jianhua, et al. Review on Key Technology of the Hexapod Robot[J]. Journal of Machine Design, 2015, 32(10): 1-8.
6	Belter Dominik, Wietrzykowski Jan, Skrzypczyński Piotr. Employing Natural Terrain Semantics in Motion Planning for a Multi-legged Robot[J]. Journal of Intelligent & Robotic Systems, 2019, 93(3): 723-743.
7	Estremera J, Cobano J A, Gonzalez de Santos P. Continuous Free-crab Gaits for Hexapod Robots on a Natural Terrain with Forbidden Zones: An Application to Humanitarian Demining[J]. Robotics and Autonomous Systems, 2010, 58(5): 700-711.
8	Estremera Joaquin, Pablo Gonzalez de Santos. Free Gaits for Quadruped Robots over Irregular Terrain[J]. The International Journal of Robotics Research, 2002, 21(2): 115-130.
9	Chen Zhijun, Liu Jimu, Gao Feng. Real-time Gait Planning Method for Six-legged Robots to Optimize the Performances of Terrain Adaptability and Walking Speed[J]. Mechanism and Machine Theory, 2022, 168: 104545.
10	Chen Xi, Wang Liquan, Ye Xiufen, et al. Prototype Development and Gait Planning of Biologically Inspired Multi-legged Crablike Robot[J]. Mechatronics, 2013, 23(4): 429-444.
11	Xu Peng, Ding Liang, Wang Zhikai, et al. Contact Sequence Planning for Hexapod Robots in Sparse Foothold Environment Based on Monte-carlo Tree[J]. IEEE Robotics and Automation Letters, 2022, 7(2): 826-833.
12	Mustafa Suphi Erden, Leblebicioğlu Kemal. Free Gait Generation with Reinforcement Learning for a Six-legged Robot[J]. Robotics and Autonomous Systems, 2008, 56(3): 199-212.
13	Chen Zhihua, Li Jiehao, Wang Shoukun, et al. Flexible Gait Transition for Six Wheel-legged Robot with Unstructured Terrains[J]. Robotics and Autonomous Systems, 2022, 150: 103989.
14	Coelho Joana, Ribeiro Fernando, Dias Bruno, et al. Trends in the Control of Hexapod Robots: A Survey[J]. Robotics, 2021, 10(3): 100.
15	Mnih V, Kavukcuoglu K, Silver D, et al. Human-level Control Through Deep Reinforcement Learning[J]. Nature, 2015, 518(7540): 529-533.
16	傅汇乔, 唐开强, 邓归洲, 等. 基于深度强化学习的六足机器人运动规划[J]. 智能科学与技术学报, 2020, 2(4): 361-371.
	Fu Huiqiao, Tang Kaiqiang, Deng Guizhou, et al. Motion Planning for Hexapod Robot Using Deep Reinforcement Learning[J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(4): 361-371.
17	董星宇, 唐开强, 傅汇乔, 等. 基于强化学习的六足机器人动态避障研究[J]. 传感器与微系统, 2022, 41(1): 19-23.
	Dong Xingyu, Tang Kaiqiang, Fu Huiqiao, et al. Study on Dynamic Obstacle Avoidance for Hexapod Robot Based on Reinforcement Learning[J]. Transducer and Microsystem Technologies, 2022, 41(1): 19-23.
18	Fu Huiqiao, Tang Kaiqiang, Li Peng, et al. Deep Reinforcement Learning for Multi-contact Motion Planning of Hexapod Robots[C]//Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. California: IJCAI, 2021: 2381-2388.
19	Tsounis Vassilios, Alge Mitja, Lee Joonho, et al. DeepGait: Planning and Control of Quadrupedal Gaits Using Deep Reinforcement Learning[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 3699-3706.
20	Winkler A W. Optimization-based Motion Planning for Legged Robots[D]. Zurich: ETH Zurich, 2018.
21	Haarnoja T, Zhou A, Abbeel P, et al. Soft Actor-critic: Off-policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor[C]//Proceedings of the 35th International Conference on Machine Learning. Chia Laguna Resort, Sardinia, Italy: PMLR, 2018: 1861-1870.
22	In Kim Jae, Hong Mineui, Lee Kyungjae, et al. Learning to Walk a Tripod Mobile Robot Using Nonlinear Soft Vibration Actuators with Entropy Adaptive Reinforcement Learning[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 2317-2324.
23	孙辉辉, 胡春鹤, 张军国. 移动机器人运动规划中的深度强化学习方法[J]. 控制与决策, 2021, 36(6): 1281-1292.
	Sun Huihui, Hu Chunhe, Zhang Junguo. Deep Reinforcement Learning for Motion Planning of Mobile Robots[J]. Control and Decision, 2021, 36(6): 1281-1292.

i	α_i /(°)	a_i	θ_i
1	90	l₁	θ₁
2	0	l₂	θ₂
3	0	l₃	θ₃

参数	基节	髋关节	膝关节
长度/mm	60	120	145.8
转角范围/(°)	[-45, 45]	[0, 45]	[-135, -90]

环境	平均回合步数	平均回合成功率/%
I	25	100
II	31	98
III	64	52

[1]	赵莹莹, 董普森, 朱天晨, 李凡, 苏运, 邰振赢, 孙庆赟, 凡航. 面向电网拓扑调度仿真的采样效率优化方法研究[J]. 系统仿真学报, 2024, 36(2): 283-295.
[2]	潘海南, 陈柏良, 黄开宏, 任君凯, 程创, 卢惠民, 张辉. 基于深度强化学习的履带机器人摆臂控制方法[J]. 系统仿真学报, 2024, 36(2): 405-414.
[3]	安靖, 司光亚, 张雷. 基于深度强化学习的立体投送策略优化方法研究[J]. 系统仿真学报, 2024, 36(1): 39-49.
[4]	林俊强, 王红军, 邹湘军, 张坡, 李承恩, 周益鹏, 姚书杰. 基于DPPO的移动采摘机器人避障路径规划及仿真[J]. 系统仿真学报, 2023, 35(8): 1692-1704.
[5]	刘家义, 王刚, 付强, 郭相科, 王思远. 基于分配策略优化算法的智能防空任务分配[J]. 系统仿真学报, 2023, 35(8): 1705-1716.
[6]	杨来义, 毕敬, 苑海涛. 基于SAC算法的移动机器人智能路径规划[J]. 系统仿真学报, 2023, 35(8): 1726-1736.
[7]	丁飞, 沙宇晨, 洪莹, 蒯晓, 张登银. 智能网联汽车计算卸载与边缘缓存联合优化策略[J]. 系统仿真学报, 2023, 35(6): 1203-1214.
[8]	戴宇轩, 崔承刚. 基于深度强化学习的Boost变换器控制策略[J]. 系统仿真学报, 2023, 35(5): 1109-1119.
[9]	徐浩添, 秦龙, 曾俊杰, 胡越, 张琪. 基于深度强化学习的对手建模方法研究综述[J]. 系统仿真学报, 2023, 35(4): 671-694.
[10]	史佳洁, 杨鹏, 皮雁南. 基于机器学习的地铁行人流在线优化控制研究[J]. 系统仿真学报, 2023, 35(2): 386-395.
[11]	琚翔, 苏圣超, 徐超杰, 何蓓蓓. 边缘计算下基于深度强化学习的车联网任务调度[J]. 系统仿真学报, 2023, 35(12): 2550-2559.
[12]	王宇琨, 王泽, 董力维, 李妮. 基于分层的智能建模方法的多机空战行为建模[J]. 系统仿真学报, 2023, 35(10): 2249-2261.
[13]	张森, 张孟炎, 邵敬平, 普杰信. 基于随机策略搜索的多机三维路径规划方法[J]. 系统仿真学报, 2022, 34(6): 1286-1295.
[14]	倪凌佳, 黄晓霞, 李红旮, 张子博. 基于协作式深度强化学习的火灾应急疏散仿真研究[J]. 系统仿真学报, 2022, 34(6): 1353-1366.
[15]	王红微, 杨鹏. 基于深度强化学习的机场货运业务优化研究[J]. 系统仿真学报, 2022, 34(3): 651-660.

基于DRL和自由步态的六足机器人运动规划研究

Research on Motion Planning of Hexapod Robot Based on DRL and Free Gait

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 18

参考文献 23

相关文章 15

编辑推荐

Metrics

本文评价