系统仿真学报 ›› 2024, Vol. 36 ›› Issue (12): 2971-2983.doi: 10.16182/j.issn1004731x.joss.23-1233

• 论文 • 上一篇    

融合强化学习的工业机器人数字孪生仿真方法研究

缪天越, 王璐, 何家孝, 谢能刚   

  1. 安徽工业大学 机械工程学院,安徽 马鞍山 243032
  • 收稿日期:2023-10-12 修回日期:2023-11-12 出版日期:2024-12-20 发布日期:2024-12-20
  • 通讯作者: 王璐
  • 第一作者简介:缪天越(1998-),男,硕士生,研究方向为数字孪生、工业机器人。
  • 基金资助:
    安徽省自然科学基金(2108085MG237)

Research on Digital Twin Simulation Method of Industrial Robot Integrated with Reinforcement Learning

Miao Tianyue, Wang Lu, He Jiaxiao, Xie Nenggang   

  1. School of Mechanical Engineering, Anhui University of Technology, Maanshan 243032, China
  • Received:2023-10-12 Revised:2023-11-12 Online:2024-12-20 Published:2024-12-20
  • Contact: Wang Lu

摘要:

针对工业机器人领域构建的数字孪生系统功能不全面、应用场景较为单一带来的通用性不高的问题,提出一种具有较高通用性的工业机器人数字孪生系统的构建方法。设计了数字孪生四维系统架构,对四维系统各部分组成及作用进行分析,并基于四维系统规划系统等级,定义了融合强化学习的虚替实概念;通过构建多属性虚拟模型,利用TCP通信协议搭建数据通信系统进行虚实数据交互,结合机器人正逆运动学分析,实现虚实映射与控制功能;构建了强化学习虚拟场景,使用虚拟机器人模型代替实体机器人进行强化学习训练,实现自动规划路径功能。实验结果验证了该系统的可行性和可靠性,为进一步丰富工业机器人数字孪生系统功能提供了新方案。

关键词: 数字孪生, 工业机器人, 强化学习, 四维模型, 虚实映射

Abstract:

In response to the lack of comprehensive functionality and limited application scenarios in the current field of industrial robot digital twin systems, which results in low versatility, a method for constructing a digital twin system for industrial robots with high versatility is proposed. A four-dimensional system architecture for the digital twin is designed, and the components and functions of the four-dimensional system are analyzed, based on the system level planning of the four-dimensional system, the concept of integrating reinforcement learning into the virtual replacement of real concept is defined. By constructing a multi-attribute virtual model and using TCP communication protocol to build a data communication system for virtual-real data interaction, combined with robot forward and inverse kinematic analysis, the virtual-real mapping and control functions are achieved. A reinforcement learning virtual scene is constructed, using a virtual robot model to replace the physical robot for reinforcement learning training, to achieve automatic path planning functionality. The experimental results verify the feasibility and reliability of the developed digital twin system, providing a new solution for further enriching the functionality of industrial robot digital twin systems.

Key words: digital twin, industrial robots, reinforcement learning, four-dimensional model, virtual-real mapping

中图分类号: