The Synchronous Grasping Method of Virtual-real Assembly Robot Based on Digital Twin

doi:10.16182/j.issn1004731x.joss.23-0575

Abstract

Abstract:

A method based on digital twin for assembly robot virtual-real synchronization and grasping is proposed to address the issues of poor intelligent grasping accuracy and difficult data processing in assembly tasks for industrial robots. Based on the digital twin, a digital twin assembly robot virtual-real synchronization and grasping architecture is designed. The OPC UA information model is built by classifying multi-source heterogeneous data, and the OPC UA communication protocol is used as a bridge for data communication of the assembly robot, achieving virtual-real synchronization. The convolutional neural network is further trained using the virtual robot to improve the grasping accuracy of the materials. The correctness and effectiveness of the proposed method are validated in the developed robot digital twin prototype system, providing a new solution for achieving digital twin virtual-real synchronization and precise grasping for assembly robots.

Key words: digital twin, assembly robot, real-time data, grab, OPC UA, virtual and real synchronization

CLC Number:

TP391.9

Xu Jian, Liu Gaofeng, Zhao Yijian, Zheng Zili, Yan Huanying. The Synchronous Grasping Method of Virtual-real Assembly Robot Based on Digital Twin[J]. Journal of System Simulation, 2024, 36(9): 2181-2192.

Figures/Tables 16

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Fig. 11

Fig. 12

Fig. 13

Table 1

Joint angle data

点位	J1	J2	J3	J4	J5	J6
$A 1$	-70.21	91.32	345.16	112.58	132.98	16.51
$A 2$	-70.12	91.27	345.08	112.47	132.84	16.42
$Δ A$	0.09	0.05	0.08	0.11	0.14	0.09
$B 1$	72.18	121.21	112.93	109.28	39.98	-72.56
$B 2$	72.09	121.08	112.87	109.19	40.05	-72.49
$Δ B$	0.09	0.13	0.06	0.09	0.07	0.07
$C 1$	140.31	-48.68	37.12	-65.89	48.75	94.87
$C 2$	140.23	-48.56	37.19	-65.74	48.63	94.93
$Δ C$	0.08	0.12	0.07	0.15	0.12	0.06

Table 1

Fig. 14

Table 2

References 20

1	Hu Fuwen. Mutual Information-enhanced Digital Twin Promotes Vision-guided Robotic Grasping[J]. Advanced Engineering Informatics, 2022, 52: 101562.
2	Kleeberger Kilian, Bormann Richard, Kraus Werner, et al. A Survey on Learning-based Robotic Grasping[J]. Current Robotics Reports, 2020, 1(4): 239-249.
3	Real E, Moore S, Selle A, et al. Large-scale Evolution of Image Classifiers[C]//Proceedings of the 34th International Conference on Machine Learning. Cambridge: JMLR, 2017: 2902-2911.
4	Akinola I, Angelova A, Lu Yao, et al. Visionary: Vision Architecture Discovery for Robot Learning[C]//2021 IEEE International Conference on Robotics and Automation (ICRA). Piscataway, NJ, USA: IEEE, 2021: 10779-10785.
5	Liu Yongkui, Xu He, Liu Ding, et al. A Digital Twin-based Sim-to-real Transfer for Deep Reinforcement Learning-enabled Industrial Robot Grasping[J]. Robotics and Computer-Integrated Manufacturing, 2022, 78: 102365.
6	Grieves M. Digital Twin: Manufacturing Excellence Through Virtual Factory Replication[J]. White paper, 2014, 1: 1-7.
7	陶飞, 刘蔚然, 张萌, 等. 数字孪生五维模型及十大领域应用[J]. 计算机集成制造系统, 2019, 25(1): 1-18.
	Tao Fei, Liu Weiran, Zhang Meng, et al. Five-dimension Digital Twin Model and Its Ten Applications[J]. Computer Integrated Manufacturing Systems, 2019, 25(1): 1-18.
8	Tipary Bence, Erdős Gábor. Generic Development Methodology for Flexible Robotic Pick-and-place Workcells Based on Digital Twin[J]. Robotics and Computer-Integrated Manufacturing, 2021, 71: 102140.
9	徐健, 宋鑫, 刘秀平, 等. 基于数字孪生的装配机器人建模及系统实现[J]. 系统仿真学报, 2023, 35(7): 1497-1507.
	Xu Jian, Song Xin, Liu Xiuping, et al. Modeling and System Realization of Assembly Robot Based on Digital Twin[J]. Journal of System Simulation, 2023, 35(7): 1497-1507.
10	刘怀兰, 赵文杰, 李世壮, 等. 数字孪生车间机器人虚实驱动系统构建方法[J]. 中国机械工程, 2022, 33(21): 2623-2632.
	Liu Huailan, Zhao Wenjie, Li Shizhuang, et al. Construction Method of Virtual-real Drive Systems for Robots in Digital Twin Workshops[J]. China Mechanical Engineering, 2022, 33(21): 2623-2632.
11	Kuts Vladimir, Otto Tauno, Tähemaa Toivo, et al. Digital Twin Based Synchronised Control and Simulation of the Industrial Robotic Cell Using Virtual Reality[J]. Journal of Machine Engineering, 2019, 19(1): 128-144.
12	王剑, 王好臣, 李学伟, 等. 基于OPC UA的数字孪生车间信息物理融合系统[J]. 现代制造工程, 2023(4): 43-50.
	Wang Jian, Wang Haochen, Li Xuewei, et al. Digital Twin Workshop Information Physical Fusion System Based on OPC UA[J]. Modern Manufacturing Engineering, 2023(4): 43-50.
13	乔峰丽, 苗鸿宾, 纪慧君, 等. 面向数字孪生的工业机械臂手眼标定方法的研究[J]. 机床与液压, 2023, 51(1): 31-35.
	Qiao Fengli, Miao Hongbin, Ji Huijun, et al. Research on Hand-eye Calibration Method for Digital Twin Oriented Industrial Manipulator[J]. Machine Tool & Hydraulics, 2023, 51(1): 31-35.
14	马喜平, 李迪, 姚侠楠, 等. 基于Tsai两步法的视觉点胶系统相机标定方法[J]. 自动化与仪表, 2018, 33(5): 1-4, 18.
	Ma Xiping, Li Di, Yao Xianan, et al. Camera Calibration of Visual Dispensing System Based on Tsai's Two-step Method[J]. Automation & Instrumentation, 2018, 33(5): 1-4, 18.
15	Kyu Tae Park, Lee Donggun, Sang Do Noh. Operation Procedures of a Work-center-level Digital Twin for Sustainable and Smart Manufacturing[J]. International Journal of Precision Engineering and Manufacturing-Green Technology, 2020, 7(3): 791-814.
16	周学广, 吕伟栋, 袁志民. 基于着色Petri网的舰艇指挥控制信息流建模研究[J]. 系统仿真学报, 2019, 31(5): 828-842.
	Zhou Xueguang, Weidong Lü, Yuan Zhimin. Colored Petri Net Based Modeling on Warship Command and Control Information Flow[J]. Journal of System Simulation, 2019, 31(5): 828-842.
17	Perzylo Alexander, Profanter Stefan, Rickert Markus, et al. OPC UA NodeSet Ontologies as a Pillar of Representing Semantic Digital Twins of Manufacturing Resources[C]//2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA). Piscataway, NJ, USA: IEEE, 2019: 1085-1092.
18	束长宝, 于照, 张继勇. 基于TCP/IP的网络通信及其应用[J]. 微计算机信息, 2006(36): 157-159.
	Shu Changbao, Yu Zhao, Zhang Jiyong. Network Communication and Its Applications Based on TCP/IP[J]. Microcomputer Information, 2006(36): 157-159.
19	Lenz I, Lee H, Saxena A. Deep Learning for Detecting Robotic Grasps[J]. International Journal of Robotics Research, 2015, 34(4/5): 705-724.
20	Wohlkinger Walter, Aldoma Aitor, Rusu R B, et al. 3DNet: Large-scale Object Class Recognition from CAD Models[C]//2012 IEEE International Conference on Robotics and Automation. Piscataway, NJ, USA: IEEE, 2012: 5384-5391.

抓取	原始CNN	数字孪生优化CNN
虚拟	0.58	0.63
真实	0.48	0.56

[1]	Lu Houjun, Zhu Yifei, Rong Yanping, Zhang Wanghui. Digital Twin Modeling Method for Bulk Cargo Stacks Based on 2D LiDAR [J]. Journal of System Simulation, 2025, 37(9): 2269-2286.
[2]	Liu Yongkui, Yang Kang, Tuo Benben, Pan Yaduo, Wang Xinyu, Wang Yihan, Gong Yongqian, Zhang Lin, Wang Lihui, Lin Tingyu, Zi Bin, Li Yuan, You Wei, Xu Xun. Digital Twinned Industrial Robot: Conceptual Framework, Key Technologies, and Case Study [J]. Journal of System Simulation, 2025, 37(7): 1723-1752.
[3]	Liu Tao, Li Hanxi, Yin Yong, Liu Jialun. Research Review of Intelligent Navigation Simulation Technology and Its Applications [J]. Journal of System Simulation, 2025, 37(7): 1684-1709.
[4]	Chen Qinghua, Liang Zuoyou, Guan Weijuan, Ji Jiadong, Liu Ping. Construction Method of Digital Twin System for High-low Temperature Test Chamber [J]. Journal of System Simulation, 2025, 37(6): 1400-1411.
[5]	Zhang Wenjia, Zhang Heming. Research on Grey-box Modeling Method of Digital Twins for Cantilever Structure [J]. Journal of System Simulation, 2025, 37(5): 1158-1168.
[6]	Zhang Huimai, Hu Xiaoya, Zhou Chunjie. Digital Twin Framework for the Generation and Optimization of Security Policies for TSN Industrial Control Systems [J]. Journal of System Simulation, 2025, 37(4): 861-874.
[7]	Jiang Lun, Wang Dajiang, Sun Wenlei, Bao Shenghui, Liu Han, Chang Saike. Research on Transformer Fault Diagnosis Method Based on Digital Twin [J]. Journal of System Simulation, 2025, 37(3): 775-790.
[8]	Hu Tianxiang, Ye Hui, Yang Xiaofei. Construction of a Digital Twin-based Ship Manufacturing Workshop Monitoring System [J]. Journal of System Simulation, 2025, 37(2): 517-528.
[9]	Zheng Jiayu, Mai Zhuxue, Chen Zheyi. Optimization of Service Caching and Computation Offloading in Digital Twin Cloud-edge Networks [J]. Journal of System Simulation, 2025, 37(11): 2741-2753.
[10]	Zhang Xiyang, Lin Xusheng, Zhou Rui, Hu Yi. Research on the Digital Twin Architecture and Application of CNC System [J]. Journal of System Simulation, 2025, 37(1): 183-198.
[11]	Wu Qinghui, BaoYaqing , Zhao Zhongxin, Huang Xu, Wei Yuchen. Research and Implementation of Digital Twin System for Mine Drainage Monitoring [J]. Journal of System Simulation, 2025, 37(1): 199-210.
[12]	Zhao Baiting, Shi Jianguo, Jia Xiaofen. Research on Digital Twin System of Rockshaft Hoist [J]. Journal of System Simulation, 2024, 36(9): 2054-2064.
[13]	Li Dongxue, Liu Yan, Shen Boyao, Jing Yongteng, Ma Qiang, Liu Ran. Carbon Footprint Analysis and Low-carbon Optimization Method Simulation Study of Power Transformer Based on Digital Twin Technology [J]. Journal of System Simulation, 2024, 36(9): 2075-2085.
[14]	Cao Yu, Li Jie, Wang Fang, Liu Zhixiang, Wang Xueliang. Digital Twin Method of Stress Field of Deep Submersible Spherical Shell Based on Simulation Database [J]. Journal of System Simulation, 2024, 36(8): 1764-1779.
[15]	Lei Zhen, Liu Yuhua, Ding Kai, Chen Haoxiang, Li Dongwei. Digital Twin-Driven Structural Thermal Deformation Compensation System for Radio Telescopes [J]. Journal of System Simulation, 2024, 36(8): 1869-1883.