Annotation-free 6-DoF Grasp Detection Method Integrating Physical and Geometric Priors

doi:10.16182/j.issn1004731x.joss.25-0502

Abstract

Abstract:

To improve the stability and cross-category generalization capability of grasp pose estimation in complex stacked scenes, an annotation-free 6-DoF grasp detection method integrating physical rules and geometric structure priors was proposed. In the offline stage, a template library of feasible grasp poses was constructed based on multi-physical constraints, without relying on manual grasp annotations. In the network design, the modeling of structural symmetry of objects and spatial overlap relationships was introduced; a geometric guidance mechanism with occlusion perception and exposure modeling capabilities was designed, and robust pose alignment of target objects was achieved by combining keypoint regression. A multi-type stacked dataset IPA-Stack++ was constructed, and tests were conducted in grasping tasks of single-category and mixed-category industrial parts. Experimental results show that the grasp success rate of the proposed method in single-type and multi-type scenes is significantly superior to other existing methods of the same type, demonstrating good grasp stability and versatility.

Key words: 6-DoF grasp detection, annotation-free learning, physical rule constraint, geometric structure prior, multi-task point cloud perception

CLC Number:

TP391.9

Shi Min, Guo Shisheng, Wang Suqin, Li Zhaoxin, Zhu Dengming. Annotation-free 6-DoF Grasp Detection Method Integrating Physical and Geometric Priors[J]. Journal of System Simulation, 2026, 38(5): 1290-1302.

Figures/Tables 16

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 1

Table 2

Table 3

Table 4

Table 5

Table 6

Table 7

Fig. 9

References 22

[1]	徐健, 刘高峰, 赵一剑, 等. 装配机器人的数字孪生虚实同步及抓取方法[J]. 系统仿真学报, 2024, 36(9): 2181-2192.
	Xu Jian, Liu Gaofeng, Zhao Yijian, et al. The Synchronous Grasping Method of Virtual-real Assembly Robot Based on Digital Twin[J]. Journal of System Simulation, 2024, 36(9): 2181-2192.
[2]	Zhang Jing, Tao Dacheng. Empowering Things with Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things[J]. IEEE Internet of Things Journal, 2021, 8(10): 7789-7817.
[3]	Wang Shengfan, Jiang Xin, Zhao Jie, et al. Efficient Fully Convolution Neural Network for Generating Pixel Wise Robotic Grasps with High Resolution Images[C]//2019 IEEE International Conference on Robotics and Biomimetics (ROBIO). Piscataway: IEEE, 2019: 474-480.
[4]	Mahler J, Liang J, Niyaz S, et al. Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics[EB/OL]. (2017-08-08) [2025-03-24]. .
[5]	Yu Sheng, Zhai Dihua, Xia Yuanqing, et al. SE-ResUNet: A Novel Robotic Grasp Detection Method[J]. IEEE Robotics and Automation Letters, 2022, 7(2): 5238-5245.
[6]	Morrison Douglas, Corke Peter, Leitner Jürgen. Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach[EB/OL]. (2018-05-15) [2025-02-14]. .
[7]	Kaynar Furkan, Rajagopalan Sudarshan, Zhou Shaobo, et al. Remote Task-oriented Grasp Area Teaching by Non-experts Through Interactive Segmentation and Few-shot Learning[EB/OL]. (2023-03-17) [2025-03-24]. .
[8]	Liu Jirong, Zhang Ruo, Fang Haoshu, et al. Target-referenced Reactive Grasping for Dynamic Objects[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2023: 8824-8833.
[9]	Fang Haoshu, Wang Chenxi, Gou Minghao, et al. GraspNet-1Billion: A Large-scale Benchmark for General Object Grasping[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 11441-11450.
[10]	Zhao Binglei, Zhang Hanbo, Lan Xuguang, et al. REGNet: REgion-based Grasp Network for End-to-end Grasp Detection in Point Clouds[C]//2021 IEEE International Conference on Robotics and Automation (ICRA). Piscataway: IEEE, 2021: 13474-13480.
[11]	Ren Guangli, Geng Wenjie, Guan Peiyu, et al. Pixel-wise Grasp Detection via Twin Deconvolution and Multi-dimensional Attention[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(8): 4002-4010.
[12]	Liang Hongzhuo, Ma Xiaojian, Li Shuang, et al. PointNetGPD: Detecting Grasp Configurations from Point Sets[C]//2019 International Conference on Robotics and Automation (ICRA). Piscataway: IEEE, 2019: 3629-3635.
[13]	Qin Yuzhe, Chen Rui, Zhu Hao, et al. S4G: Amodal Single-view Single-shot SE(3) Grasp Detection in Cluttered Scenes[C]//Proceedings of the Conference on Robot Learning. Chia Laguna Resort: PMLR, 2020: 53-65.
[14]	Wang Chenxi, Fang Haoshu, Gou Minghao, et al. Graspness Discovery in Clutters for Fast and Accurate Grasp Detection[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2021: 15944-15953.
[15]	徐胜军, 崔智伟, 史亚, 等. 多尺度区域注意力堆叠物体抓取检测网络[J]. 激光与光电子学进展, 2025, 62(10): 224-235.
	Xu Shengjun, Cui Zhiwei, Shi Ya, et al. Regional-attention Stacked-object Grasp Detection Network[J]. Laser & Optoelectronics Progress, 2025, 62(10): 224-235.
[16]	Shi Min, Hou Jingzhao, Li Zhaoxin, et al. NG-net: No-grasp Annotation Grasp Detection Network for Stacked Scenes[J]. Journal of Intelligent Manufacturing, 2025, 36(2): 1477-1490.
[17]	石敏, 侯京召, 朱登明, 等. 无抓取标注的堆叠物体抓取位姿检测方法[J]. 计算机辅助设计与图形学学报, 2025, 37(9): 1632-1642.
	Shi Min, Hou Jingzhao, Zhu Dengming, et al. No-label Stacked Object Grasp Pose Detection Algorithm[J]. Journal of Computer-Aided Design & Computer Graphics, 2025, 37(9): 1632-1642.
[18]	Huang Dingtao, Lin Ente, Chen Lipeng, et al. SD-net: Symmetric-aware Keypoint Prediction and Domain Adaptation for 6D Pose Estimation in Bin-picking Scenarios[C]//2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway: IEEE, 2024: 2747-2754.
[19]	Dong Zhikai, Liu Sicheng, Zhou Tao, et al. PPR-net: Point-wise Pose Regression Network for Instance Segmentation and 6D Pose Estimation in Bin-picking Scenarios[C]//2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway: IEEE, 2019: 1773-1780.
[20]	Zeng Long, Weijie Lü, Dong Zhikai, et al. PPR-Net++: Accurate 6-D Pose Estimation in Stacked Scenarios[J]. IEEE Transactions on Automation Science and Engineering, 2022, 19(4): 3139-3151.
[21]	Kleeberger Kilian, Landgraf Christian, Huber Marco F. Large-scale 6D Object Pose Estimation Dataset for Industrial Bin-picking[C]//2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway: IEEE, 2019: 2573-2578.
[22]	楚红雨, 冷齐齐, 张晓强, 等. 融入注意力机制的多模特征机械臂抓取位姿检测[J]. 控制与决策, 2024, 39(3): 777-785.
	Chu Hongyu, Leng Qiqi, Zhang Xiaoqiang, et al. Multi-modal Feature Robotic Arm Grasping Pose Detection with Attention Mechanism[J]. Control and Decision, 2024, 39(3): 777-785.

方法	Pepper	Tless20	Tless29	Gear	GearsSaft
PointNetGPD	84	75	74	80	82
NG-Net	98	80	96	93	97
No-Label^[17]	99	91	94	98	98
本文算法	99	98	99	99	99

方法	Pepper	Tless20	Tless29	Gear
PointNetGPD	72	64	52	56
NG-Net	92	80	96	88
No-Label	80	76	88	82
本文算法	98	96	98	98

方法	单类型场景下					多类型场景下
方法	Pepper	Tless20	Tless29	Gear	Gearshaft	多类型场景下
PointNetGPD	3.381	4.862	4.415	4.363	3.553	9.080
No-Label	1.083	1.407	1.435	3.655	1.222	3.516
本文算法	1.071	1.464	1.481	3.653	0.951	3.371

抓取模板数量	单类型场景下					多类型场景下
抓取模板数量	Pepper	Tless20	Tless29	Gear	Gearshaft	多类型场景下
180	0.123 5	0.127 2	0.215 5	0.141 5	0.146 6	0.171 3
200	0.158 2	0.137 3	0.237 0	0.167 0	0.158 6	0.189 9
220	0.162 6	0.156 6	0.258 5	0.179 9	0.193 5	0.211 4

场景	方法	Pepper	Tless20	Tless29	Gear	Gearshaft
单类型	基于关键点	99	98	99	99	99
单类型	直接回归	96	95	94	97	98
多类型	基于关键点	98	96	98	98
多类型	直接回归	94	94	92	96