Research on 3D Object Detection Method with Cross-module Attention

doi:10.16182/j.issn1004731x.joss.23-FZ0843E

Abstract

Abstract:

To address the issue of feature loss that occurs during the extraction and transmission of target features in 3D object detection tasks using point cloud data， this study proposes an object detection method based on cross-module attention. This method incorporates a channel attention module and a spatial attention module to enhance the crucial feature information. Through feature transformation， the features from different stages of the attention module are connected to mitigate the loss of features during the extraction and transmission process. To tackle the problem of inadequate detection performance in target detection networks for objects of different scales， a cross-scale feature extraction and fusion method is introduced. This method enhances the network’s ability to acquire multilevel features by employing multi-scale feature extraction and fusion techniques. Experimental results demonstrate that the proposed method achieves state-of-the-art performance while maintaining a real-time inference speed of 33 Hz.

Key words: 3D object detection, voxel-based network, attention module, multi-scale feature

CLC Number:

TP391

Xu Renjie, Zhang Xiaoming, Wang Chen, Wu Peng. Research on 3D Object Detection Method with Cross-module Attention[J]. Journal of System Simulation, 2023, 35(12): 2680-2691.

Figures/Tables 10

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Table 2

Fig. 5

Fig. 6

Table 3

Table 4

References 22

1	Lang A H, Vora S, Caesar H, et al. PointPillars: Fast Encoders for Object Detection from Point Clouds[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2019: 12689-12697.
2	Yan Yan, Mao Yuxing, Li Bo. SECOND: Sparsely Embedded Convolutional Detection[J]. Sensors, 2018, 18(10): 3337.
3	Zhou Yin, Tuzel O. VoxelNet: End-to-end Learning for Point Cloud Based 3D Object Detection[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2018: 4490-4499.
4	He Chenhang, Zeng Hui, Huang Jianqiang, et al. Structure Aware Single-stage 3D Object Detection from Point Cloud[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2020: 11870-11879.
5	Simon M, Milz S, Amende K, et al. Complex-YOLO: An Euler-region-proposal for Real-time 3D Object Detection on Point Clouds[C]//Computer Vision-ECCV 2018 Workshops. Cham: Springer International Publishing, 2019: 197-209.
6	Yang Bin, Luo Wenjie, Urtasun R. PIXOR: Real-time 3D Object Detection from Point Clouds[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2018: 7652-7660.
7	Ye Maosheng, Xu Shuangjie, Cao Tongyi. HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2020: 1628-1637.
8	Shi Shaoshuai, Guo Chaoxu, Jiang Li, et al. PV-RCNN: Point-voxel Feature Set Abstraction for 3D Object Detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2020: 10526-10535.
9	Chen Xiaozhi, Ma Huimin, Wan Ji, et al. Multi-view 3D Object Detection Network for Autonomous Driving[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2017: 6526-6534.
10	Engelcke M, Rao D, Wang D Z, et al. Vote3Deep: Fast Object Detection in 3D Point Clouds Using Efficient Convolutional Neural Networks[C]//2017 IEEE International Conference on Robotics and Automation (ICRA). Piscataway, NJ, USA: IEEE, 2017: 1355-1361.
11	Li Bo. 3D Fully Convolutional Network for Vehicle Detection in Point Cloud[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway, NJ, USA: IEEE, 2017: 1513-1518.
12	Qi C R, Liu Wei, Wu Chenxia, et al. Frustum PointNets for 3D Object Detection from RGB-D Data[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2018: 918-927.
13	Shi Shaoshuai, Wang Xiaogang, Li Hongsheng. PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2019: 770-779.
14	Wang Zhixin, Jia Kui. Frustum ConvNet: Sliding Frustums to Aggregate Local Point-wise Features for Amodal 3D Object Detection[C]//2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway, NJ, USA: IEEE, 2019: 1742-1749.
15	Yang Zetong, Sun Yanan, Liu Shu, et al. 3DSSD: Point-based 3D Single Stage Object Detector[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2020: 11037-11045.
16	Yang Zetong, Sun Yanan, Liu Shu, et al. STD: Sparse-to-dense 3D Object Detector for Point Cloud[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2019: 1951-1960.
17	Qi C R, Su Hao, Mo Kaichun, et al. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2017: 77-85.
18	Qi C R, Yi Li, Su Hao, et al. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2017: 5105-5114.
19	Kuang Hongwu, Wang Bei, An Jianping, et al. Voxel-FPN: Multi-scale Voxel Feature Aggregation for 3D Object Detection from LIDAR Point Clouds[J]. Sensors, 2020, 20(3): 704.
20	Lin T Y, Dollár Piotr, Girshick R, et al. Feature Pyramid Networks for Object Detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2017: 936-944.
21	Geiger A, Lenz P, Urtasun R. Are We Ready for Autonomous Driving? The KITTI Vision Benchmark Suite[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2012: 3354-3361.
22	Ku J, Mozifian M, Lee J, et al. Joint 3D Proposal Generation and Object Detection from View Aggregation[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway, NJ, USA: IEEE, 2018: 1-8.

Method	t/s	Car			Pedestrian			Cyclists
Method	t/s	Easy	Moderate	Hard	Easy	Moderate	Hard	Easy	Moderate	Hard
F-PointNet ^[12]	0.17	88.70	84.00	75.33	58.09	50.22	47.20	75.38	61.96	54.68
AVOD ^[22]	0.08	86.80	85.44	77.73	42.51	35.24	33.97	63.66	47.74	46.55
MV3D ^[9]	0.24	85.82	77.00	68.94	N/A	N/A	N/A	N/A	N/A	N/A
SECOND ^[2]	0.05	88.07	79.37	77.95	55.10	46.27	44.76	73.67	56.04	48.78
VoxelNet ^[3]	0.23	89.35	79.26	77.39	46.13	40.74	38.11	66.70	54.76	50.55
ours	0.03	89.99	87.39	85.79	61.07	55.26	51.09	84.87	70.55	66.74

Method	t/s	Car			Pedestrian			Cyclists
Method	t/s	Easy	Moderate	Hard	Easy	Moderate	Hard	Easy	Moderate	Hard
F-PointNet ^[12]	0.17	81.20	70.39	62.19	51.21	44.89	40.23	71.96	56.77	50.39
AVOD ^[22]	0.08	73.59	65.78	58.38	38.28	31.51	26.98	60.11	44.90	38.80
MV3D ^[9]	0.24	66.77	52.73	51.31	N/A	N/A	N/A	N/A	N/A	N/A
SECOND ^[2]	0.05	83.13	73.66	66.20	51.07	42.56	37.29	70.51	53.85	46.90
VoxelNet ^[3]	0.23	77.47	65.11	57.73	39.48	33.69	31.51	61.22	48.36	44.37
ours	0.03	87.19	77.45	75.67	54.94	49.61	45.56	80.34	64.83	62.05

A1	A2	A3	A4	MSF	CMA	Easy	Moderate	Hard
						89.96	87.07	79.66
	√	√				89.82	87.10	84.45
	√	√	√			89.15	86.71	84.90
				√		89.75	87.23	82.54
	√	√	√		√	89.51	86.99	84.27
		√	√	√	√	89.70	87.17	85.60
	√	√	√	√	√	89.99	87.39	85.79
√	√	√	√	√	√	89.73	87.25	85.49

A1	A2	A3	A4	MSF	CMA	Easy	Moderate	Hard
						82.51	66.15	61.28
	√	√				79.25	68.32	64.53
	√	√	√			82.82	70.36	66.22
	√	√	√		√	80.77	67.07	62.00
				√		83.12	68.51	63.34
		√	√	√	√	83.44	68.07	63.58
	√	√	√	√	√	84.87	70.55	66.75
√	√	√	√	√	√	82.16	68.62	64.04

[1]	Hao Yu, Jinxia Jiang, Xiaohan Lai, Feng Mei, Qing Wang. Surface Defect Detection of Power Equipment Using Adaptive Receptive Field Network [J]. Journal of System Simulation, 2023, 35(7): 1572-1580.
[2]	Yu Fu, Yao Zhang, Meng Zhao, Mianzhao Wang, Jiangpeng Zheng, Chen Jia, Shengyong Chen. Fixed-Wing UAV Detection Based on Simulated Data Transfer Learning [J]. Journal of System Simulation, 2023, 35(5): 998-1007.
[3]	Shiqi Lin, Jikai Wang, Haoyuan Pei, Hao Zhao, Zonghai Chen. Monocular Semantic SLAM Method Based on Object Relation Description [J]. Journal of System Simulation, 2022, 34(2): 278-284.