Format

Send to

Choose Destination
Sensors (Basel). 2019 Mar 23;19(6). pii: E1434. doi: 10.3390/s19061434.

One-Stage Multi-Sensor Data Fusion Convolutional Neural Network for 3D Object Detection.

Li M1, Hu Y2,3, Zhao N4,5, Qian Q6.

Author information

1
State Key Laboratory of Pulsed Power Laser Technology, College of Electronic Engineering (National University of Defence Technology), Hefei 230037, China. 15555483329@163.com.
2
State Key Laboratory of Pulsed Power Laser Technology, College of Electronic Engineering (National University of Defence Technology), Hefei 230037, China. skl_hyh@163.com.
3
Anhui Province Key Laboratory of Electronic Restriction Technology, Hefei 230037, China. skl_hyh@163.com.
4
State Key Laboratory of Pulsed Power Laser Technology, College of Electronic Engineering (National University of Defence Technology), Hefei 230037, China. southfly@163.com.
5
Anhui Province Key Laboratory of Electronic Restriction Technology, Hefei 230037, China. southfly@163.com.
6
State Key Laboratory of Pulsed Power Laser Technology, College of Electronic Engineering (National University of Defence Technology), Hefei 230037, China. creamy777@163.com.

Abstract

Three-dimensional (3D) object detection has important applications in robotics, automatic loading, automatic driving and other scenarios. With the improvement of devices, people can collect multi-sensor/multimodal data from a variety of sensors such as Lidar and cameras. In order to make full use of various information advantages and improve the performance of object detection, we proposed a Complex-Retina network, a convolution neural network for 3D object detection based on multi-sensor data fusion. Firstly, a unified architecture with two feature extraction networks was designed, and the feature extraction of point clouds and images from different sensors realized synchronously. Then, we set a series of 3D anchors and projected them to the feature maps, which were cropped into 2D anchors with the same size and fused together. Finally, the object classification and 3D bounding box regression were carried out on the multipath of fully connected layers. The proposed network is a one-stage convolution neural network, which achieves the balance between the accuracy and speed of object detection. The experiments on KITTI datasets show that the proposed network is superior to the contrast algorithms in average precision (AP) and time consumption, which shows the effectiveness of the proposed network.

KEYWORDS:

convolutional neural network; multi-sensor; object detection; point cloud

Supplemental Content

Full text links

Icon for Multidisciplinary Digital Publishing Institute (MDPI) Icon for PubMed Central
Loading ...
Support Center