ssn

SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds

SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds

Abstract

Multi-class 3D object detection aims to localize and classify objects of multiple categories from point clouds. Due to the nature of point clouds, i.e. unstructured, sparse and noisy, some features benefit-ting multi-class discrimination are underexploited, such as shape information. In this paper, we propose a novel 3D shape signature to explore the shape information from point clouds. By incorporating operations of symmetry, convex hull and chebyshev fitting, the proposed shape sig-nature is not only compact and effective but also robust to the noise, which serves as a soft constraint to improve the feature capability of multi-class discrimination. Based on the proposed shape signature, we develop the shape signature networks (SSN) for 3D object detection, which consist of pyramid feature encoding part, shape-aware grouping heads and explicit shape encoding objective. Experiments show that the proposed method performs remarkably better than existing methods on two large-scale datasets. Furthermore, our shape signature can act as a plug-and-play component and ablation study shows its effectiveness and good scalability.

Introduction

We implement PointPillars with Shape-aware grouping heads used in the SSN and provide the results and checkpoints on the nuScenes and Lyft dataset.

Results and models

NuScenes

Backbone	Lr schd	Mem (GB)	Inf time (fps)	mAP	NDS	Download
SECFPN	2x	16.4		35.17	49.76	model | log
SSN	2x	3.6		40.91	54.44	model | log
RegNetX-400MF-SECFPN	2x	16.4		41.15	55.20	model | log
RegNetX-400MF-SSN	2x	5.1		46.65	58.24	model | log

Lyft

Backbone	Lr schd	Mem (GB)	Inf time (fps)	Private Score	Public Score	Download
SECFPN	2x	12.2		13.9	14.1	model | log
SSN	2x	8.5		17.5	17.5	model | log
RegNetX-400MF-SSN	2x	7.4		17.9	18	model | log

Note:

The main difference of the shape-aware grouping heads with the original SECOND FPN heads is that the former groups objects with similar sizes and shapes together, and design shape-specific heads for each group. Heavier heads (with more convolutions and large strides) are designed for large objects while smaller heads for small objects. Note that there may appear different feature map sizes in the outputs, so an anchor generator tailored to these feature maps is also needed in the implementation.

Users could try other settings in terms of the head design. Here we basically refer to the implementation HERE.

Citation

@inproceedings{zhu2020ssn,
  title={SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds},
  author={Zhu, Xinge and Ma, Yuexin and Wang, Tai and Xu, Yan and Shi, Jianping and Lin, Dahua},
  booktitle={Proceedings of the European Conference on Computer Vision},
  year={2020}
}