会员服务 ·

95 FPS！超快速3D目标检测网络开源了！SFA3D：基于LiDAR的实时、准确的3D目标检测模型

2020 年 11 月 14 日 CVer

点击上方“CVer”，选择加"星标"置顶

重磅干货，第一时间送达

前言

Amusi 发现了一个超快速3D目标检测网络！SFA3D：基于LiDAR的实时、准确的3D目标检测模型，在GTX 1080 Ti上速度高达95 FPS！代码现已开源！

主要特性：

1. 快速训练和推理；

2. Anchor-free的方法；

3. 无NMS；

4. 支持分布式数据并行训练演示Demo详见视频

项目链接：https://github.com/maudzung/SFA3D

点击下面该视频，皆可查看演示Demo

Super Fast and Accurate 3D Object Detection based on 3D LiDAR Point Clouds

Features

Super fast and accurate 3D object detection based on LiDAR
Fast training, fast inference
An Anchor-free approach
No Non-Max-Suppression
Support distributed data parallel training
Release pre-trained models

The technical details are described here

Update 2020.09.06: Add ROS source code. The great work has been done by @AhmedARadwan. The implementation is here

Demonstration (on a single GTX 1080Ti)

2. Getting Started

2.1. Requirement

The instructions for setting up a virtual environment is here.

git clone https://github.com/maudzung/SFA3D.git SFA3D
cd SFA3D/
pip install .

2.2. Data Preparation

Download the 3D KITTI detection dataset from here.

The downloaded data includes:

Velodyne point clouds (29 GB)
Training labels of object data set (5 MB)
Camera calibration matrices of object data set (16 MB)
Left color images of object data set (12 GB) (For visualization purpose only)

Please make sure that you construct the source code & dataset directories structure as below.

2.3. How to run

2.3.1. Visualize the dataset

To visualize 3D point clouds with 3D boxes, let's execute:

cd sfa/data_process/
python kitti_dataset.py

2.3.2. Inference

The pre-trained model was pushed to this repo.

python test.py --gpu_idx 0 --peak_thresh 0.2

2.3.3. Making demonstration

python demo_2_sides.py --gpu_idx 0 --peak_thresh 0.2

The data for the demonstration will be automatically downloaded by executing the above command.

2.3.4. Training

2.3.4.1. Single machine, single gpu

python train.py --gpu_idx 0

2.3.4.2. Distributed Data Parallel Training

Single machine (node), multiple GPUs

python train.py --multiprocessing-distributed --world-size 1 --rank 0 --batch_size 64 --num_workers 8

Two machines (two nodes), multiple GPUs

python train.py --dist-url 'tcp://IP_OF_NODE1:FREEPORT' --multiprocessing-distributed --world-size 2 --rank 0 --batch_size 64 --num_workers 8

python train.py --dist-url 'tcp://IP_OF_NODE2:FREEPORT' --multiprocessing-distributed --world-size 2 --rank 1 --batch_size 64 --num_workers 8

Second machine
First machine

Tensorboard

To track the training progress, go to the logs/ folder and

cd logs/<saved_fn>/tensorboard/
tensorboard --logdir=./

Then go to http://localhost:6006/

Contact

If you think this work is useful, please give me a star!
If you find any errors or have any suggestions, please contact me (Email: nguyenmaudung93.kstn@gmail.com).
Thank you!

Citation

@misc{Super-Fast-Accurate-3D-Object-Detection-PyTorch,
  author =       {Nguyen Mau Dung},
  title =        {{Super-Fast-Accurate-3D-Object-Detection-PyTorch}},
  howpublished = {\url{https://github.com/maudzung/Super-Fast-Accurate-3D-Object-Detection}},
  year =         {2020}
}

References

[1] CenterNet: Objects as Points paper, PyTorch Implementation
[2] RTM3D: PyTorch Implementation
[3] Libra_R-CNN: PyTorch Implementation

The YOLO-based models with the same BEV maps input:
[4] Complex-YOLO: v4, v3, v2

3D LiDAR Point pre-processing:
[5] VoxelNet: PyTorch Implementation

Folder structure

${ROOT}
└── checkpoints/
    ├── fpn_resnet_18/    
        ├── fpn_resnet_18_epoch_300.pth
└── dataset/    
    └── kitti/
        ├──ImageSets/
        │   ├── test.txt
        │   ├── train.txt
        │   └── val.txt
        ├── training/
        │   ├── image_2/ (left color camera)
        │   ├── calib/
        │   ├── label_2/
        │   └── velodyne/
        └── testing/  
        │   ├── image_2/ (left color camera)
        │   ├── calib/
        │   └── velodyne/
        └── classes_names.txt
└── sfa/
    ├── config/
    │   ├── train_config.py
    │   └── kitti_config.py
    ├── data_process/
    │   ├── kitti_dataloader.py
    │   ├── kitti_dataset.py
    │   └── kitti_data_utils.py
    ├── models/
    │   ├── fpn_resnet.py
    │   ├── resnet.py
    │   └── model_utils.py
    └── utils/
    │   ├── demo_utils.py
    │   ├── evaluation_utils.py
    │   ├── logger.py
    │   ├── misc.py
    │   ├── torch_utils.py
    │   ├── train_utils.py
    │   └── visualization_utils.py
    ├── demo_2_sides.py
    ├── demo_front.py
    ├── test.py
    └── train.py
├── README.md 
└── requirements.txt

项目代码下载

后台回复：SFA3D，即可下载上述项目代码

下载：CVPR / ECCV 2020开源代码

在CVer公众号后台回复：CVPR2020，即可下载CVPR 2020代码开源的论文合集

在CVer公众号后台回复：ECCV2020，即可下载ECCV 2020代码开源的论文合集

重磅！CVer-论文写作与投稿交流群成立

扫码添加CVer助手，可申请加入CVer-论文写作与投稿 微信交流群，目前已满2400+人，旨在交流顶会（CVPR/ICCV/ECCV/NIPS/ICML/ICLR/AAAI等）、顶刊（IJCV/TPAMI/TIP等）、SCI、EI、中文核心等写作与投稿事宜。

同时也可申请加入CVer大群和细分方向技术群，细分方向已涵盖：目标检测、图像分割、目标跟踪、人脸检测&识别、OCR、姿态估计、超分辨率、SLAM、医疗影像、Re-ID、GAN、NAS、深度估计、自动驾驶、强化学习、车道线检测、模型剪枝&压缩、去噪、去雾、去雨、风格迁移、遥感图像、行为识别、视频理解、图像融合、图像检索、论文投稿&交流、PyTorch和TensorFlow等群。

一定要备注：研究方向+地点+学校/公司+昵称（如论文写作+上海+上交+卡卡），根据格式备注，可更快被通过且邀请进群