3D Object Detection for Autonomous Driving using Deep Learning (Master's Thesis Project)

www.fregu856.com/
PyTorch implementation: github.com/fre...
Frustum-PointNet trained on the KITTI dataset: 0:00.
Image-Only model trained on the KITTI dataset: 1:58.
Frustum-PointNet trained on the synthetic SYN dataset: 3:57.
Extended Frustum-PointNet trained on the KITTI dataset: 5:55.
Sequence order: KITTI test sequence 0011, 0002, 0007, 0001.
The main video shows the estimated 3Dbboxes visualized in the input LiDAR data, the bottom-right video shows the estimated 3Dbboxes visualized in the input image data, and the bottom-left video shows the 2Dbboxes which are taken as input to the models.
Detection is performed independently on each individual frame.
(Master's Thesis) Automotive 3D Object Detection Without Target Domain Annotations, Fredrik Gustafsson and Erik Linder-Norén, 2018: urn.kb.se/resol...
Master's thesis project at Linköping University, in collaboration with Zenuity.
Abstract:
In this thesis we study a perception problem in the context of autonomous driving. Specifically, we study the computer vision problem of 3D object detection, in which objects should be detected from various sensor data and their position in the 3D world should be estimated. We also study the application of Generative Adversarial Networks in domain adaptation techniques, aiming to improve the 3D object detection model’s ability to transfer between different domains.
The state-of-the-art Frustum-PointNet architecture for LiDAR-based 3D object detection was implemented and found to closely match its reported performance when trained and evaluated on the KITTI dataset. The architecture was also found to transfer reasonably well from the synthetic SYN dataset to KITTI, and is thus believed to be usable in a semi-automatic 3D bounding box annotation process. The Frustum-PointNet architecture was also extended to explicitly utilize image features, which surprisingly degraded its detection performance. Furthermore, an image-only 3D object detection model was designed and implemented, which was found to compare quite favourably with current state-of-the-art in terms of detection performance.
Additionally, the PixelDA approach was adopted and successfully applied to the MNIST to MNIST-M domain adaptation problem, which validated the idea that unsupervised domain adaptation using Generative Adversarial Networks can improve the performance of a task network for a dataset lacking ground truth annotations. Surprisingly, the approach did however not significantly improve upon the performance of the image-based 3D object detection models when trained on the SYN dataset and evaluated on KITTI.