CARLA Simulated Data for Rare Road Object Detection

September 2021

CARLA Simulated Data for Rare Road Object Detection

Authors:

Tom Bu, Xinhe Zhang, Christoph Mertz, and John M. Dolan

Abstract:

Labeled data are paramount for modern, deep learning object detection models. However, such data are not always available, either due to time and financial constraints or due to the rarity of certain objects. In this paper, we show that the CARLA simulator can be used effectively to provide automatic annotations for custom street-view objects, boosting datasets for objects with few labels. We evaluate our models on real world images and show that low-shot training data expanded by synthetic images rendered in CARLA can provide better performance than training models with low shot examples alone. To overcome the sim-to-real domain gap, we perform domain randomization by taking advantage of CARLA’s diverse simulations of weather conditions, actors, and maps. We train detectors on CARLA-generated images of two different object classes and evaluate them on publicly available datasets. We provide access to our synthetic fire hydrant and crosswalk datasets as well as provide step-by-step instructions to generate custom datasets in CARLA.
@conference{Bu-2021-130704,
author = {Tom Bu And Xinhe Zhang And Christoph Mertz And John M. Dolan},
title = {CARLA Simulated Data for Rare Road Object Detection},
booktitle = {Proceedings of IEEE International Intelligent Transportation Systems Conference (ITSC '21)},
year = {2021},
month = {September},
pages = {2794 - 2801},
keywords = {CARLA, Simulated Data, Computer Vision},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.