Forecasting Time-to-Collision from Monocular Video: Feasibility, Dataset, and Challenges

November 2019

Forecasting Time-to-Collision from Monocular Video: Feasibility, Dataset, and Challenges

Authors:

Aashi Manglik, Xinshuo Weng, Eshed Ohn-Bar, and Kris Kitani
Conference Paper
Proceedings of (IROS) IEEE/RSJ International Conference on Intelligent Robots and Systems

Abstract:

We explore the possibility of using a single monocular camera to forecast the time to collision between a suitcase-shaped robot being pushed by its user and other nearby pedestrians. We develop a purely image-based deep learning approach that directly estimates the time to collision without the need of relying on explicit geometric depth estimates or velocity information to predict future collisions. While previous work has focused on detecting immediate collision in the context of navigating Unmanned Aerial Vehicles, the detection was limited to a binary variable (ie, collision or no collision). We propose a more fine-grained approach to collision forecasting by predicting the exact time to collision in terms of milliseconds, which is more helpful for collision avoidance in the context of dynamic path planning. To evaluate our method, we have collected a novel dataset of over 13,000 indoor video segments each showing a trajectory of at least one person ending in a close proximity (a near collision) with the camera mounted on a mobile suitcase-shaped platform. Using this dataset, we do extensive experimentation on different temporal windows as input using an exhaustive list of state-of-the-art convolutional neural networks (CNNs). Our results show that our proposed multi-stream CNN is the best model for predicting time to near-collision. The average prediction error of our time to near collision is 0.75 seconds across the test videos.
@conference{Manglik-2019-124274,
author = {Aashi Manglik And Xinshuo Weng And Eshed Ohn-Bar And Kris Kitani},
title = {Forecasting Time-to-Collision from Monocular Video: Feasibility, Dataset, and Challenges},
booktitle = {Proceedings of (IROS) IEEE/RSJ International Conference on Intelligent Robots and Systems},
year = {2019},
month = {November},
pages = {8081 - 8088},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.