SocNavBench: A Grounded Simulation Testing Framework for Evaluating Social Navigation

August 2021

SocNavBench: A Grounded Simulation Testing Framework for Evaluating Social Navigation

Authors:

Abhijat Biswas, Allan Wang, Gustavo Silvera, Aaron Steinfeld, and Henny Admoni

Abstract:

The human-robot interaction (HRI) community has developed many methods for robots to navigate safely and socially alongside humans. However, experimental procedures to evaluate these works are usually constructed on a per-method basis. Such disparate evaluations make it difficult to compare the performance of such methods across the literature. To bridge this gap, we introduce SocNavBench, a simulation framework for evaluating social navigation algorithms. SocNavBench comprises a simulator with photo-realistic capabilities and curated social navigation scenarios grounded in real-world pedestrian data. We also provide an implementation of a suite of metrics to quantify the performance of navigation algorithms on these scenarios. Altogether, SocNavBench provides a test framework for evaluating disparate social navigation methods in a consistent and interpretable manner. To illustrate its use, we demonstrate testing three existing social navigation methods and a baseline method on SocNavBench, showing how the suite of metrics helps infer their performance trade-offs. Our code is open-source, allowing the addition of new scenarios and metrics by the community to help evolve SocNavBench to reflect advancements in our understanding of social navigation.

Notes:

Special Issue: Test Methods for Human-Robot Teaming Performance Evaluations https://doi.org/10.1145/3476413
@article{Biswas-2021-129970,
author = {Abhijat Biswas And Allan Wang And Gustavo Silvera And Aaron Steinfeld And Henny Admoni},
title = {SocNavBench: A Grounded Simulation Testing Framework for Evaluating Social Navigation},
journal = {Proceedings of ACM Transactions on Human-Robot Interaction (THRI)},
year = {2021},
month = {August},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.