Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning

May 2017

Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning

Authors:

Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J. Lim, Abhinav Gupta, Li Fei-Fei, and Ali Farhadi
Conference Paper
Proceedings of (ICRA) International Conference on Robotics and Automation

Abstract:

Two less addressed issues of deep reinforcement learning are (1) lack of generalization capability to new target goals, and (2) data inefficiency i.e., the model requires several (and often costly) episodes of trial and error to converge, which makes it impractical to be applied to real-world scenarios. In this paper, we address these two issues and apply our model to the task of target-driven visual navigation. To address the first issue, we propose an actor-critic model whose policy is a function of the goal as well as the current state, which allows to better generalize. To address the second issue, we propose AI2-THOR framework, which provides an environment with high-quality 3D scenes and physics engine. Our framework enables agents to take actions and interact with objects. Hence, we can collect a huge number of training samples efficiently.

We show that our proposed method (1) converges faster than the state-of-the-art deep reinforcement learning methods, (2) generalizes across targets and across scenes, (3) generalizes to a real robot scenario with a small amount of fine-tuning (although the model is trained in simulation), (4) is end-to-end trainable and does not need feature engineering, feature matching between frames or 3D reconstruction of the environment.
@conference{Zhu-2017-113326,
author = {Yuke Zhu And Roozbeh Mottaghi And Eric Kolve And Joseph J. Lim And Abhinav Gupta And Li Fei-Fei And Ali Farhadi},
title = {Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning},
booktitle = {Proceedings of (ICRA) International Conference on Robotics and Automation},
year = {2017},
month = {May},
pages = {3357 - 3364},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.