Augmenting GAIL with BC for sample efficient imitation learning

November 2020

Augmenting GAIL with BC for sample efficient imitation learning

Conference Paper
Proceedings of (CoRL) Conference on Robot Learning

Abstract:

Imitation learning is the problem of recovering an expert policy without access to a reward signal. Behavior cloning and GAIL are two widely used methods for performing imitation learning. Behavior cloning converges in a few iterations but doesn't achieve peak performance due to its inherent iid assumption about the state-action distribution. GAIL addresses the issue by accounting for the temporal dependencies when performing a state distribution matching between the agent and the expert. Although GAIL is sample efficient in the number of expert trajectories required, it is still not very sample efficient in terms of the environment interactions needed for convergence of the policy. Given the complementary benefits of both methods, we present a simple and elegant method to combine both methods to enable stable and sample efficient learning. Our algorithm is very simple to implement and integrates with different policy gradient algorithms. We demonstrate the effectiveness of the algorithm in low dimensional control tasks, gridworlds and in high dimensional image-based tasks.
@conference{Jena-2020-126865,
author = {Rohit Jena And Changliu Liu And Katia Sycara},
title = {Augmenting GAIL with BC for sample efficient imitation learning},
booktitle = {Proceedings of (CoRL) Conference on Robot Learning},
year = {2020},
month = {November},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.