Adaptive Auxiliary Task Weighting for Reinforcement Learning

December 2019

Adaptive Auxiliary Task Weighting for Reinforcement Learning

Authors:

Xingyu Lin, Harjatin Baweja, George Kantor, and David Held
Conference Paper
Proceedings of (NeurIPS) Neural Information Processing Systems

Abstract:

Reinforcement learning is known to be sample inefficient, preventing its application to many real-world problems, especially with high dimensional observations like images. Transferring knowledge from other auxiliary tasks is a powerful tool for improving the learning efficiency. However, the usage of auxiliary tasks has been limited so far due to the difficulty in selecting and combining different auxiliary tasks. In this work, we propose a principled online learning algorithm that dynamically combines different auxiliary tasks to speed up training for reinforcement learning. Our method is based on the idea that auxiliary tasks should provide gradient directions that, in the long term, help to decrease the loss of the main task. We show in various environments that our algorithm can effectively combine a variety of different auxiliary tasks and achieves significant speedup compared to previous heuristic approches of adapting auxiliary task weights.
@conference{Lin-2019-121963,
author = {Xingyu Lin And Harjatin Baweja And George Kantor And David Held},
title = {Adaptive Auxiliary Task Weighting for Reinforcement Learning},
booktitle = {Proceedings of (NeurIPS) Neural Information Processing Systems},
year = {2019},
month = {December},
pages = {4773 - 4784},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.