Generative Hybrid Representations for Activity Forecasting with No-Regret Learning

June 2020

Generative Hybrid Representations for Activity Forecasting with No-Regret Learning

Authors:

Jiaqi Guan, Ye Yuan, Kris Kitani, and Nicholas Rhinehart
Conference Paper
Proceedings of (CVPR) Computer Vision and Pattern Recognition

Abstract:

Automatically reasoning about future human behaviors is a difficult problem but has significant practical applications to assistive systems. Part of this difficulty stems from learning systems' inability to represent all kinds of behaviors. Some behaviors, such as motion, are best described with continuous representations, whereas others, such as picking up a cup, are best described with discrete representations. Furthermore, human behavior is generally not fixed: people can change their habits and routines. This suggests these systems must be able to learn and adapt continuously. In this work, we develop an efficient deep generative model to jointly forecast a person's future discrete actions and continuous motions. On a large-scale egocentric dataset, EPIC-KITCHENS, we observe our method generates high-quality and diverse samples while exhibiting better generalization than related generative models. Finally, we propose a variant to continually learn our model from streaming data, observe its practical effectiveness, and theoretically justify its learning efficiency.
@conference{Guan-2020-120757,
author = {Jiaqi Guan And Ye Yuan And Kris Kitani And Nicholas Rhinehart},
title = {Generative Hybrid Representations for Activity Forecasting with No-Regret Learning},
booktitle = {Proceedings of (CVPR) Computer Vision and Pattern Recognition},
year = {2020},
month = {June},
pages = {170 - 179},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.