First-Person Activity Forecasting with Online Inverse Reinforcement Learning

October 2017

First-Person Activity Forecasting with Online Inverse Reinforcement Learning

Authors:

Nicholas Rhinehart and Kris M. Kitani
Conference Paper
Proceedings of (ICCV) International Conference on Computer Vision

Abstract:

We address the problem of incrementally modeling and forecasting long-term goals of a first-person camera wearer: what the user will do, where they will go, and what goal they are attempting to reach. In contrast to prior work in trajectory forecasting, our algorithm, DARKO, goes further to reason about semantic states (will I pick up an object?), and future goal states that are far both in terms of space and time. DARKO learns and forecasts from first-person visual observations of the user's daily behaviors via an Online Inverse Reinforcement Learning (IRL) approach. Classical IRL discovers only the rewards in a batch setting, whereas DARKO discovers the states, transitions, rewards, and goals of a user from streaming data. Among other results, we show DARKO forecasts goals better than competing methods in both noisy and ideal settings, and our approach is theoretically and empirically no-regret.

Notes:

Marr Prize (Best Paper) Honorable Mention
@conference{Rhinehart-2017-101747,
author = {Nicholas Rhinehart And Kris M. Kitani},
title = {First-Person Activity Forecasting with Online Inverse Reinforcement Learning},
booktitle = {Proceedings of (ICCV) International Conference on Computer Vision},
year = {2017},
month = {October},
pages = {3716 - 3725},
publisher = {IEEE},
keywords = {Activity Forecasting, Inverse Reinforcement Learning, First-Person Video, Online Learning},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.