Approximate MaxEnt Inverse Optimal Control and its Application for Mental Simulation of Human Interactions

January 2015

Approximate MaxEnt Inverse Optimal Control and its Application for Mental Simulation of Human Interactions

Authors:

De-An Huang, Amir-Massoud Farahmand, Kris M. Kitani, and J. Andrew (Drew) Bagnell

Abstract:

Maximum entropy inverse optimal control (MaxEnt IOC) is an effective means of discovering the underlying cost function of demonstrated human activity and can be used to predict human behavior over low-dimensional state spaces (i.e., forecasting of 2D trajectories). To enable inference in very large state spaces, we introduce an approximate MaxEnt IOC procedure to address the fundamental computational bottleneck stemming from calculating the partition function via dynamic programming. Approximate MaxEnt IOC is based on two components: approximate dynamic programming and Monte Carlo sampling. We analyze this approximation approach and provide a finite-sample error upper bound on its excess loss. We validate the proposed method in the context of analyzing dual-agent interactions from video, where we use approximate MaxEnt IOC to simulate mental images of a single agents body pose sequence (a high-dimensional image space). We experiment with sequences image data taken from RGB and RGBD data and show that it is possible to learn cost functions that lead to accurate predictions in high- dimensional problems that were previously intractable.
@conference{Huang-2015-5899,
author = {De-An Huang And Amir-Massoud Farahmand And Kris M. Kitani And J. Andrew (Drew) Bagnell},
title = {Approximate MaxEnt Inverse Optimal Control and its Application for Mental Simulation of Human Interactions},
booktitle = {Proceedings of 29th AAAI Conference on Artificial Intelligence (AAAI '15)},
year = {2015},
month = {January},
pages = {2673 - 2679},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.