Video-based 3D motion capture through biped control

July 2012

Video-based 3D motion capture through biped control

Authors:

Marek Vondrak, Leonid Sigal, Jessica Hodgins, and Odest Jenkins

Abstract:

Marker-less motion capture is a challenging problem, particularly when only monocular video is available. We estimate human motion from monocular video by recovering three-dimensional controllers capable of implicitly simulating the observed human behavior and replaying this behavior in other environments and under physical perturbations. Our approach employs a state-space biped controller with a balance feedback mechanism that encodes control as a sequence of simple control tasks. Transitions among these tasks are triggered on time and on proprioceptive events (e.g., contact). Inference takes the form of optimal control where we optimize a high-dimensional vector of control parameters and the structure of the controller based on an objective function that compares the resulting simulated motion with input observations. We illustrate our approach by automatically estimating controllers for a variety of motions directly from monocular video. We show that the estimation of controller structure through incremental optimization and refinement leads to controllers that are more stable and that better approximate the reference motion. We demonstrate our approach by capturing sequences of walking, jumping, and gymnastics.
@article{Vondrak-2012-121996,
author = {Marek Vondrak And Leonid Sigal And Jessica Hodgins And Odest Jenkins},
title = {Video-based 3D motion capture through biped control},
journal = {Proceedings of ACM Transactions On Graphics (TOG)},
year = {2012},
month = {July},
volume = {31},
number = {4},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.