3D Hand Pose Detection in Egocentric RGB-D Images

September 2014

3D Hand Pose Detection in Egocentric RGB-D Images

Authors:

Gregory Rogez, James S. Supancic, Maryam Khademi, Jose Maria Martinez Montiel, and Deva Ramanan

Abstract:

We focus on the task of everyday hand pose estimation from egocentric viewpoints. For this task, we show that depth sensors are particularly informative for extracting near-field interactions of the camera wearer with his/her environment. Despite the recent advances in full-body pose estimation using Kinect-like sensors, reliable monocular hand pose estimation in RGB-D images is still an unsolved problem. The problem is considerably exacerbated when analyzing hands performing daily activities from a first-person viewpoint, due to severe occlusions arising from object manipulations and a limited field-of-view. Our system addresses these difficulties by exploiting strong priors over viewpoint and pose in a discriminative tracking-by-detection framework. Our priors are operationalized through a photorealistic synthetic model of egocentric scenes, which is used to generate training data for learning depth-based pose classifiers. We evaluate our approach on an annotated dataset of real egocentric object manipulation scenes and compare to both commercial and academic approaches. Our method provides state-of-the-art performance for both hand detection and pose estimation in egocentric RGB-D images.
@workshop{Rogez-2014-121190,
author = {Gregory Rogez And James S. Supancic And Maryam Khademi And Jose Maria Martinez Montiel And Deva Ramanan},
title = {3D Hand Pose Detection in Egocentric RGB-D Images},
booktitle = {Proceedings of ECCV '14 Workshop on Consumer Depth Cameras for Computer Vision},
year = {2014},
month = {September},
pages = {356 - 371},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.