Hand Keypoint Detection in Single Images Using Multiview Bootstrapping

July 2017

Hand Keypoint Detection in Single Images Using Multiview Bootstrapping

Authors:

Tomas Simon, Hanbyul Joo, Iain Matthews, and Yaser Sheikh
Conference Paper
Proceedings of (CVPR) Computer Vision and Pattern Recognition

Abstract:

We present an approach that uses a multi-camera system to train fine-grained detectors for keypoints that are prone to occlusion, such as the joints of a hand. We call this procedure multiview bootstrapping: first, an initial keypoint detector is used to produce noisy labels in multiple views of the hand. The noisy detections are then triangulated in 3D using multiview geometry or marked as outliers. Finally, the reprojected triangulations are used as new labeled training data to improve the detector. We repeat this process, generating more labeled data in each iteration. We derive a result analytically relating the minimum number of views to achieve target true and false positive rates for a given detector. The method is used to train a hand keypoint detector for single images. The resulting keypoint detector runs in realtime on RGB images and has accuracy comparable to methods that use depth sensors. The single view detector, triangulated over multiple views, enables 3D markerless hand motion capture with complex object interactions.
@conference{Simon-2017-122184,
author = {Tomas Simon And Hanbyul Joo And Iain Matthews And Yaser Sheikh},
title = {Hand Keypoint Detection in Single Images Using Multiview Bootstrapping},
booktitle = {Proceedings of (CVPR) Computer Vision and Pattern Recognition},
year = {2017},
month = {July},
pages = {4645 - 4653},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.