MAP Visibility Estimation for Large-Scale Dynamic 3D Reconstruction

June 2014

MAP Visibility Estimation for Large-Scale Dynamic 3D Reconstruction

Authors:

Hanbyul Joo, Hyun Soo Park, and Yaser Sheikh
Conference Paper
Proceedings of (CVPR) Computer Vision and Pattern Recognition

Abstract:

Many traditional challenges in reconstructing 3D motion, such as matching across wide baselines and handling occlusion, reduce in significance as the number of unique viewpoints increases. However, to obtain this benefit, a new challenge arises: estimating precisely which cameras observe which points at each instant in time. We present a maximum a posteriori (MAP) estimate of the time-varying visibility of the target points to reconstruct the 3D motion of an event from a large number of cameras. Our algorithm takes, as input, camera poses and image sequences, and outputs the time-varying set of the cameras in which a target patch is visible and its reconstructed trajectory. We model visibility estimation as a MAP estimate by incorporating various cues including photometric consistency, motion consistency, and geometric consistency, in conjunction with a prior that rewards consistent visibilities in proximal cameras. An optimal estimate of visibility is obtained by finding the minimum cut of a capacitated graph over cameras. We demonstrate that our method estimates visibility with greater accuracy, and increases tracking performance producing longer trajectories, at more locations, and at higher accuracies than methods that ignore visibility or use photometric consistency alone.
@conference{Joo-2014-122192,
author = {Hanbyul Joo And Hyun Soo Park And Yaser Sheikh},
title = {MAP Visibility Estimation for Large-Scale Dynamic 3D Reconstruction},
booktitle = {Proceedings of (CVPR) Computer Vision and Pattern Recognition},
year = {2014},
month = {June},
pages = {1122 - 1129},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.