Photogeometric Scene Flow for High-Detail Dynamic 3D Reconstruction

December 2015

Photogeometric Scene Flow for High-Detail Dynamic 3D Reconstruction

Authors:

Paulo F. U. Gotardo, Tomas Simon, Yaser Sheikh, and Iain Matthews
Conference Paper
Proceedings of (ICCV) International Conference on Computer Vision

Abstract:

Photometric stereo (PS) is an established technique for high-detail reconstruction of 3D geometry and appearance. To correct for surface integration errors, PS is often combined with multiview stereo (MVS). With dynamic objects, PS reconstruction also faces the problem of computing optical flow (OF) for image alignment under rapid changes in illumination. Current PS methods typically compute optical flow and MVS as independent stages, each one with its own limitations and errors introduced by early regularization. In contrast, scene flow methods estimate geometry and motion, but lack the fine detail from PS. This paper proposes photogeometric scene flow (PGSF) for high-quality dynamic 3D reconstruction. PGSF performs PS, OF, and MVS simultaneously. It is based on two key observations: (i) while image alignment improves PS, PS allows for surfaces to be relit to improve alignment, (ii) PS provides surface gradients that render the smoothness term in MVS unnecessary, leading to truly data-driven, continuous depth estimates. This synergy is demonstrated in the quality of the resulting RGB appearance, 3D geometry, and 3D motion.
@conference{Gotardo-2015-122190,
author = {Paulo F. U. Gotardo And Tomas Simon And Yaser Sheikh And Iain Matthews},
title = {Photogeometric Scene Flow for High-Detail Dynamic 3D Reconstruction},
booktitle = {Proceedings of (ICCV) International Conference on Computer Vision},
year = {2015},
month = {December},
pages = {846 - 854},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.