Photometric Mesh Optimization for Video-Aligned 3D Object Reconstruction

June 2019

Photometric Mesh Optimization for Video-Aligned 3D Object Reconstruction

Authors:

C. Lin, O. Wang, B. C. Russell, E. Shechtman, V. G. Kim, M. Fisher, and S. Lucey
Conference Paper
Proceedings of (CVPR) Computer Vision and Pattern Recognition

Abstract:

In this paper, we address the problem of 3D object mesh reconstruction from RGB videos. Our approach combines the best of multi-view geometric and data-driven methods for 3D reconstruction by optimizing object meshes for multi-view photometric consistency while constraining mesh deformations with a shape prior. We pose this as a piecewise image alignment problem for each mesh face projection. Our approach allows us to update shape parameters from the photometric error without any depth or mask information. Moreover, we show how to avoid a degeneracy of zero photometric gradients via rasterizing from a virtual viewpoint. We demonstrate 3D object mesh reconstruction results from both synthetic and real-world videos with our photometric mesh optimization, which is unachievable with either naïve mesh generation networks or traditional pipelines of surface reconstruction without heavy manual post-processing.
@conference{Lin-2019-121013,
author = {C. Lin And O. Wang And B. C. Russell And E. Shechtman And V. G. Kim And M. Fisher And S. Lucey},
title = {Photometric Mesh Optimization for Video-Aligned 3D Object Reconstruction},
booktitle = {Proceedings of (CVPR) Computer Vision and Pattern Recognition},
year = {2019},
month = {June},
pages = {969 - 978},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.