Viewpoint-consistent 3D Face Alignment

September 2018

Viewpoint-consistent 3D Face Alignment

Authors:

Sergely Tulyakov, Laszlo A. Jeni, Jeffrey F. Cohn, and Nicu Sebe

Abstract:

Most approaches to face alignment treat the face as a 2D object, which fails to represent depth variation and is vulnerable to loss of shape consistency when the face rotates along a 3D axis. Because faces commonly rotate three dimensionally, 2D approaches are vulnerable to significant error. 3D morphable models, employed as a second step in 2D+3D approaches are robust to face rotation but are computationally too expensive for many applications, yet their ability to maintain viewpoint consistency is unknown. We present an alternative approach that estimates 3D face landmarks in a single face image. The method uses a regression forest-based algorithm that adds a third dimension to the common cascade pipeline. 3D face landmarks are estimated directly, which avoids fitting a 3D morphable model. The proposed method achieves viewpoint consistency in a computationally efficient manner that is robust to 3D face rotation. To train and test our approach, we introduce the Multi-PIE Viewpoint Consistent database. In empirical tests, the proposed method achieved simple yet effective head pose estimation and viewpoint consistency on multiple measures relative to alternative approaches.
@article{Tulyakov-2018-119649,
author = {Sergely Tulyakov And Laszlo A. Jeni And Jeffrey F. Cohn And Nicu Sebe},
title = {Viewpoint-consistent 3D Face Alignment},
journal = {Proceedings of IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2018},
month = {September},
volume = {40},
number = {9},
pages = {2250 - 2264},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.