Dense 3D Face Alignment from 2D Videos in Real-Time

May 2015

Dense 3D Face Alignment from 2D Videos in Real-Time


Abstract:

To enable real-time, person-independent 3D registration from 2D video, we developed a 3D cascade regression approach in which facial landmarks remain invariant across pose over a range of approximately 60 degrees. From a single 2D image of a person's face, a dense 3D shape is registered in real time for each frame. The algorithm utilizes a fast cascade regression framework trained on high-resolution 3D face-scans of posed and spontaneous emotion expression. The algorithm first estimates the location of a dense set of markers and their visibility, then reconstructs face shapes by fitting a part-based 3D model. Because no assumptions are required about illumination or surface properties, the method can be applied to a wide range of imaging conditions that include 2D video and uncalibrated multi-view video. The method has been validated in a battery of experiments that evaluate its precision of 3D reconstruction and extension to multi-view reconstruction. Experimental findings strongly support the validity of real-time, 3D registration and reconstruction from 2D video. The software is available online at http://zface.org.
@conference{Jeni-2015-119669,
author = {Laszlo A. Jeni And Jeffrey F. Cohn And Takeo Kanade},
title = {Dense 3D Face Alignment from 2D Videos in Real-Time},
booktitle = {Proceedings of 11th IEEE International Conference and Workshops on Automatic Face & Gesture Recognition (FG '15)},
year = {2015},
month = {May},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.