PAttNet: Patch-attentive deep network for action unit detection

September 2019

PAttNet: Patch-attentive deep network for action unit detection

Authors:

Itir Onal Ertugrul, Laszlo A. Jeni, and Jeffrey F. Cohn

Abstract:

Facial action units (AUs) refer to specific facial locations. Recent efforts in automatic AU detection have focused on learning their representations. Two factors have limited progress. One is that current approaches implicitly assume that facial patches are robust to head rotation. The other is that the relation between patches and AUs is pre-defined or ignored. Both assumptions are problematic. We propose a patch-attentive deep network called PAttNet for AU detection that learns mappings of patches and AUs, controls for 3D head and face rotation, and exploits co-occurrence among AUs. We encode patches with separate convolutional neural networks (CNNs) and weight the contribution of each patch to detection of specific AUs using a sigmoid patch attention mechanism. Unlike conventional softmax attention mechanisms, a sigmoidal attention mechanism allows multiple patches to contribute to detection of specific AUs. The latter is important because AUs often co-occur and multiple patches may be needed to detect them reliably. On the BP4D dataset, PAttNet improves upon state-of-the-art by 3.7%. Visualization of the learned attention maps reveal power of this patch-based approach.
@conference{Ertugrul-2019-119657,
author = {Itir Onal Ertugrul And Laszlo A. Jeni And Jeffrey F. Cohn},
title = {PAttNet: Patch-attentive deep network for action unit detection},
booktitle = {Proceedings of British Machine Vision Conference (BMVC '19)},
year = {2019},
month = {September},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.