Attentional Pooling for Action Recognition

December 2017

Attentional Pooling for Action Recognition

Authors:

R. Girdhar and D. Ramanan
Conference Paper
Proceedings of (NeurIPS) Neural Information Processing Systems

Abstract:

We introduce a simple yet surprisingly powerful model to incorporate attention in action recognition and human object interaction tasks. Our proposed attention module can be trained with or without extra supervision, and gives a sizable boost in accuracy while keeping the network size and computational cost nearly the same. It leads to significant improvements over state of the art base architecture on three standard action recognition benchmarks across still images and videos, and establishes new state of the art on MPII dataset with 12.5% relative improvement. We also perform an extensive analysis of our attention module both empirically and analytically. In terms of the latter, we introduce a novel derivation of bottom-up and top-down attention as low-rank approximations of bilinear pooling methods (typically used for fine-grained classification). From this perspective, our attention formulation suggests a novel characterization of action recognition as a fine-grained recognition problem.
@conference{Girdhar-2017-121172,
author = {R. Girdhar And D. Ramanan},
title = {Attentional Pooling for Action Recognition},
booktitle = {Proceedings of (NeurIPS) Neural Information Processing Systems},
year = {2017},
month = {December},
pages = {33 - 44},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.