ActionVLAD: Learning Spatio-Temporal Aggregation for Action Classification

July 2017

ActionVLAD: Learning Spatio-Temporal Aggregation for Action Classification

Authors:

Rohit Girdhar, Deva Ramanan, Abhinav Gupta, Josef Sivic, and Bryan Russell
Conference Paper
Proceedings of (CVPR) Computer Vision and Pattern Recognition

Abstract:

In this work, we introduce a new video representation for action classification that aggregates local convolutional features across the entire spatio-temporal extent of the video. We do so by integrating state-of-the-art two-stream networks with learnable spatio-temporal feature aggregation. The resulting architecture is end-to-end trainable for whole-video classification. We investigate different strategies for pooling across space and time and combining signals from the different streams. We find that: (i) it is important to pool jointly across space and time, but (ii) appearance and motion streams are best aggregated into their own separate representations. Finally, we show that our representation outperforms the two-stream base architecture by a large margin (13% relative) as well as out-performs other baselines with comparable base architectures on HMDB51, UCF101, and Charades video classification benchmarks.
@conference{Girdhar-2017-113316,
author = {Rohit Girdhar And Deva Ramanan And Abhinav Gupta And Josef Sivic And Bryan Russell},
title = {ActionVLAD: Learning Spatio-Temporal Aggregation for Action Classification},
booktitle = {Proceedings of (CVPR) Computer Vision and Pattern Recognition},
year = {2017},
month = {July},
pages = {3165 - 3174},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.