Efficient online multi-person 2d pose tracking with recurrent spatio-temporal affinity fields

June 2019

Efficient online multi-person 2d pose tracking with recurrent spatio-temporal affinity fields

Authors:

Yaadhav Raaj, Haroon Idrees, Gines Hidalgo Martinez, and Yaser Sheikh
Conference Paper
Proceedings of (CVPR) Computer Vision and Pattern Recognition

Abstract:

We present an online approach to efficiently and simultaneously detect and track 2D poses of multiple people in a video sequence. We build upon Part Affinity Field (PAF) representation designed for static images, and propose an architecture that can encode and predict Spatio-Temporal Affinity Fields (STAF) across a video sequence. In particular, we propose a novel temporal topology cross-linked across limbs which can consistently handle body motions of a wide range of magnitudes. Additionally, we make the overall approach recurrent in nature, where the network ingests STAF heatmaps from previous frames and estimates those for the current frame. Our approach uses only online inference and tracking, and is currently the fastest and the most accurate bottom-up approach that is runtime-invariant to the number of people in the scene and accuracy-invariant to input frame rate of camera. Running at~ 30 fps on a single GPU at single scale, it achieves highly competitive results on the PoseTrack benchmarks.
@conference{Raaj-2019-122173,
author = {Yaadhav Raaj And Haroon Idrees And Gines Hidalgo Martinez And Yaser Sheikh},
title = {Efficient online multi-person 2d pose tracking with recurrent spatio-temporal affinity fields},
booktitle = {Proceedings of (CVPR) Computer Vision and Pattern Recognition},
year = {2019},
month = {June},
pages = {4615 - 4623},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.