Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition

October 2009

Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition

Authors:

Abhinav Gupta, Aniruddha Kembhavi, and Larry S. Davis

Abstract:

Interpretation of images and videos containing humans interacting with different objects is a daunting task. It involves understanding scene or event, analyzing human movements, recognizing manipulable objects, and observing the effect of the human movement on those objects. While each of these perceptual tasks can be conducted independently, recognition rate improves when interactions between them are considered. Motivated by psychological studies of human perception, we present a Bayesian approach which integrates various perceptual tasks involved in understanding human-object interactions. Previous approaches to object and action recognition rely on static shape or appearance feature matching and motion analysis, respectively. Our approach goes beyond these traditional approaches and applies spatial and functional constraints on each of the perceptual elements for coherent semantic interpretation. Such constraints allow us to recognize objects and actions when the appearances are not discriminative enough. We also demonstrate the use of such constraints in recognition of actions from static images without using any motion information.
@article{Gupta-2009-113368,
author = {Abhinav Gupta And Aniruddha Kembhavi And Larry S. Davis},
title = {Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition},
journal = {Proceedings of IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2009},
month = {October},
volume = {31},
number = {10},
pages = {1775 - 1789},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.