The More You Know: Using Knowledge Graphs for Image Classification

July 2017

The More You Know: Using Knowledge Graphs for Image Classification

Authors:

Kenneth Marino, Ruslan Salakhutdinov, and Abhinav Gupta
Conference Paper
Proceedings of (CVPR) Computer Vision and Pattern Recognition

Abstract:

One characteristic that sets humans apart from modern learning-based computer vision algorithms is the ability to acquire knowledge about the world and use that knowledge to reason about the visual world. Humans can learn about the characteristics of objects and the relationships that occur between them to learn a large variety of visual concepts, often with few examples. This paper investigates the use of structured prior knowledge in the form of knowledge graphs and shows that using this knowledge improves performance on image classification. We build on recent work on end-to-end learning on graphs, introducing the Graph Search Neural Network as a way of efficiently incorporating large knowledge graphs into a vision classification pipeline. We show in a number of experiments that our method outperforms standard neural network baselines for multi-label classification.
@conference{Marino-2017-113310,
author = {Kenneth Marino And Ruslan Salakhutdinov And Abhinav Gupta},
title = {The More You Know: Using Knowledge Graphs for Image Classification},
booktitle = {Proceedings of (CVPR) Computer Vision and Pattern Recognition},
year = {2017},
month = {July},
pages = {20 - 28},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.