A 4D Light-Field Dataset and CNN Architectures for Material Recognition

October 2016

A 4D Light-Field Dataset and CNN Architectures for Material Recognition

Authors:

Ting-Chun Wang, Jun-Yan Zhu, Hiroaki Ebi, Manmohan Chandraker, Alexei A. Efros, and Ravi Ramamoorthi
Conference Paper
Proceedings of (ECCV) European Conference on Computer Vision

Abstract:

We introduce a new light-field dataset of materials, and take advantage of the recent success of deep learning to perform material recognition on the 4Dlight-field. Our dataset contains 12 material categories, each with 100 images taken with a Lytro Illum, from which we extract about 30,000 patches in total. To the best of our knowledge, this is the first mid-size dataset for light-field images. Our main goal is to investigate whether the additional information in a light-field (such as multiple sub-aperture views and view-dependent reflectance effects) can aid material recognition. Since recognition networks have not been trained on 4D images before, we propose and compare several novel CNN architectures to train on light-field images. In our experiments, the best performing CNN architecture achieves a 7% boost compared with 2D image classification (70% to 77%). These results constitute important baselines that can spur further research in the use of CNNs for light-field applications. Upon publication, our dataset also enables other novel applications of light-fields, including object detection, image segmentation and view interpolation.
@conference{Wang-2016-125697,
author = {Ting-Chun Wang And Jun-Yan Zhu And Hiroaki Ebi And Manmohan Chandraker And Alexei A. Efros And Ravi Ramamoorthi},
title = {A 4D Light-Field Dataset and CNN Architectures for Material Recognition},
booktitle = {Proceedings of (ECCV) European Conference on Computer Vision},
year = {2016},
month = {October},
pages = {121 - 138},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.