Deep Component Analysis via Alternating Direction Neural Networks

September 2018

Deep Component Analysis via Alternating Direction Neural Networks

Authors:

C. Murdock, M. Chang, and S. Lucey
Conference Paper
Proceedings of (ECCV) European Conference on Computer Vision

Abstract:

Despite a lack of theoretical understanding, deep neural networks have achieved unparalleled performance in a wide range of applications. On the other hand, shallow representation learning with component analysis is associated with rich intuition and theory, but smaller capacity often limits its usefulness. To bridge this gap, we introduce Deep Component Analysis (DeepCA), an expressive multilayer model formulation that enforces hierarchical structure through constraints on latent variables in each layer. For inference, we propose a differentiable optimization algorithm implemented using recurrent Alternating Direction Neural Networks (ADNNs) that enable parameter learning using standard backpropagation. By interpreting feed-forward networks as single-iteration approximations of inference in our model, we provide both a novel theoretical perspective for understanding them and a practical technique for constraining predictions with prior knowledge. Experimentally, we demonstrate performance improvements on a variety of tasks, including single-image depth prediction with sparse output constraints.
@conference{Murdock-2018-121015,
author = {C. Murdock And M. Chang And S. Lucey},
title = {Deep Component Analysis via Alternating Direction Neural Networks},
booktitle = {Proceedings of (ECCV) European Conference on Computer Vision},
year = {2018},
month = {September},
pages = {851 - 867},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.