Clustering Appearance for Scene Analysis

June 2006

Clustering Appearance for Scene Analysis

Authors:

Sanjeev Jagannatha Koppal and Srinivasa G. Narasimhan
Conference Paper
Proceedings of (CVPR) Computer Vision and Pattern Recognition

Abstract:

We propose a new approach called "appearance clustering" for scene analysis. The key idea in this approach is that the scene points can be clustered according to their surface normals, even when the geometry, material and lighting are all unknown. We achieve this by analyzing an image sequence of a scene as it is illuminated by a smoothly moving distant source. Each pixel thus gives rise to a "continuous appearance profile" that yields information about derivatives of the BRDF w.r.t source direction. This information is directly related to the surface normal of the scene point when the source path follows an unstructured trajectory (obtained, say, by "hand-waving"). Based on this observation, we transform the appearance profiles and propose a metric that can be used with any unsupervised clustering algorithm to obtain iso-normal clusters. We successfully demonstrate appearance clustering for complex indoor and outdoor scenes. In addition, iso-normal clusters serve as excellent priors for scene geometry and can strongly impact any vision algorithm that attempts to estimate material, geometry and/or lighting properties in a scene from images. We demonstrate this impact for applications such as diffuse and specular separation, both calibrated and uncalibrated photometric stereo of non-lambertian scenes, light source estimation and texture transfer.
@conference{Koppal-2006-9496,
author = {Sanjeev Jagannatha Koppal And Srinivasa G. Narasimhan},
title = {Clustering Appearance for Scene Analysis},
booktitle = {Proceedings of (CVPR) Computer Vision and Pattern Recognition},
year = {2006},
month = {June},
pages = {1323 - 1330},
keywords = {appearance modeling, physics based vision},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.