Controlling the False Discovery Rate in Astrophysical Data Analysis

December 2001

Controlling the False Discovery Rate in Astrophysical Data Analysis

Authors:

C. Miller, C. Genovese, R. Nichol, L. Wasserman, A. Connolly, D. Reichart, A. Hopkins, J. Schneider, and A. Moore

Abstract:

The false-discovery rate (FDR) is a new statistical procedure to control the number of mistakes made when performing multiple hypothesis tests, i.e., when comparing many data against a given model hypothesis. The key advantage of FDR is that it allows one to a priori control the average fraction of false rejections made (when comparing with the null hypothesis) over the total number of rejections performed. We compare FDR with the standard procedure of rejecting all tests that do not match the null hypothesis above some arbitrarily chosen confidence limit, e.g., 2 σ, or at the 95% confidence level. We find a similar rate of correct detections, but with significantly fewer false detections. Moreover, the FDR procedure is quick and easy to compute and can be trivially adapted to work with correlated data. The purpose of this paper is to introduce the FDR procedure to the astrophysics community. We illustrate the power of FDR through several astronomical examples, including the detection of features against a smooth one-dimensional function, e.g., seeing the "baryon wiggles" in a power spectrum of matter fluctuations, and source pixel detection in imaging data. In this era of large data sets and high-precision measurements, FDR provides the means to adaptively control a scientifically meaningful quantity—the fraction of false discoveries over total discoveries.
@article{Miller-2001-119730,
author = {C. Miller And C. Genovese And R. Nichol And L. Wasserman And A. Connolly And D. Reichart And A. Hopkins And J. Schneider And A. Moore},
title = {Controlling the False Discovery Rate in Astrophysical Data Analysis},
journal = {Proceedings of Astronomical Journal},
year = {2001},
month = {December},
volume = {122},
number = {6},
pages = {3492 - 3505},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.