Expensive Function Optimization with Stochastic Binary Outcomes

June 2013

Expensive Function Optimization with Stochastic Binary Outcomes

Conference Paper
Proceedings of (ICML) International Conference on Machine Learning

Abstract:

Real world systems often have parameterized controllers which can be tuned to improve performance. Bayesian optimization methods provide for efficient optimization of these controllers, so as to reduce the number of required experiments on the expensive physical system. In this paper we address Bayesian optimization in the setting where performance is only observed through a stochastic binary outcome - success or failure of the experiment. Unlike bandit problems, the goal is to maximize the system performance after this offline training phase rather than minimize regret during training. In this work we define the stochastic binary optimization problem and propose an approach using an adaptation of Gaussian Processes for classification that presents a Bayesian optimization framework for this problem. We propose an experiment selection metric for this setting based on expected improvement. We demonstrate the algorithm's performance on synthetic problems and on a real snake robot learning to move over an obstacle.
@conference{Tesch-2013-7743,
author = {Matthew Tesch And Jeff Schneider And Howie Choset},
title = {Expensive Function Optimization with Stochastic Binary Outcomes},
booktitle = {Proceedings of (ICML) International Conference on Machine Learning},
year = {2013},
month = {June},
pages = {1283 - 1291},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.