July
2011
Balancing Safety and Exploitability in Opponent Modeling
Authors:
Abstract:
Opponent modeling is a critical mechanism in repeated games. It allows a player to adapt its strategy in order to better respond to the presumed preferences of his opponents. We
introduce a new modeling technique that adaptively balances exploitability and risk reduction. An opponent’s strategy is modeled with a set of possible strategies that contain the actual strategy with a high probability. The algorithm is safe as the expected payoff is above the minimax payoff with a high probability, and can exploit the opponents’ preferences when sufficient observations have been obtained. We apply them to normal-form games and stochastic games with a finite number of stages. The performance of the proposed approach is first demonstrated on repeated rock-paper-scissors games. Subsequently, the approach is evaluated in a humanrobot table-tennis setting where the robot player learns to prepare to return a served ball. By modeling the human players, the robot chooses a forehand, backhand or middle preparation pose before they serve. The learned strategies can exploit the opponent’s preferences, leading to a higher rate of successful returns.
introduce a new modeling technique that adaptively balances exploitability and risk reduction. An opponent’s strategy is modeled with a set of possible strategies that contain the actual strategy with a high probability. The algorithm is safe as the expected payoff is above the minimax payoff with a high probability, and can exploit the opponents’ preferences when sufficient observations have been obtained. We apply them to normal-form games and stochastic games with a finite number of stages. The performance of the proposed approach is first demonstrated on repeated rock-paper-scissors games. Subsequently, the approach is evaluated in a humanrobot table-tennis setting where the robot player learns to prepare to return a served ball. By modeling the human players, the robot chooses a forehand, backhand or middle preparation pose before they serve. The learned strategies can exploit the opponent’s preferences, leading to a higher rate of successful returns.
copied = false, 2000);
">
@conference{Wang-2011-107891,
author = {Wang, Z. And Boularias, A. And Muelling, K. And Peters, J.},
title = {Balancing Safety and Exploitability in Opponent Modeling},
booktitle = {Proceedings of 25th AAAI Conference on Artificial Intelligence (AAAI '11)},
year = {2011},
month = {July},
pages = {1515 - 1520},
}
author = {Wang, Z. And Boularias, A. And Muelling, K. And Peters, J.},
title = {Balancing Safety and Exploitability in Opponent Modeling},
booktitle = {Proceedings of 25th AAAI Conference on Artificial Intelligence (AAAI '11)},
year = {2011},
month = {July},
pages = {1515 - 1520},
}