Affordable Access

Attainability of Boundary Points under Reinforcement Learning

Authors
Disciplines
  • Economics
  • Mathematics

Abstract

noncon51.dvi Attainability of Boundary Points under Reinforcement Learning Ed Hopkins∗ Department of Economics University of Edinburgh Edinburgh EH8 9JY, UK Martin Posch Department of Medical Statistics University of Vienna Vienna 1090, Austria July, 2003 Abstract This paper investigates the properties of the most common form of reinforce- ment learning (the “basic model” of Erev and Roth, American Economic Review, 88, 848-881, 1998). Stochastic approximation theory has been used to analyse the local stability of fixed points under this learning process. However, as we show, when such points are on the boundary of the state space, for example, pure strat- egy equilibria, standard results from the theory of stochastic approximation do not apply. We offer what we believe to be the correct treatment of boundary points, and provide a new and more general result: this model of learning converges with zero probability to fixed points which are unstable under the Maynard Smith or adjusted version of the evolutionary replicator dynamics. For two player games these are the fixed points that are linearly unstable under the standard replicator dynamics. Journal of Economic Literature classification numbers: C72, C73, D83 Keywords: Learning in games, reinforcement learning, stochastic approximation, replicator dynamics. ∗We thank Josef Hofbauer for his advice and encouragement. We are also grateful to Alan Beggs and two anonymous referees for helpful comments. Errors remain our own. Both au- thors contributed equally to this work. [email protected], http://homepages.ed.ac.uk/ehk/; Mar- [email protected], http://www.univie.ac.at/medstat/posch/posch.html 1 Introduction Whilst equilibrium analysis has been the mainstay of economic theory for many years, economists have more recently turned to non-equilibrium explanations of human be- haviour based on learning models. This approach has found considerable success in explaining how people behave in economic experiments (Roth a

There are no comments yet on this publication. Be the first to share your thoughts.