Affordable Access

Publisher Website

Spiking neural network model of free-energy-based reinforcement learning

BMC Neuroscience
Springer (Biomed Central Ltd.)
Publication Date
DOI: 10.1186/1471-2202-12-s1-p244
  • Poster Presentation
  • Computer Science
  • Mathematics


Spiking neural network model of free-energy-based reinforcement learning POSTER PRESENTATION Open Access Spiking neural network model of free-energy-based reinforcement learning Takashi Nakano*, Makoto Otsuka From Twentieth Annual Computational Neuroscience Meeting: CNS*2011 Stockholm, Sweden. 23-28 July 2011 Reinforcement learning is a theoretical framework for learning how to act in an unknown environment through trial and errors. One reinforcement learning framework proposed by Sallans and Hinton [1], which we call free-energy-based reinforcement learning (FERL), possesses many desirable characteristics such as an ability to deal with high-dimensional sensory inputs and goal-directed representation learning, and neurally plausible characteristics such as population coding of action-value and a Hebbian learning rule modulated by reward prediction errors. These characteristics imply that FERL is possibly implemented in the brain. In order to understand the neural implementation of the reinforcement learning and pursue the neural plausibil- ity of FERL, we implemented FERL in a more realistic spiking neural network than binary stochastic neurons. An FERL framework uses a restricted Boltzmann machine (RBM) as a building block. The RBM is an energy-based statistical model with binary nodes sepa- rated in visible and hidden layers. In the RBM, due to its connectivity, the posterior distribution over hidden given visible nodes is statistically decoupled, yielding the simple computation of posterior distribution [2]. An RBM is implemented using a spiking neural network with leaky integrate- and-fire neurons. The network is composed of state, action, and hidden layers. The state * Correspondence: [email protected] Okinawa Institute of Science and Technology, Onna, Okinawa 904-0412, Japan Figure 1 Performance of the spiking neural network. A. The free-energies estimated by both the spiking neural network and the original RBM. They are highly correlated (correlation coefficient, r =

There are no comments yet on this publication. Be the first to share your thoughts.