Format

Send to

Choose Destination
Neural Comput. 2017 Feb;29(2):368-393. doi: 10.1162/NECO_a_00917. Epub 2016 Nov 21.

Neural Circuits Trained with Standard Reinforcement Learning Can Accumulate Probabilistic Information during Decision Making.

Author information

1
Medical Research Council Brain Network Dynamics Unit, University of Oxford, Oxford, OX1 3QT, U.K., and Institute of Pharmacy and Molecular Biotechnology, University of Heidelberg, D-69120 Heidelberg, Germany n.kurzawa@stud.uni-heidelberg.de.
2
Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, U.K. christopher.summerfield@psy.ox.ac.uk.
3
Medical Research Council Brain Network Dynamics Unit, University of Oxford, Oxford OX1 3UD, U.K., and Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford OX1 3UD, U.K. rafal.bogacz@ndcn.ox.ac.uk.

Abstract

Much experimental evidence suggests that during decision making, neural circuits accumulate evidence supporting alternative options. A computational model well describing this accumulation for choices between two options assumes that the brain integrates the log ratios of the likelihoods of the sensory inputs given the two options. Several models have been proposed for how neural circuits can learn these log-likelihood ratios from experience, but all of these models introduced novel and specially dedicated synaptic plasticity rules. Here we show that for a certain wide class of tasks, the log-likelihood ratios are approximately linearly proportional to the expected rewards for selecting actions. Therefore, a simple model based on standard reinforcement learning rules is able to estimate the log-likelihood ratios from experience and on each trial accumulate the log-likelihood ratios associated with presented stimuli while selecting an action. The simulations of the model replicate experimental data on both behavior and neural activity in tasks requiring accumulation of probabilistic cues. Our results suggest that there is no need for the brain to support dedicated plasticity rules, as the standard mechanisms proposed to describe reinforcement learning can enable the neural circuits to perform efficient probabilistic inference.

Supplemental Content

Full text links

Icon for Atypon Icon for PubMed Central
Loading ...
Support Center