Format

Send to

Choose Destination
Proc Biol Sci. 2018 Nov 21;285(1891). pii: 20181645. doi: 10.1098/rspb.2018.1645.

Rethinking dopamine as generalized prediction error.

Author information

1
Intramural Research Program of the National Institute on Drug Abuse, NIH, Bethesda, MD, USA.
2
Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, MD, USA.
3
Department of Neuroscience, Johns Hopkins School of Medicine, Baltimore, MD, USA.
4
Department of Psychology and Center for Brain Science, Harvard University, Cambridge, MA, USA gershman@fas.harvard.edu.

Abstract

Midbrain dopamine neurons are commonly thought to report a reward prediction error (RPE), as hypothesized by reinforcement learning (RL) theory. While this theory has been highly successful, several lines of evidence suggest that dopamine activity also encodes sensory prediction errors unrelated to reward. Here, we develop a new theory of dopamine function that embraces a broader conceptualization of prediction errors. By signalling errors in both sensory and reward predictions, dopamine supports a form of RL that lies between model-based and model-free algorithms. This account remains consistent with current canon regarding the correspondence between dopamine transients and RPEs, while also accounting for new data suggesting a role for these signals in phenomena such as sensory preconditioning and identity unblocking, which ostensibly draw upon knowledge beyond reward predictions.

KEYWORDS:

reinforcement learning; successor representation; temporal difference learning

PMID:
30464063
PMCID:
PMC6253385
DOI:
10.1098/rspb.2018.1645
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Atypon Icon for PubMed Central
Loading ...
Support Center