Format

Send to

Choose Destination
Philos Trans R Soc Lond B Biol Sci. 2014 Nov 5;369(1655). pii: 20130478. doi: 10.1098/rstb.2013.0478.

The algorithmic anatomy of model-based evaluation.

Author information

1
Department of Psychology and Center for Neural Science, New York University, 4 Washington Place Suite 888, New York, NY 10003, USA.
2
Gatsby Computational Neuroscience Unit, UCL, 17 Queen Square, London WC1N 3AR, UK dayan@gatsby.ucl.ac.uk.

Abstract

Despite many debates in the first half of the twentieth century, it is now largely a truism that humans and other animals build models of their environments and use them for prediction and control. However, model-based (MB) reasoning presents severe computational challenges. Alternative, computationally simpler, model-free (MF) schemes have been suggested in the reinforcement learning literature, and have afforded influential accounts of behavioural and neural data. Here, we study the realization of MB calculations, and the ways that this might be woven together with MF values and evaluation methods. There are as yet mostly only hints in the literature as to the resulting tapestry, so we offer more preview than review.

KEYWORDS:

Monte Carlo tree search; model-based reasoning; model-free reasoning; orbitofrontal cortex; reinforcement learning; striatum

PMID:
25267820
PMCID:
PMC4186231
DOI:
10.1098/rstb.2013.0478
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Atypon Icon for PubMed Central
Loading ...
Support Center