Format

Send to

Choose Destination
Psychol Sci. 2017 Sep;28(9):1321-1333. doi: 10.1177/0956797617708288. Epub 2017 Jul 21.

Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems.

Author information

1
1 Department of Psychology, Harvard University.
2
2 Center for Brain Science, Harvard University.

Abstract

Human behavior is sometimes determined by habit and other times by goal-directed planning. Modern reinforcement-learning theories formalize this distinction as a competition between a computationally cheap but inaccurate model-free system that gives rise to habits and a computationally expensive but accurate model-based system that implements planning. It is unclear, however, how people choose to allocate control between these systems. Here, we propose that arbitration occurs by comparing each system's task-specific costs and benefits. To investigate this proposal, we conducted two experiments showing that people increase model-based control when it achieves greater accuracy than model-free control, and especially when the rewards of accurate performance are amplified. In contrast, they are insensitive to reward amplification when model-based and model-free control yield equivalent accuracy. This suggests that humans adaptively balance habitual and planned action through on-line cost-benefit analysis.

KEYWORDS:

cognitive control; decision making; open data; open materials; reinforcement learning

PMID:
28731839
DOI:
10.1177/0956797617708288
[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for Atypon
Loading ...
Support Center