Format

Send to

Choose Destination
Elife. 2018 Mar 7;7. pii: e32962. doi: 10.7554/eLife.32962.

Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior.

Author information

1
Laboratory of Brain and Cognition, National Institutes of Health, Bethesda, United States.
2
Department of Psychology, New York University, New York City, United States.
3
Neuroscience Program, Bates College, Maine, United States.
4
Princeton Neuroscience Institute, Princeton University, Princeton, United States.
5
Stanford Vision Lab, Stanford University, Stanford, United States.
6
Department of Psychology, University of Illinois, Urbana-Champaign, United States.
7
Beckman Institute, University of Illinois, Urbana-Champaign, United States.

Abstract

Inherent correlations between visual and semantic features in real-world scenes make it difficult to determine how different scene properties contribute to neural representations. Here, we assessed the contributions of multiple properties to scene representation by partitioning the variance explained in human behavioral and brain measurements by three feature models whose inter-correlations were minimized a priori through stimulus preselection. Behavioral assessments of scene similarity reflected unique contributions from a functional feature model indicating potential actions in scenes as well as high-level visual features from a deep neural network (DNN). In contrast, similarity of cortical responses in scene-selective areas was uniquely explained by mid- and high-level DNN features only, while an object label model did not contribute uniquely to either domain. The striking dissociation between functional and DNN features in their contribution to behavioral and brain representations of scenes indicates that scene-selective cortex represents only a subset of behaviorally relevant scene information.

TRIAL REGISTRATION:

ClinicalTrials.gov NCT00001360.

KEYWORDS:

behavioral categorization; computational model; deep neural network; fMRI; human; neuroscience; scene perception; variance partitioning

PMID:
29513219
PMCID:
PMC5860866
DOI:
10.7554/eLife.32962
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for eLife Sciences Publications, Ltd Icon for PubMed Central
Loading ...
Support Center