Display Settings:

Format

Send to:

Choose Destination
See comment in PubMed Commons below
Nat Neurosci. 2009 May;12(5):535-40. doi: 10.1038/nn.2303.

Circular analysis in systems neuroscience: the dangers of double dipping.

Author information

  • 1Laboratory of Brain and Cognition, US National Institute of Mental Health, Bethesda, Maryland, USA. nikokriegeskorte@gmail.com

Abstract

A neuroscientific experiment typically generates a large amount of data, of which only a small fraction is analyzed in detail and presented in a publication. However, selection among noisy measurements can render circular an otherwise appropriate analysis and invalidate results. Here we argue that systems neuroscience needs to adjust some widespread practices to avoid the circularity that can arise from selection. In particular, 'double dipping', the use of the same dataset for selection and selective analysis, will give distorted descriptive statistics and invalid statistical inference whenever the results statistics are not inherently independent of the selection criteria under the null hypothesis. To demonstrate the problem, we apply widely used analyses to noise data known to not contain the experimental effects in question. Spurious effects can appear in the context of both univariate activation analysis and multivariate pattern-information analysis. We suggest a policy for avoiding circularity.

PMID:
19396166
[PubMed - indexed for MEDLINE]
PMCID:
PMC2841687
Free PMC Article
PubMed Commons home

PubMed Commons

0 comments
How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for Nature Publishing Group Icon for PubMed Central
    Loading ...
    Write to the Help Desk