Format

Send to

Choose Destination
Perspect Psychol Sci. 2016 Jul;11(4):539-44. doi: 10.1177/1745691616646366.

What Should Researchers Expect When They Replicate Studies? A Statistical View of Replicability in Psychological Science.

Author information

1
Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health.
2
Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health jtleek@gmail.com.

Abstract

A recent study of the replicability of key psychological findings is a major contribution toward understanding the human side of the scientific process. Despite the careful and nuanced analysis reported, the simple narrative disseminated by the mass, social, and scientific media was that in only 36% of the studies were the original results replicated. In the current study, however, we showed that 77% of the replication effect sizes reported were within a 95% prediction interval calculated using the original effect size. Our analysis suggests two critical issues in understanding replication of psychological studies. First, researchers' intuitive expectations for what a replication should show do not always match with statistical estimates of replication. Second, when the results of original studies are very imprecise, they create wide prediction intervals-and a broad range of replication effects that are consistent with the original estimates. This may lead to effects that replicate successfully, in that replication results are consistent with statistical expectations, but do not provide much information about the size (or existence) of the true effect. In this light, the results of the Reproducibility Project: Psychology can be viewed as statistically consistent with what one might expect when performing a large-scale replication experiment.

KEYWORDS:

Reproducibility Project: Psychology; p values; prediction intervals; replication; reproducibility

PMID:
27474140
PMCID:
PMC4968573
DOI:
10.1177/1745691616646366
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Atypon Icon for PubMed Central
Loading ...
Support Center