Format

Send to

Choose Destination
Perspect Psychol Sci. 2012 Nov;7(6):543-54. doi: 10.1177/1745691612459060.

The Rules of the Game Called Psychological Science.

Author information

1
University of Amsterdam, The Netherlands M.Bakker1@uva.nl.
2
University of Amsterdam, The Netherlands.
3
Tilburg University, The Netherlands.

Abstract

If science were a game, a dominant rule would probably be to collect results that are statistically significant. Several reviews of the psychological literature have shown that around 96% of papers involving the use of null hypothesis significance testing report significant outcomes for their main results but that the typical studies are insufficiently powerful for such a track record. We explain this paradox by showing that the use of several small underpowered samples often represents a more efficient research strategy (in terms of finding p < .05) than does the use of one larger (more powerful) sample. Publication bias and the most efficient strategy lead to inflated effects and high rates of false positives, especially when researchers also resorted to questionable research practices, such as adding participants after intermediate testing. We provide simulations that highlight the severity of such biases in meta-analyses. We consider 13 meta-analyses covering 281 primary studies in various fields of psychology and find indications of biases and/or an excess of significant results in seven. These results highlight the need for sufficiently powerful replications and changes in journal policies.

KEYWORDS:

false positives; power; publication bias; replication; sample size

PMID:
26168111
DOI:
10.1177/1745691612459060

Supplemental Content

Full text links

Icon for Atypon
Loading ...
Support Center