Send to

Choose Destination
Perspect Psychol Sci. 2011 May;6(3):291-8. doi: 10.1177/1745691611406923.

Statistical Evidence in Experimental Psychology: An Empirical Comparison Using 855 t Tests.

Author information

Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands.
Department of Cognitive Sciences, University of California, Irvine.
Department of Psychological Sciences, University of Missouri-Columbia.


Statistical inference in psychology has traditionally relied heavily on p-value significance testing. This approach to drawing conclusions from data, however, has been widely criticized, and two types of remedies have been advocated. The first proposal is to supplement p values with complementary measures of evidence, such as effect sizes. The second is to replace inference with Bayesian measures of evidence, such as the Bayes factor. The authors provide a practical comparison of p values, effect sizes, and default Bayes factors as measures of statistical evidence, using 855 recently published t tests in psychology. The comparison yields two main results. First, although p values and default Bayes factors almost always agree about what hypothesis is better supported by the data, the measures often disagree about the strength of this support; for 70% of the data sets for which the p value falls between .01 and .05, the default Bayes factor indicates that the evidence is only anecdotal. Second, effect sizes can provide additional evidence to p values and default Bayes factors. The authors conclude that the Bayesian approach is comparatively prudent, preventing researchers from overestimating the evidence in favor of an effect.


t test; p value; Bayes factor; effect size; hypothesis testing


Supplemental Content

Full text links

Icon for Atypon
Loading ...
Support Center