Format

Send to

Choose Destination
J Exp Psychol Gen. 2014 Oct;143(5):2020-45. doi: 10.1037/xge0000014. Epub 2014 Aug 11.

Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli.

Author information

1
Department of Psychology and Neuroscience.
2
Department of Psychology, University of Connecticut.
3
Department of Psychology.

Abstract

Researchers designing experiments in which a sample of participants responds to a sample of stimuli are faced with difficult questions about optimal study design. The conventional procedures of statistical power analysis fail to provide appropriate answers to these questions because they are based on statistical models in which stimuli are not assumed to be a source of random variation in the data, models that are inappropriate for experiments involving crossed random factors of participants and stimuli. In this article, we present new methods of power analysis for designs with crossed random factors, and we give detailed, practical guidance to psychology researchers planning experiments in which a sample of participants responds to a sample of stimuli. We extensively examine 5 commonly used experimental designs, describe how to estimate statistical power in each, and provide power analysis results based on a reasonable set of default parameter values. We then develop general conclusions and formulate rules of thumb concerning the optimal design of experiments in which a sample of participants responds to a sample of stimuli. We show that in crossed designs, statistical power typically does not approach unity as the number of participants goes to infinity but instead approaches a maximum attainable power value that is possibly small, depending on the stimulus sample. We also consider the statistical merits of designs involving multiple stimulus blocks. Finally, we provide a simple and flexible Web-based power application to aid researchers in planning studies with samples of stimuli.

PMID:
25111580
DOI:
10.1037/xge0000014
[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for American Psychological Association
Loading ...
Support Center