Send to

Choose Destination
See comment in PubMed Commons below
Eval Rev. 2009 Dec;33(6):539-67. doi: 10.1177/0193841X09350590.

An approach for addressing the multiple testing problem in social policy impact evaluations.

Author information

  • 1Mathematica Policy Research, Inc., Princeton, New Jersey 08543, USA.


In social policy evaluations, the multiple testing problem occurs due to the many hypothesis tests that are typically conducted across multiple outcomes and subgroups, which can lead to spurious impact findings. This article discusses a framework for addressing this problem that balances Types I and II errors. The framework involves specifying confirmatory and exploratory analyses in study protocols, delineating confirmatory outcome domains, conducting t tests on composite domain outcomes, and applying multiplicity corrections to composites across domains to obtain summative impact evidence. The article presents statistical background and discusses multiplicity issues for subgroup analyses, designs with multiple treatments, and reporting.

[PubMed - indexed for MEDLINE]
PubMed Commons home

PubMed Commons

How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for HighWire
    Loading ...
    Support Center