Send to

Choose Destination
Health Policy. 2015 Apr;119(4):511-21. doi: 10.1016/j.healthpol.2014.10.006. Epub 2014 Oct 22.

Inclusion of quasi-experimental studies in systematic reviews of health systems research.

Author information

Department of Global Health, Boston University School of Public Health, Boston, USA; Center for Global Health and Development, Boston University, Boston, USA.
Norwegian Institute of Public Health, Oslo, Norway; Department of Global Health and Population, Harvard T.H. Chan School of Public Health, Boston, USA; Department of Health Management and Health Economics, Institute for Health and Society, University of Oslo, Norway; Harvard Global Health Institute, Harvard University, Cambridge, USA.
Behavior and Health Research Unit, University of Cambridge, Cambridge, UK.
Department of Medicine, University of Ottawa, Canada; Department and Epidemiology and Community Health, University of Ottawa, Canada.
Department of Global Health and Population, Harvard T.H. Chan School of Public Health, Boston, USA; Wellcome Trust Africa Centre for Health and Population Studies, University of KwaZulu-Natal, Mtubatuba, South Africa. Electronic address:


Systematic reviews of health systems research commonly limit studies for evidence synthesis to randomized controlled trials. However, well-conducted quasi-experimental studies can provide strong evidence for causal inference. With this article, we aim to stimulate and inform discussions on including quasi-experiments in systematic reviews of health systems research. We define quasi-experimental studies as those that estimate causal effect sizes using exogenous variation in the exposure of interest that is not directly controlled by the researcher. We incorporate this definition into a non-hierarchical three-class taxonomy of study designs - experiments, quasi-experiments, and non-experiments. Based on a review of practice in three disciplines related to health systems research (epidemiology, economics, and political science), we discuss five commonly used study designs that fit our definition of quasi-experiments: natural experiments, instrumental variable analyses, regression discontinuity analyses, interrupted times series studies, and difference studies including controlled before-and-after designs, difference-in-difference designs and fixed effects analyses of panel data. We further review current practices regarding quasi-experimental studies in three non-health fields that utilize systematic reviews (education, development, and environment studies) to inform the design of approaches for synthesizing quasi-experimental evidence in health systems research. Ultimately, the aim of any review is practical: to provide useful information for policymakers, practitioners, and researchers. Future work should focus on building a consensus among users and producers of systematic reviews regarding the inclusion of quasi-experiments.


Difference-in-difference studies; Instrumental variable analysis; Interrupted time series; Quasi-experimental studies; Regression discontinuity designs; Systematic reviews

[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for Elsevier Science
Loading ...
Support Center