Send to

Choose Destination
Res Synth Methods. 2013 Mar;4(1):12-25. doi: 10.1002/jrsm.1056. Epub 2012 Sep 25.

Issues relating to study design and risk of bias when including non-randomized studies in systematic reviews on the effects of interventions.

Author information

MRC Biostatistics Unit, Cambridge, U.K.
Centre for Reviews and Dissemination, University of York, York, U.K.
Health Services Research Unit, University of Aberdeen, Aberdeen, U.K.
Bristol Heart Institute, University of Bristol, Bristol Royal Infirmary, Bristol, U.K.
Public Health, Epidemiology and Biostatistics, University of Birmingham, Birmingham, U.K.
Community Information and Epidemiological Technologies, Institute of Population Health, University of Ottawa, Ottawa, Canada.
College of Education and Human Development, University of Louisville, Louisville, KY, USA.
Department of Medicine, University of Ottawa.
Department of Epidemiology and Community Medicine, University of Ottawa, Ottawa, Canada.


Non-randomized studies may provide valuable evidence on the effects of interventions. They are the main source of evidence on the intended effects of some types of interventions and often provide the only evidence about the effects of interventions on long-term outcomes, rare events or adverse effects. Therefore, systematic reviews on the effects of interventions may include various types of non-randomized studies. In this second paper in a series, we address how review authors might articulate the particular non-randomized study designs they will include and how they might evaluate, in general terms, the extent to which a particular non-randomized study is at risk of important biases. We offer guidance for describing and classifying different non-randomized designs based on specific features of the studies in place of using non-informative study design labels. We also suggest criteria to consider when deciding whether to include non-randomized studies. We conclude that a taxonomy of study designs based on study design features is needed. Review authors need new tools specifically to assess the risk of bias for some non-randomized designs that involve a different inferential logic compared with parallel group trials.


bias; non-randomized studies; study design; systematic reviews


Supplemental Content

Full text links

Icon for Wiley
Loading ...
Support Center