Send to

Choose Destination
J Clin Epidemiol. 2018 Oct;102:50-62. doi: 10.1016/j.jclinepi.2018.05.024. Epub 2018 Jun 4.

Inadequate diversity of information resources searched in US-affiliated systematic reviews and meta-analyses: 2005-2016.

Author information

Department of Quantitative Health Sciences, UMass Medical School, Worcester, MA, USA.
Department of Pharmacology, ESI Medical College, Kolkata, West Bengal, India.
Department of Computer Science, University of Massachusetts Lowell, Lowell, MA, USA; Department of Medicine, UMass Medical School, Worcester, MA, USA; Department of Computer Science, University of Massachusetts Amherst, Amherst, MA, USA; Center for Healthcare Organization & Implementation Research, Edith Nourse Rogers Memorial Veterans Hospital, Bedford, MA, USA. Electronic address:



Systematic reviews and meta-analyses (SRMAs) rely upon comprehensive searches into diverse resources that catalog primary studies. However, since what constitutes a comprehensive search is unclear, we examined trends in databases searched from 2005-2016, surrounding the publication of search guidelines in 2013, and associations between resources searched and evidence of publication bias in SRMAs involving human subjects.


To ensure comparability of included SRMAs over the 12 years in the face of a near 100-fold increase of international SRMAs (mainly genetic studies from China) during this period, we focused on USA-affiliated SRMAs, manually reviewing 100 randomly selected SRMAs from those published in each year. After excluding articles (mainly for inadequate detail or out-of-scope methods), we identified factors associated with the databases searched, used network analysis to see which resources were simultaneously searched, and used logistic regression to link information sources searched with a lower chance of finding publication bias.


Among 817 SRMA articles studied, the common resources used were Medline (95%), EMBASE (44%), and Cochrane (41%). Methods journal SRMAs were most likely to use registries and grey literature resources. We found substantial co-searching of resources with only published materials, and not complemented by searches of registries and the grey literature. The 2013 guideline did not substantially increase searching of registries and grey literature resources to retrieve primary studies for the SRMAs. When used to augment Medline, Scopus (in all SRMAs) and (in SRMAs with safety outcomes) were negatively associated with publication bias.


Even SRMAs that search multiple sources tend to search similar resources. Our study supports searching Scopus and CTG in addition to Medline to reduce the chance of publication bias.


Evidence synthesis; Grey literature; Literature databases; Meta-analysis; Publication bias; Systematic review; Trial registries

Supplemental Content

Full text links

Icon for Elsevier Science Icon for PubMed Central
Loading ...
Support Center