Send to

Choose Destination
Res Synth Methods. 2010 Jul;1(3-4):169-84. doi: 10.1002/jrsm.19. Epub 2011 Mar 4.

Meta-research: The art of getting it wrong.

Ioannidis JP1,2,3,4,5,6.

Author information

Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.,
Stanford Prevention Research Center, Stanford University School of Medicine, Stanford, U.S.A..,
Department of Medicine, Tufts University School of Medicine, Boston, U.S.A..,
Center for Genetic Epidemiology and Modeling, ICRHPS, Tufts Medical Center, Boston, U.S.A..,
Genetics/Genomics, Tufts Clinical and Translational Science Institute, Botson, U.S.A..,
Department of Epidemiology, Harvard School of Public Health, Boston, U.S.A..,


Meta-analysis has major strengths, but sometimes it can often lead to wrong and misleading answers. In this SRSM presidential address, I discuss some case studies that exemplify these problems, including examples from meta-analyses of both clinical trials and observational associations. I also discuss issues of effect size estimation, bias (in particular significance-chasing biases), and credibility in meta-research. I examine the factors that affect the credibility of meta-analyses, including magnitude of effects, multiplicity of analyses, scale of data, flexibility of analyses, reporting, and conflicts of interest. Under the current circumstances, a survey of expert meta-analysts attending the SRSM meeting showed that most of them believe that the true effect is practically equally likely to lie within the 95% confidence interval of a meta-analysis or outside of it. Finally, I address the placement of meta-analysis in the wider current research agenda and make a plea for adoption of more prospective meta-designs. In many/most/all fields, all primary original research may be designed, executed, and interpreted as a prospective meta-analysis.


bias; effect size; meta‐analysis; reporting bias


Supplemental Content

Full text links

Icon for Wiley
Loading ...
Support Center