Format

Send to

Choose Destination
BMJ. 2012 Jan 3;344:d7762. doi: 10.1136/bmj.d7762.

Assessment of publication bias, selection bias, and unavailable data in meta-analyses using individual participant data: a database survey.

Author information

1
MRC Midlands Hub for Trials Methodology Research, School of Health and Population Sciences, University of Birmingham, Birmingham B15 2TT, UK.

Abstract

OBJECTIVE:

To examine the potential for publication bias, data availability bias, and reviewer selection bias in recently published meta-analyses that use individual participant data and to investigate whether authors of such meta-analyses seemed aware of these issues.

DESIGN:

In a database of 383 meta-analyses of individual participant data that were published between 1991 and March 2009, we surveyed the 31 most recent meta-analyses of randomised trials that examined whether an intervention was effective. Identification of relevant articles and data extraction was undertaken by one author and checked by another.

RESULTS:

Only nine (29%) of the 31 meta-analyses included individual participant data from "grey literature" (such as unpublished studies) in their primary meta-analysis, and the potential for publication bias was discussed or investigated in just 10 (32%). Sixteen (52%) of the 31 meta-analyses did not obtain all the individual participant data requested, yet five of these (31%) did not mention this as a potential limitation, and only six (38%) examined how trials without individual participant data might affect the conclusions. In nine (29%) of the meta-analyses reviewer selection bias was a potential issue, as the identification of relevant trials was either not stated or based on a more selective, non-systematic approach. Investigation of four meta-analyses containing data from ≥10 trials revealed one with an asymmetric funnel plot consistent with publication bias, and the inclusion of studies without individual participant data revealed additional heterogeneity between trials.

CONCLUSIONS:

Publication, availability, and selection biases are a potential concern for meta-analyses of individual participant data, but many reviewers neglect to examine or discuss them. These issues warn against uncritically viewing any meta-analysis that uses individual participant data as the most reliable. Reviewers should seek individual participant data from all studies identified by a systematic review; include, where possible, aggregate data from any studies lacking individual participant data to consider their potential impact; and investigate funnel plot asymmetry in line with recent guidelines.

PMID:
22214758
DOI:
10.1136/bmj.d7762
[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for HighWire
Loading ...
Support Center