Format

Send to

Choose Destination
See comment in PubMed Commons below
Syst Rev. 2017 Feb 15;6(1):28. doi: 10.1186/s13643-017-0422-x.

Comparison of two independent systematic reviews of trials of recombinant human bone morphogenetic protein-2 (rhBMP-2): the Yale Open Data Access Medtronic Project.

Low J1, Ross JS2,3,4,5, Ritchie JD2, Gross CP3,4, Lehman R6, Lin H2,7, Fu R8, Stewart LA9, Krumholz HM10,11,12,13.

Author information

1
Department of Internal Medicine, Yale School of Medicine, 333 Cedar Street, New Haven, CT, 06510, USA.
2
Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, 1 Church Street, Suite 200, New Haven, CT, 06510, USA.
3
Section of General Internal Medicine, Department of Internal Medicine, Yale School of Medicine, P.O. Box 208093, New Haven, CT, 06510, USA.
4
Robert Wood Johnson Foundation Clinical Scholars Program, P.O. Box 208088, New Haven, CT, 06510, USA.
5
Department of Health Policy and Management, Yale School of Public Health, P.O. Box 208034, New Haven, CT, 06520, USA.
6
Department of Primary Care Health Sciences, Oxford University, 33 St Ebbes Street, Oxford, OX1 1PU, UK.
7
Department of Biostatistics, Yale School of Public Health, P.O. Box 208034, New Haven, CT, 06520, USA.
8
Department of Public Health and Preventive Medicine, Pacific Northwest Evidence-based Practice Center, Oregon Health & Science University, 3181 S.W. Sam Jackson Park Road, Portland, OR, 97239-3098, USA.
9
Centre for Reviews and Dissemination, University of York, Heslington, York, YO10 5DD, UK.
10
Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, 1 Church Street, Suite 200, New Haven, CT, 06510, USA. harlan.krumholz@yale.edu.
11
Robert Wood Johnson Foundation Clinical Scholars Program, P.O. Box 208088, New Haven, CT, 06510, USA. harlan.krumholz@yale.edu.
12
Department of Health Policy and Management, Yale School of Public Health, P.O. Box 208034, New Haven, CT, 06520, USA. harlan.krumholz@yale.edu.
13
Section of Cardiovascular Medicine, Department of Internal Medicine, Yale School of Medicine, P.O. Box 208093, New Haven, CT, 06510, USA. harlan.krumholz@yale.edu.

Abstract

BACKGROUND:

It is uncertain whether the replication of systematic reviews, particularly those with the same objectives and resources, would employ similar methods and/or arrive at identical findings. We compared the results and conclusions of two concurrent systematic reviews undertaken by two independent research teams provided with the same objectives, resources, and individual participant-level data.

METHODS:

Two centers in the USA and UK were each provided with participant-level data on 17 multi-site clinical trials of recombinant human bone morphogenetic protein-2 (rhBMP-2). The teams were blinded to each other's methods and findings until after publication. We conducted a retrospective structured comparison of the results of the two systematic reviews. The main outcome measures included (1) trial inclusion criteria; (2) statistical methods; (3) summary efficacy and risk estimates; and (4) conclusions.

RESULTS:

The two research teams' meta-analyses inclusion criteria were broadly similar but differed slightly in trial inclusion and research methodology. They obtained similar results in summary estimates of most clinical outcomes and adverse events. Center A incorporated all trials into summary estimates of efficacy and harms, while Center B concentrated on analyses stratified by surgical approach. Center A found a statistically significant, but small, benefit whereas Center B reported no advantage. In the analysis of harms, neither showed an increased cancer risk at 48 months, although Center B reported a significant increase at 24 months. Conclusions reflected these differences in summary estimates of benefit balanced with small but potentially important risk of harm.

CONCLUSIONS:

Two independent groups given the same research objectives, data, resources, funding, and time produced broad general agreement but differed in several areas. These differences, the importance of which is debatable, indicate the value of the availability of data to allow for more than a single approach and a single interpretation of the data.

SYSTEMATIC REVIEW REGISTRATION:

PROSPERO CRD42012002040 and CRD42012001907 .

KEYWORDS:

Data interpretation; Data sharing; Meta-analysis; Systematic review

PubMed Commons home

PubMed Commons

0 comments
How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for BioMed Central Icon for PubMed Central
    Loading ...
    Support Center