Send to

Choose Destination
BMJ. 2017 Jun 8;357:j2490. doi: 10.1136/bmj.j2490.

Evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research study.

Author information

INSERM, U1153, Paris, France
Cochrane France, Paris, France.
Centre d'Épidémiologie Clinique, Hôpital Hôtel Dieu, AP-HP (Assistance Publique des Hôpitaux de Paris), Paris, France.
Faculté de Médecine, Université Paris Descartes, Sorbonne Paris Cité, Paris, France
Clinical Epidemiology Program, Ottawa Hospital Research Institute, School of Epidemiology, Public Health and Preventive Medicine, Canadian EQUATOR Centre, University of Ottawa, Ottawa, ON, Canada.
Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA.
Centre for Statistics in Medicine, Oxford, UK.
Columbia University, Mailman School of Public Health, Department of Epidemiology, New York, NY, USA.


Objective To examine how poor reporting and inadequate methods for key methodological features in randomised controlled trials (RCTs) have changed over the past three decades.Design Mapping of trials included in Cochrane reviews.Data sources Data from RCTs included in all Cochrane reviews published between March 2011 and September 2014 reporting an evaluation of the Cochrane risk of bias items: sequence generation, allocation concealment, blinding, and incomplete outcome data.Data extraction For each RCT, we extracted consensus on risk of bias made by the review authors and identified the primary reference to extract publication year and journal. We matched journal names with Journal Citation Reports to get 2014 impact factors.Main outcomes measures We considered the proportions of trials rated by review authors at unclear and high risk of bias as surrogates for poor reporting and inadequate methods, respectively.Results We analysed 20 920 RCTs (from 2001 reviews) published in 3136 journals. The proportion of trials with unclear risk of bias was 48.7% for sequence generation and 57.5% for allocation concealment; the proportion of those with high risk of bias was 4.0% and 7.2%, respectively. For blinding and incomplete outcome data, 30.6% and 24.7% of trials were at unclear risk and 33.1% and 17.1% were at high risk, respectively. Higher journal impact factor was associated with a lower proportion of trials at unclear or high risk of bias. The proportion of trials at unclear risk of bias decreased over time, especially for sequence generation, which fell from 69.1% in 1986-1990 to 31.2% in 2011-14 and for allocation concealment (70.1% to 44.6%). After excluding trials at unclear risk of bias, use of inadequate methods also decreased over time: from 14.8% to 4.6% for sequence generation and from 32.7% to 11.6% for allocation concealment.Conclusions Poor reporting and inadequate methods have decreased over time, especially for sequence generation and allocation concealment. But more could be done, especially in lower impact factor journals.

[Indexed for MEDLINE]

Conflict of interest statement

Competing interests: DM is a member of the Cochrane Library Oversight Committee and a member of the Cochrane Bias Methods Group. He received funding from the Cochrane Collaboration for an unrelated project. IB and DGA are also members of the Cochrane Bias Methods Group. DGA is supported by Cancer Research UK (C5529). KD is the director of the US Cochrane Center. PR is the director of Cochrane France. The other authors declare no competing interests.

Supplemental Content

Full text links

Icon for HighWire
Loading ...
Support Center