Format

Send to

Choose Destination
Syst Rev. 2018 Dec 21;7(1):242. doi: 10.1186/s13643-018-0915-2.

The risk of bias in observational studies of exposures (ROBINS-E) tool: concerns arising from application to observational studies of exposures.

Author information

1
Charles Perkins Centre and School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, D17, The Hub, 6th floor, Sydney, New South Wales, 2006, Australia. Lisa.bero@sydney.edu.au.
2
Charles Perkins Centre and School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, D17, The Hub, 6th floor, Sydney, New South Wales, 2006, Australia.
3
School of Medical Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia.
4
National Health and Medical Research Council, Canberra, Australia.
5
Department of Ob/Gyn & the Institute for Health Policy Studies, University of California, San Francisco, USA.
6
Department of Health Sciences, California State University, East Bay, San Francisco, USA.
7
School of Pharmacy, University of California, San Francisco, USA.
8
Charles Perkins Centre, The University of Sydney, Sydney, Australia.
9
School of Pharmacy, Faculty of Medicine and Health and Charles Perkins Centre, The University of Sydney, Sydney, Australia.

Abstract

BACKGROUND:

Systematic reviews, which assess the risk of bias in included studies, are increasingly used to develop environmental hazard assessments and public health guidelines. These research areas typically rely on evidence from human observational studies of exposures, yet there are currently no universally accepted standards for assessing risk of bias in such studies. The risk of bias in non-randomised studies of exposures (ROBINS-E) tool has been developed by building upon tools for risk of bias assessment of randomised trials, diagnostic test accuracy studies and observational studies of interventions. This paper reports our experience with the application of the ROBINS-E tool.

METHODS:

We applied ROBINS-E to 74 exposure studies (60 cohort studies, 14 case-control studies) in 3 areas: environmental risk, dietary exposure and drug harm. All investigators provided written feedback, and we documented verbal discussion of the tool. We inductively and iteratively classified the feedback into 7 themes based on commonalities and differences until all the feedback was accounted for in the themes. We present a description of each theme.

RESULTS:

We identified practical concerns with the premise that ROBINS-E is a structured comparison of the observational study being rated to the 'ideal' randomised controlled trial. ROBINS-E assesses 7 domains of bias, but relevant questions related to some critical sources of bias, such as exposure and funding source, are not assessed. ROBINS-E fails to discriminate between studies with a single risk of bias or multiple risks of bias. ROBINS-E is severely limited at determining whether confounders will bias study outcomes. The construct of co-exposures was difficult to distinguish from confounders. Applying ROBINS-E was time-consuming and confusing.

CONCLUSIONS:

Our experience suggests that the ROBINS-E tool does not meet the need for an international standard for evaluating human observational studies for questions of harm relevant to public and environmental health. We propose that a simpler tool, based on empirical evidence of bias, would provide accurate measures of risk of bias and is more likely to meet the needs of the environmental and public health community.

KEYWORDS:

Cochrane; Environment; GRADE; Guidelines; Nutrition; Observational study; Public health guidelines; Quality assessment; Risk of bias; Systematic review

PMID:
30577874
PMCID:
PMC6302384
DOI:
10.1186/s13643-018-0915-2
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for BioMed Central Icon for PubMed Central
Loading ...
Support Center