NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Cover of Observational Studies: Empirical Evidence of Their Contributions to Comparative Effectiveness Reviews

Observational Studies: Empirical Evidence of Their Contributions to Comparative Effectiveness Reviews

Research White Papers

Investigators: , MPH, , PhD, and , PhD.

Alberta Evidence-based Practice Center
Rockville (MD): Agency for Healthcare Research and Quality (US); .
Report No.: 14-EHC002-EF

Structured Abstract

Introduction:

Although observational studies are increasingly being used to address gaps in the evidence from randomized controlled trials, the effect they have on the results and conclusions of systematic reviews is unclear. Our objectives were to evaluate: (1) how often observational studies are searched for and included in comparative effectiveness reviews (CERs); (2) the rationale for including or excluding observational studies; (3) how data from observational studies are appraised, analyzed, and graded; and (4) the impact of observational studies on the strength of evidence (SOE) and overall conclusions.

Methods:

In June 2013 we searched the Effective Health Care Program Web site for final reports of CERs. One reviewer screened titles, abstracts, and Key Questions for CERs that examined a therapeutic or preventive intervention provided at an individual patient level. We selected a 25 percent sample of the most recent eligible CERs. Data were extracted by one reviewer and verified by a second reviewer. We extracted the number and type of study designs included and the approaches to quality assessment, presentation of results, and grading the SOE. We identified all comparisons for which both trials and observational studies provided data, and evaluated whether observational studies had an impact on the SOE and conclusions. We applied an RCT filter to the searches to determine the impact on search yield.

Results:

From 129 records we identified 88 eligible CERs. Our final sample included 23 CERs published since November 2012. EPCs searched for observational studies in 20 CERs, of which 18 included a median of 11 (interquartile range: 2, 31) studies. Sixteen CERs incorporated the observational studies in their SOE assessments. We identified 78 comparisons from 12 CERs for which both trials and observational studies provided evidence; observational studies had an impact on SOE and conclusions for 19 (24 percent) of the comparisons. There was considerable diversity across the CERs regarding decisions to include or exclude observational studies, the study designs considered, and the approaches used to appraise, synthesize, and grade the SOE. Applying an RCT filter reduced the search yield by 65 percent (range 39 to 92 percent).

Discussion:

Reporting guidelines and methods guidance relating to observational studies are needed in order to ensure clarity and consistency across Evidence-based Practice Centers. It was not always clear that the inclusion of observational studies added value in light of the additional resources needed to search for, select, appraise, and analyze such studies.

Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services1, Contract No. 290-2013-00013-I. Prepared by: Alberta Evidence-based Practice Center, Edmonton, Alberta, Canada

Suggested citation:

Seida JC, Dryden DM, Hartling L. Observational Studies: Empirical Evidence of Their Contributions to Comparative Effectiveness Reviews. Methods White Paper (Prepared by the University of Alberta Evidence-based Practice Center under Contract No. 290-2013-00013-I.) AHRQ Publication No. 14-EHC002-EF. Rockville, MD: Agency for Healthcare Research and Quality. December 2013. www.effectivehealthcare.ahrq.gov/reports/final.cfm.

This report is based on research conducted by the Alberta Evidence-based Practice Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-2013-00013-I). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.

The information in this report is intended to help health care decisionmakers—patients and clinicians, health system leaders, and policymakers, among others—make well-informed decisions and thereby improve the quality of health care services. This report is not intended to be a substitute for the application of clinical judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical reference and in conjunction with all other pertinent information, i.e., in the context of available resources and circumstances presented by individual patients.

This report may be used, in whole or in part, as the basis for development of clinical practice guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of such derivative products may not be stated or implied.

None of the investigators have any affiliations or financial involvement that conflicts with the material presented in this report.

1

540 Gaither Road, Rockville, MD 20850; www‚Äč.ahrq.gov

Bookshelf ID: NBK174900PMID: 24404632

Views

Related information

Similar articles in PubMed

See reviews...See all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...