U.S. flag

An official website of the United States government

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Cover of Reporting the Findings of Updated Systematic Reviews of Comparative Effectiveness

Reporting the Findings of Updated Systematic Reviews of Comparative Effectiveness

How Do Users Want To View New Information?

Research White Papers

Investigators: , PhD, , MD, PhD, , PhD, and , BA.

Author Information and Affiliations
Rockville (MD): Agency for Healthcare Research and Quality (US); .
Report No.: 13-EHC093-EF

Structured Abstract


To remain useful, comparative effectiveness reviews (CERs) and other systematic reviews require periodic updating. Although several studies have been conducted assessing when and how to update, no research has been conducted on optimal formats for presenting the results to users. The aim of the present study was to gather the input of various users of CERS regarding the usability of a range of formatting methods for showing the changes from the original to the update report.


Using the executive summaries of a comparative effectiveness review our Evidence-based Practice Center conducted in 2001 and the update review we conducted in 2008, we initially created five different versions of the update summary. Each succeeding version used a different format to show changes from the original to the update report (e.g., new and retired Key Questions, changes in search strategies and inclusion/exclusion criteria) and changes in the findings. To test the five differently formatted summaries, we identified several categories of users of CERs, convened an informal virtual focus group comprising various users, and asked them to evaluate the summaries on several dimensions, first via an email questionnaire and then in a group conference call where we presented the results of the questionnaire. Based on group feedback, we created two additional versions and tested them in a second focus group and among a third small group. The rationales for the selection of formats were two-fold: to imitate, and thus evaluate, the formats used by several organizations whose role is to conduct systematic reviews and updates and to create and test novel formats in response to users' suggestions.


Policymakers who rely on CERs and other systematic reviews as the basis for policy (including health insurance companies, health care organizations, research funders, and guideline makers) expressed the need to see changes in review process as well as outcomes clearly marked, (with changes in outcomes and conclusions preferably shown in graphic form), while at the same time having access to the entire set of data and the analyses on which the conclusions were based. The small group of clinicians preferred to see the skeleton of the report (Key Questions, conceptual framework, inclusion/exclusion criteria) as well as the outcomes and conclusions presented entirely in graphic form for ease of reading.


Different users of CERs clearly have different information needs. Yet whereas policymakers need access to the entire data set and analyses that comprise a systematic review (the original and the update), all users benefit from summaries that clearly show what changed in as succinct a format as possible, preferable in graphic form.

Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services1, Contract No. 290-2007-10062-I. Prepared by: Southern California Evidence-based Practice Center, RAND Corporation, Santa Monica, CA

Suggested citation:

Newberry SJ, Shekelle PG, Vaiana M, Motala A. Reporting the Findings of Updated Systematic Reviews of Comparative Effectiveness: How Do Users Want To View New Information? Research White Paper (Prepared by the Southern California Evidence-based Practice Center under Contract No. 290-2007-10062-I). AHRQ Publication No. 13-EHC093-EF. Rockville, MD: Agency for Healthcare Research and Quality. May 2013. www.effectivehealthcare.ahrq.gov/reports/final.cfm.

This report is based on research conducted by the Southern California Evidence-based Practice Center, RAND Corporation under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-2007-10062-I). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.

The information in this report is intended to help health care decisionmakers—patients and clinicians, health system leaders, and policymakers, among others—make well-informed decisions and thereby improve the quality of health care services. This report is not intended to be a substitute for the application of clinical judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical reference and in conjunction with all other pertinent information, i.e., in the context of available resources and circumstances presented by individual patients.

This report may be used, in whole or in part, as the basis for development of clinical practice guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of such derivative products may not be stated or implied.

None of the investigators have any affiliation or financial involvement that conflicts with the material presented in this report.


540 Gaither Road, Rockville, MD 20850; www‚Äč.ahrq.gov

Bookshelf ID: NBK143984PMID: 23785728


  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (855K)

Similar articles in PubMed

See reviews...See all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...