NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Cover of Methods for Benefit and Harm Assessment in Systematic Reviews

Methods for Benefit and Harm Assessment in Systematic Reviews

Methods Research Reports

Investigators: , MD, MPH, , MD, MPH, , PhD, , MD, MHS, , BSc, , MD, MPH, and , MD, PhD.

The Johns Hopkins University Evidence-based Practice Center
Rockville (MD): Agency for Healthcare Research and Quality (US); .
Report No.: 12(13)-EHC150-EF

Structured Abstract

Introduction:

Systematic reviewers are challenged by how to report and synthesize information about benefits and harms of medical interventions so that decisionmakers with varying preferences can better assess the balance of benefit and harm. Quantitative approaches exist for assessing benefits and harms, but it is unclear whether they are applicable to systematic reviews.

Objectives:

The objectives of this report are: (1) to describe the challenges of quantitative approaches for assessing benefits and harms, (2) to describe methodological characteristics of existing quantitative approaches for assessing benefits and harms, (3) to determine the role of values and preferences in assessing benefits and harms across each step of a systematic review and (4) to formulate principles for assessing benefits and harms in systematic reviews.

Process:

We formed a multidisciplinary team with expertise in clinical medicine, systematic reviews, statistics, and epidemiology. The team reviewed the literature on quantitative approaches for assessing benefits and harms of medical interventions, and held 12 weekly meetings to establish consensus about: 1) the challenges in assessing benefits and harms; 2) the methodological characteristics of approaches that have been used; and 3) the role of values and preferences when assessing benefits and harms in systematic reviews.

The team used that information to formulate principles for analyzing benefits and harms in systematic reviews so that decisionmakers are able to weigh the benefits and harms for a given population. An external panel of experts provided input in this process.

Results:

Our team identified numerous challenges for the assessment of benefits and harms. The main challenges relate to selection of health outcomes important to patients, information asymmetry (e.g., reliable and robust data on benefits with sparse data on harms), and calculation of statistical uncertainty if benefit and harm are put on the same scale using a benefit harm comparison metric, and consideration of patient preferences.

We identified 16 quantitative approaches for the assessment of benefits and harms. Twelve of the methods can be used in a systematic review because the methods can be applied with the types of summary data that are typically reported and do not require individual patient data. Simpler approaches, such as the ratio of the number needed to treat to the number needed to harm, may be suitable for relatively simple decisionmaking contexts where relevant benefit and harm outcomes are few in number and similar in importance. More complex approaches are needed for decisionmaking contexts having a large number of relevant benefits and harms.

For individual-level decisions, values and preferences are key for determining the balance of benefit and harm. Choices are made by decisionmakers that are informed by the preferences of patients and other considerations. These choices, and therefore preferences, have an important role in determining how benefits and harms are assessed in systematic reviews. These choices and preferences also affect how guideline developers frame recommendations, how regulatory bodies make decisions at the population level, and how clinicians, patients, and other end users make decisions at the individual level.

The team formulated principles to conduct comparative assessments of benefits and harms in the context of a systematic review. For example, we recommend that systematic reviews define the decisionmaking context, report the sources of evidence used (e.g., estimates of baseline risks or treatment effects), be explicit about if and how patient preferences are considered, and provide a rationale for choosing a particular quantitative approach for comparative assessment of benefits and harms.

Conclusion:

Quantitative approaches for comparative assessment of benefits and harms have strengths and limitations. The choice of a particular approach depends on the decisionmaking context, the quality and quantity of available data, and the epidemiological-statistical expertise of the systematic review team. A quantitative approach may help to improve the transparency of a review, relative to a qualitative approach, by being explicit about how benefits and harms are estimated and compared. Such transparency may help decisionmakers give proper consideration to complex information about benefits and harms.

Contents

540 Gaither Road, Rockville, MD 20850; http://www​.ahrq.gov

Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services1, Contract No. 290-2007-10061-I. Prepared by: The Johns Hopkins University Evidence-based Practice Center, Baltimore, MD

Suggested citation:

Boyd CM, Singh S, Varadhan R, Weiss CO, Sharma R, Bass EB, Puhan MA. Methods for Benefit and Harm Assessment in Systematic Reviews. Methods Research Report. (Prepared by the Johns Hopkins University Evidence-based Practice Center under contract No. 290-2007-10061-I). AHRQ Publication No. 12(13)-EHC150-EF. Rockville, MD: Agency for Healthcare Research and Quality; November 2012.

This report is based on research conducted by the Johns Hopkins University Evidence-based Practice Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-2007-10061-I). The findings and conclusions in this document are those of the author(s), who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.

The information in this report is intended to help health care decisionmakers—patients and clinicians, health system leaders, and policymakers, among others—make well informed decisions and thereby improve the quality of health care services. This report is not intended to be a substitute for the application of clinical judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical reference and in conjunction with all other pertinent information, i.e., in the context of available resources and circumstances presented by individual patients.

This report may be used, in whole or in part, as the basis for development of clinical practice guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of such derivative products may not be stated or implied.

The investigators have no relevant financial interests in the report. The investigators have no employment, consultancies, honoraria, or stock ownership or options, or royalties from any organization or entity with a financial interest or financial conflict with the subject matter discussed in the report.

1

540 Gaither Road, Rockville, MD 20850; http://www​.ahrq.gov

Bookshelf ID: NBK115750PMID: 23326898
PubReader format: click here to try

Views

Related information

Related citations in PubMed

See reviews...See all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...