NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Cover of Comparative Effectiveness Review Methods: Clinical Heterogeneity

Comparative Effectiveness Review Methods: Clinical Heterogeneity

Methods Research Reports

Investigators: , MPH, PhD, , MD, MPH, , MPH, PhD, , MPH, ScD, , BS, , MHA, , MPA, , BA, , PhD, , MD, MPH, , PhD, and , PhD.

RTI International–University of North Carolina Evidence-based Practice Center
Rockville (MD): Agency for Healthcare Research and Quality (US); .
Report No.: 10-EHC070-EF

Structured Abstract


The Agency for Healthcare Research and Quality (AHRQ) funded the RTI International—University of North Carolina at Chapel Hill Evidence-based Practice Center to determine best practices for addressing clinical heterogeneity in systematic reviews (SRs) and comparative effectiveness reviews (CERs). These best practices address critiques from patients, clinicians, policymakers, and others who assert that SRs typically focus on broad populations and, as a result, often lack information relevant to individual patients or patient subgroups.

Data sources and methods:

We used numerous data sources. We abstracted information from guidance documents prepared by U.S. and international organizations engaged in preparing reviews. We searched MEDLINE® to identify studies on how to handle clinical heterogeneity and subgroup analyses. We reviewed more than 120 SRs conducted by AHRQ’s Evidence-based Practice Centers (EPCs), the Cochrane Collaboration, the Drug Effectiveness Review Project, the United Kingdom’s National Institute for Health and Clinical Excellence and others that we identified from the Centre for Reviews and Dissemination Database of Abstracts of Reviews of Effects and Health Technology Assessment. We reviewed peer and public review comments from AHRQ’s Scientific Review Center for three CERs, and we conducted key informant interviews with authors of six SRs prepared by AHRQ’s EPCs or international organizations.


Clinical heterogeneity has been defined as the variation in study population characteristics, coexisting conditions, cointerventions, and outcomes evaluated across studies included in an SR or CER that may influence or modify the magnitude of the intervention measure of effect (e.g., odds ratio, risk ratio, risk difference). Statistical heterogeneity is defined as variability in the observed treatment effects beyond what would be expected by random error. The review organizations we studied varied in their inclusion of factors, in terms of the key questions and analysis that may modify the treatment-outcome association. They tended to give more consideration to demographic factors than to disease factors (e.g., disease severity, risk factors, coexisting disease, or cointerventions). Individual systematic reviewers whom we interviewed preferred a priori identification of effect modifiers to post hoc determination because of the latter’s data-dredging nature and the possibility of type 1 error when many subgroups are evaluated. Many publications that we identified through our literature searches did indicate that analysis of individual patient-level data in meta-analyses does allow better assessment of clinical heterogeneity, but the time, cost, and difficulty in obtaining these data are often prohibitive.


Identifying factors that may influence the treatment-outcome association is important to clinicians and patients because it helps them understand which patients will benefit most, who is least likely to benefit, and who is at greatest risk of experiencing adverse outcomes. Clear evidence-based guidance on addressing clinical heterogeneity in SRs and CERs is not available currently but would be valuable to AHRQ’s EPCs and to others conducting SRs internationally.


Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services1, Contract No. 290-2007-10056-I. Prepared by: RTI International–University of North Carolina Evidence-based Practice Center, Research Triangle Park, North Carolina.

Suggested citation:

West SL, Gartlehner G, Mansfield AJ, Poole C, Tant E, Lenfestey N, Lux LJ, Amoozegar J, Morton SC, Carey TC, Viswanathan M, Lohr KN. Comparative Effectiveness Review Methods: Clinical Heterogeneity. Agency for Healthcare Research and Quality; September 2010. Methods Research Paper. AHRQ Publication No. 10-EHC070-EF. Available at

This report is based on research conducted by the RTI International–University of North Carolina Evidence-based Practice Center(EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-2007-10056-I). The findings and conclusions in this document are those of the author(s), who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.

The information in this report is intended to help clinicians, employers, policymakers, and others make informed decisions about the provision of health care services. This report is intended as a reference and not as a substitute for clinical judgment.

This report may be used, in whole or in part, as the basis for development of clinical practice guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of such derivative products may not be stated or implied.

This document was written with support from the Effective Health Care Program at AHRQ. None of the authors has a financial interest in any of the products discussed in this document. This document is in the public domain and may be used and reprinted without permission except those copyrighted materials noted, for which further reproduction is prohibited without the specific permission of copyright holders.

The investigators have no relevant financial interests in the report. The investigators have no employment, consultancies, honoraria, or stock ownership or options, or royalties from any organization or entity with a financial interest or financial conflict with the subject matter discussed in the report.


540 Gaither Road, Rockville, MD 20850; www​

Bookshelf ID: NBK53310PMID: 21433337


Related information

Similar articles in PubMed

See reviews...See all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...