NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines; Graham R, Mancher M, Miller Wolman D, et al., editors. Clinical Practice Guidelines We Can Trust. Washington (DC): National Academies Press (US); 2011.

Cover of Clinical Practice Guidelines We Can Trust

Clinical Practice Guidelines We Can Trust.

Show details


Clinicians can no longer stay abreast of the rapidly expanding knowledge bases related to health. The number of randomized controlled trials published in MEDLINE (a medical literature database) grew from 5,000 per year in 1978–1985 to 25,000 per year in 1994–2001. Furthermore, contentions that much of the literature may be biased and not applicable to important subsets of target populations have caused its quality to be suspect. Overall, clinicians increasingly are barraged with a vast volume of evidence of uncertain value. Hence, critically appraised and synthesized scientific evidence has become fundamental to clinical practice. At the same time, and particularly under conditions of uncertainty regarding optimal decisions, clinician experiential knowledge and skill (the “art of medicine”) and patient values and preferences remain essential contributors to quality healthcare practice, in a complex interplay with science.

Clinical practice guidelines (CPGs) embody and support the interrelationships among these critical contributors to clinical decision making. Rather than dictating a one-size-fits-all approach to patient care, CPGs are able to enhance clinician and patient decision making by clearly describing and appraising the scientific evidence and reasoning (the likely benefits and harms) behind clinical recommendations, making them relevant to the individual patient encounter.

Although it remains important for CPGs to be evaluated fully for their effectiveness in improving health, when rigorously developed, they have the power to translate the complexity of scientific research findings into recommendations for clinical practice and potentially enhance healthcare quality and outcomes. However, the current state of CPG development has yet to meet this potential.


Clinical practice guidelines are ubiquitous in our healthcare system. The Guidelines International Network database currently lists more than 3,700 guidelines from 39 countries. Its U.S. counterpart, the National Guideline Clearinghouse (NGC), accepted 722 guidelines to its database in 2008 alone, so that its total collection is nearly 2,700. CPG developers and users are characterized by varied organizations such as clinical specialty societies, disease advocacy groups, federal and local agencies, health plans, and commercial companies. However, CPGs suffer from shortcomings in the guideline development process, often compounding limitations inherent in their scientific evidentiary bases. Certain factors commonly undermine the quality and trustworthiness of CPGs. These include variable quality of individual scientific studies; limitations in systematic reviews (SRs) upon which CPGs are based; lack of transparency of development groups’ methodologies (particularly with respect to evidence quality and strength of recommendation appraisals); failure to convene multi-stakeholder, multi-disciplinary guideline development groups, and corresponding non-reconciliation of conflicting guidelines; unmanaged conflicts of interest (COI); and overall failure to use rigorous methodologies in CPG development. Furthermore, evidence supporting clinical decision making and CPG development relevant to subpopulations, such as patients with comorbidities, the socially and economically disadvantaged, and those with rare conditions, is usually absent.

More generally, the quality of CPG development processes and guideline developer adherence to quality standards have remained unsatisfactory and unreliable for decades. Non-standardized development results in substantial variation in clinical recommendations. At the same time, CPGs produced within a structured environment, in which a systematic procedure or “Guidelines for Guidelines” are available to direct production are more likely to be of higher quality. Furthermore, documentation of guideline development is enhanced by developer use of appraisal instruments or tools for systematically assessing and reporting the quality of guideline development processes. While uniformly endorsed standards for clinical practice guidelines development do not yet exist, there appears to be widespread agreement regarding elements basic to quality CPG development.

The concept that quality standards should inform CPG development is a pervasive concern globally, underscored by increasing calls for international standards to hasten rigorous CPG development and appraisal. Although a number and variety of guideline development appraisal tools (e.g., The Appraisal of Guidelines for Research and Evaluation [AGREE] Tool), which point to standards, are available, they inadequately reflect the full range of quality CPG development. They commonly focus on development process and form, with only a small number attending to the quality of evidence and the strength of recommendations. Furthermore, COI, the role of judgment in the derivation of recommendations, prioritization of the recommendations, development group composition, and how to assure patient-centeredness all lack sufficient attention in current standards for CPG development. These appraisal tools also are not designed for prospective application to guideline development. There are no agreed-on standards for prospective enhancement of high-quality, trustworthy clinical practice guidelines.


In 2008, the Institute of Medicine (IOM) report Knowing What Works in Health Care recommended that the U.S. Secretary of Health and Human Services create a public–private program to develop (or endorse) and promote a common set of standards addressing the structure, process, reporting, and final products of systematic reviews of comparative effectiveness research and evidence-based clinical practice guidelines. Congress, through the Medicare Improvements for Patients and Providers Act of 2008, subsequently called on the Secretary to contract with the IOM, through the Agency for Healthcare Research and Quality (AHRQ), to undertake two studies: (1) to “identify the methodological standard for conducting systematic reviews of clinical effectiveness research on health and health care in order to ensure that organizations conducting such reviews have information on methods that are objective, scientifically valid, and consistent,” and (2) to focus on “the best methods used in developing clinical practice guidelines in order to ensure that organizations developing such guidelines have information on approaches that are objective, scientifically valid, and consistent.”

The IOM formed two committees, the Committee on Standards for Systematic Reviews of Comparative Effectiveness Research and the Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, to meet the above requests. The two committees worked independently, but in coordination with each other, because the topics were related. While the SR committee attended exclusively to methods for SR development, from formulation of the research question to derivation of the final report, the CPG committee worked from the premise that SRs reflecting the methodological standard, as defined by the SR committee, are instrumental to a trustworthy guideline development process.

The CPG committee defined “standard” as a process, action, or procedure for developing CPGs that is deemed essential to producing scientifically valid, transparent, and reproducible results. The committee examined existing standards for guidelines development, assessing whether any would ensure development of trustworthy clinical practice guidelines. Special attention was given to standards incorporating systems for appraising quality of evidence and strength of recommendations. The committee also considered methods for modifying CPGs for patients with multiple conditions; ways to reduce the number of overlapping guidelines and harmonize CPGs on the same topic; strategies to promote and evaluate adoption of development standards and trustworthy CPGs; means to distinguish trustworthy CPGs; and procedures for identifying guideline recommendations potentially appropriate for measuring the quality of healthcare systems or clinicians.


The literature assessing the best methods for guideline development has evolved dramatically in the 20 years since the IOM’s first report on the subject, Clinical Practice Guidelines: Directions for a New Program, which defined CPGs as “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.” The committee saw the need to update this definition, in accordance with the AHRQ contract, and to better reflect current consensus on what constitutes a CPG, including aspects of guideline development that the committee believes are defining characteristics. The new definition is as follows: Clinical practice guidelines are statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options.

To be trustworthy, guidelines should

  • be based on a systematic review of the existing evidence;
  • be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups;
  • consider important patient subgroups and patient preferences, as appropriate;
  • be based on an explicit and transparent process that minimizes distortions, biases, and conflicts of interest;
  • provide a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of the recommendations; and
  • be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations.

The new definition provides a clear distinction between the term “CPG” and other forms of clinical guidance derived from widely disparate development processes (e.g., consensus statements, expert advice, and appropriate use criteria). Furthermore, it underscores systematic review and both benefits and harms assessment as essential characteristics of CPGs. Although the committee recognizes that other forms of clinical guidance may have value, addressing those other forms was beyond the scope of this report. Furthermore, the committee is aware that, for many clinical domains, high-quality evidence is lacking or even nonexistent. However, even given such constraints, guideline developers may still produce trustworthy CPGs if their development reflects those committee standards detailed below.


As enumerated below, the committee’s proposed standards reflect the latest literature, expert consensus, and public comment, and the committee hopes they represent an important advance in the newest and best practice standards for CPG development. The committee expects its standards to be assessed for reliability and validity (including applicability), and to evolve as the science and experience demand. The committee has given increased attention to aspects of COI, such as details of guideline development group exclusions; aspects of guideline group composition, including training of patient and consumer representatives in evidence appraisal; the specific nature of working relationships between systematic review teams and CPG developers; critical steps in establishing evidence foundations for clinical recommendations and rating recommendations’ strength; external review of the CPG, including specified mechanisms for ensuring public stakeholder comment; and elements essential to CPG updating, including ongoing monitoring and review of the CPG-relevant scientific literature and factors indicating the need for updates. The eight standards extend across the development process from conception to completion to revision. The standards should provide sufficient flexibility to be applicable to all guideline development groups (whether evidence in a particular clinical area is lacking or abundant), unlike many development methodologies, which are specific to a particular guideline development entity and clinical problem. The committee’s eight proposed standards follow.


1.Establishing Transparency
1.1The processes by which a CPG is developed and funded should be detailed explicitly and publicly accessible.
2.Management of Conflict of Interest (COI)
2.1Prior to selection of the guideline development group (GDG), individuals being considered for membership should declare all interests and activities potentially resulting in COI with development group activity, by written disclosure to those convening the GDG:
Disclosure should reflect all current and planned commercial (including services from which a clinician derives a substantial proportion of income), non-commercial, intellectual, institutional, and patient–public activities pertinent to the potential scope of the CPG.
2.2Disclosure of COIs within GDG:
All COI of each GDG member should be reported and discussed by the prospective development group prior to the onset of his or her work.
Each panel member should explain how his or her COI could influence the CPG development process or specific recommendations.
Members of the GDG should divest themselves of financial investments they or their family members have in, and not participate in marketing activities or advisory boards of, entities whose interests could be affected by CPG recommendations.
Whenever possible GDG members should not have COI.
In some circumstances, a GDG may not be able to perform its work without members who have COIs, such as relevant clinical specialists who receive a substantial portion of their incomes from services pertinent to the CPG.
Members with COIs should represent not more than a minority of the GDG.
The chair or cochairs should not be a person(s) with COI.
Funders should have no role in CPG development.
3.Guideline Development Group Composition
3.1The GDG should be multidisciplinary and balanced, comprising a variety of methodological experts and clinicians, and populations expected to be affected by the CPG.
3.2Patient and public involvement should be facilitated by including (at least at the time of clinical question formulation and draft CPG review) a current or former patient, and a patient advocate or patient/consumer organization representative in the GDG.
3.3Strategies to increase effective participation of patient and consumer representatives, including training in appraisal of evidence, should be adopted by GDGs.
4.Clinical Practice Guideline–Systematic Review Intersection
4.1Clinical practice guideline developers should use systematic reviews that meet standards set by the Institute of Medicine’s Committee on Standards for Systematic Reviews of Comparative Effectiveness Research.
4.2When systematic reviews are conducted specifically to inform particular guidelines, the GDG and systematic review team should interact regarding the scope, approach, and output of both processes.
5.Establishing Evidence Foundations for and Rating Strength of Recommendations
5.1For each recommendation, the following should be provided:
An explanation of the reasoning underlying the recommendation, including
a clear description of potential benefits and harms;
a summary of relevant available evidence (and evidentiary gaps), description of the quality (including applicability), quantity (including com
pleteness), and consistency of the aggregate available evidence;
an explanation of the part played by values, opinion, theory, and clinical experience in deriving the recommendation.
A rating of the level of confidence in (certainty regarding) the evidence underpinning the recommendation
A rating of the strength of the recommendation in light of the preceding bullets
A description and explanation of any differences of opinion regarding the recommendation
6.Articulation of Recommendations
6.1Recommendations should be articulated in a standardized form detailing precisely what the recommended action is, and under what circumstances it should be performed.
6.2Strong recommendations should be worded so that compliance with the recommendation(s) can be evaluated.
7.External Review
7.1External reviewers should comprise a full spectrum of relevant stakeholders, including scientific and clinical experts, organizations (e.g., health care, specialty societies), agencies (e.g., federal government), patients, and representatives of the public.
7.2The authorship of external reviews submitted by individuals and/or organizations should be kept confidential unless that protection has been waived by the reviewer(s).
7.3The GDG should consider all external reviewer comments and keep a written record of the rationale for modifying or not modifying a CPG in response to reviewers’ comments.
7.4A draft of the CPG at the external review stage or immediately following it (i.e., prior to the final draft) should be made available to the general public for comment. Reasonable notice of impending publication should be provided to interested public stakeholders.
8.1The CPG publication date, date of pertinent systematic evidence review, and proposed date for future CPG review should be documented in the CPG.
8.2Literature should be monitored regularly following CPG publication to identify the emergence of new, potentially relevant evidence and to evaluate the continued validity of the CPG.
8.3CPGs should be updated when new evidence suggests the need for modification of clinically important recommendations. For example, a CPG should be updated if new evidence shows that a recommended intervention causes previously unknown substantial harm; that a new intervention is significantly superior to a previously recommended intervention from an efficacy or harms perspective; or that a recommendation can be applied to new populations.

The committee derived several recommendations directly relevant to the ultimate effectiveness of the eight standards in increasing the quality and trustworthiness of CPGs and enhancing healthcare quality and patient outcomes.


The committee views all eight proposed standards as essential elements in the development of trustworthy guidelines. Thus, the committee recommends the following:

To be trustworthy, a clinical practice guideline should comply with proposed standards 1–8.

Optimally, CPG developers should adhere to these proposed standards and CPG users should adopt CPGs compliant with these proposed standards.

Some guideline developers will readily adapt their development process to embrace these eight standards; however, not all developers will be able to do so, and a process of evolutionary adoption over time may be more practical. Although certain standards, such as those directed to patient and public involvement in the CPG development process and external review, may appear particularly resource intensive, strategies to increase effective public participation can minimize this burden.

The committee understands that the uniqueness of guideline development contexts may seemingly preclude certain developers from fully adhering to the standards the committee has proposed. For example, certain clinical areas (e.g., rare malignant tumors) are characterized by an exceptional dearth of scientific literature and an urgent need to deliver patient care. The committee recognizes that developers in this instance may conclude they are unable to comply with Standard 4: “Clinical practice guideline developers should use systematic reviews that meet standards set by the IOM’s Committee on Standards for Systematic Reviews of Comparative Effectiveness Research.” However, SRs that conclude there are no high-quality randomized controlled trials or observational studies on a particular clinical question would still fulfill Standard 4. In all cases, whether evidence is limited or abundant, guideline development groups should comply with the complementary Standard 5: “Establishing evidence foundations for and rating strength of recommendations” by providing a summary of relevant available evidence (and evidentiary gaps), descriptions of the quality (including applicability), quantity (including completeness), and consistency of the aggregate available evidence; an explanation of the part played by values, opinion, theory, or clinical experience in deriving recommendations; a judgment regarding the level of confidence in (certainty regarding) the evidence underpinning the recommendations; and a rating of the strength of recommendations.

For certain clinical areas, such as rare diseases, there may be no disease group or clinical specialty society with resources to develop trustworthy CPGs. In these cases, outside funding assistance could spur the development of needed guidelines. The committee urges organizations desiring to produce such guidelines to coordinate their efforts and pool resources with related organizations. This could also strengthen their efforts to seek support from foundations, government agencies, and other sources without conflict. The Department of Health and Human Services (HHS) should promote the identification of best practices in CPG development, guided by the committee’s proposed standards, and should assist in training individuals in specific technical skills needed in the CPG process, particularly patient and consumer representatives.

Furthermore, to encourage the promulgation and adoption of standards, the committee recommends HHS create a mechanism to identify trustworthy guidelines. Such identification will serve three purposes, as follows:

  • Promote wider adoption of the IOM standards by developers because there will be an advantage attached to CPGs publicly identified as trustworthy.
  • Provide CPG users with an easy guide to identify trustworthy ones.
  • Promote adoption of trustworthy CPGs.

The committee recommends

The Secretary of HHS should establish a public–private mechanism to examine, at the request of developer organizations, the procedures they use to produce their clinical practice guidelines and to certify whether these organizations’ CPG development procedures comply with standards for trustworthy CPGs.

Although AHRQ is not directly involved in CPG development, it does play a vital role in the dissemination and evaluation of guidelines and creation of guideline development methodologies. The NGC is a highly useful guideline dissemination tool. AHRQ should continue to operate this service, and expand its capacities to provide syntheses of recommendations by clinical topic and conduct research on best guideline development practices. As a central repository for all CPGs, the committee does not believe the NGC should be restricted to listing only those CPGs identified as trustworthy. However, the NGC’s contribution may be of questionable value when listing guidelines providing too little information for an informed reader to judge quality and trustworthiness. To be a constructive resource, the NGC should eliminate CPGs for which trustworthiness cannot be determined, and identify the trustworthiness of those retained. Further, the committee recommends that AHRQ pilot-test and assess the reliability and validity of the IOM’s proposed standards, and evaluate their effects on healthcare quality and patient outcomes. The committee expects its standards to evolve as science and experience regarding CPG development demand.

The committee recommends

The Agency for Healthcare Research and Quality (AHRQ) should do the following:

  • Require the National Guideline Clearinghouse (NGC) to provide a clear indication of the extent to which clinical practice guidelines (CPGs) submitted to it adhere to standards for trustworthiness.
  • Conduct research on the causes of inconsistent CPGs, and strategies to encourage their harmonization.
  • Assess the strengths and weaknesses of proposed IOM standards by pilot-test; estimate the validity and reliability of proposed standards; evaluate the effectiveness of interventions to encourage standards’ implementation; and evaluate the effects of standards on CPG development, healthcare quality, and patient outcomes.


Promoting uptake and use of CPGs at the point of care represents a final translation hurdle to move scientific findings into practice. An important guiding principle for promoting adoption of CPGs is that attributes of the CPG (e.g., ease of use, strength of the evidence) as perceived by users and stakeholders are neither stable features nor sure determinants of adoption. Rather it is the interaction among characteristics of the CPG (e.g., specificity, clarity), the intended users (physicians, nurses, pharmacists), and a particular context of practice (e.g., inpatient, ambulatory, long-term care) that determines rate and extent of adoption. Active dissemination and adoption strategies used by implementers to promote use of trustworthy CPGs include academic detailing; audit and feedback and public reporting of performance; opinion leaders; clinical reminders and quick reference guides; payment mechanisms; and shared decision-making aides.

Organizations and health systems can also provide necessary resources, workflow modifications, and infrastructures for CPG implementation by all relevant users, and engage clinician stakeholders in the implementation process. Fundamentally, for trustworthy guidelines to affect quality of care and patient outcomes they must be implemented; hence, the committee offers the following recommendation:

Effective multifaceted implementation strategies targeting all relevant populations affected by CPGs, should be employed by implementers to promote adherence to trustworthy CPGs.

Increased adoption of electronic health records and computer-aided clinical decision support (CDS) will offer unique opportunities to rapidly move clinical knowledge from the scientific literature to the patient encounter. To achieve this goal, guideline developers should structure CPGs to facilitate ready implementation of electronic clinical decision support by health systems (e.g., clinical practices, payers, delivery systems, hospitals). Furthermore, CPG developers should specify definitive and important gaps in scientific evidence for practice recommendations, including those relevant to the target population, to facilitate understanding of potential limitations of clinical decision support. Formal organizational relationships among CPG developers, implementers, and CDS designers are encouraged to align requirements for CDS with the needs and standards of CPG developers. The committee recommends guideline developers and implementers take the following actions to advance this aim:

Guideline developers should structure the format, vocabulary, and content of CPGs (e.g., specific statements of evidence, the target population) to facilitate ready implementation of computer-aided CDS by end-users.

CPG developers, CPG implementers, and CDS designers should collaborate in an effort to align their needs with one another.


Clinical decisions are made under uncertainty. Yet, as medical, biomedical, and health services research advance, translation of scientific evidence increasingly reduces uncertainty in clinical practice. However, requisite to this promise are clinician and patient access to trustworthy clinical practice guidelines informed by high-quality evidence and a guideline development process reflective of best practices. The committee believes the eight standards proposed herein, when embraced by guideline developers, have the capacity to increase quality and trustworthiness of CPGs and ultimately enhance healthcare quality and patient outcomes.

Copyright 2011 by the National Academy of Sciences. All rights reserved.
Bookshelf ID: NBK209538


  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (1.4M)

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...