• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of qualsafetyQuality and Safety in Health CareCurrent TOCInstructions for authors
Qual Saf Health Care. Feb 2003; 12(1): 18–23.
PMCID: PMC1743672

Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project

Abstract

Background: International interest in clinical practice guidelines has never been greater but many published guidelines do not meet the basic quality requirements. There have been renewed calls for validated criteria to assess the quality of guidelines.

Objective: To develop and validate an international instrument for assessing the quality of the process and reporting of clinical practice guideline development.

Methods: The instrument was developed through a multi-staged process of item generation, selection and scaling, field testing, and refinement procedures. 100 guidelines selected from 11 participating countries were evaluated independently by 194 appraisers with the instrument. Following refinement the instrument was further field tested on three guidelines per country by a new set of 70 appraisers.

Results: The final version of the instrument contained 23 items grouped into six quality domains with a 4 point Likert scale to score each item (scope and purpose, stakeholder involvement, rigour of development, clarity and presentation, applicability, editorial independence). 95% of appraisers found the instrument useful for assessing guidelines. Reliability was acceptable for most domains (Cronbach's alpha 0.64–0.88). Guidelines produced as part of an established guideline programme had significantly higher scores on editorial independence and, after the publication of a national policy, had significantly higher quality scores on rigour of development (p<0.005). Guidelines with technical documentation had higher scores on that domain (p<0.0001).

Conclusions: This is the first time an appraisal instrument for clinical practice guidelines has been developed and tested internationally. The instrument is sensitive to differences in important aspects of guidelines and can be used consistently and easily by a wide range of professionals from different backgrounds. The adoption of common standards should improve the consistency and quality of the reporting of guideline development worldwide and provide a framework to encourage international comparison of clinical practice guidelines.

Full Text

The Full Text of this article is available as a PDF (107K).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.
  • Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999 Feb 20;318(7182):527–530. [PMC free article] [PubMed]
  • Shekelle PG, Woolf SH, Eccles M, Grimshaw J. Clinical guidelines: developing guidelines. BMJ. 1999 Feb 27;318(7183):593–596. [PMC free article] [PubMed]
  • Grimshaw J, Russell I. Achieving health gain through clinical guidelines. I: Developing scientifically valid guidelines. Qual Health Care. 1993 Dec;2(4):243–248. [PMC free article] [PubMed]
  • Grilli R, Magrini N, Penna A, Mura G, Liberati A. Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet. 2000 Jan 8;355(9198):103–106. [PubMed]
  • Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999 May 26;281(20):1900–1905. [PubMed]
  • Grol R, Dalhuijsen J, Thomas S, Veld C, Rutten G, Mokkink H. Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ. 1998 Sep 26;317(7162):858–861. [PMC free article] [PubMed]
  • Thomson R, Lavender M, Madhok R. How to ensure that guidelines are effective. BMJ. 1995 Jul 22;311(6999):237–242. [PMC free article] [PubMed]
  • Bland JM, Altman DG. Cronbach's alpha. BMJ. 1997 Feb 22;314(7080):572–572. [PMC free article] [PubMed]
  • Hayward RS, Wilson MC, Tunis SR, Bass EB, Guyatt G. Users' guides to the medical literature. VIII. How to use clinical practice guidelines. A. Are the recommendations valid? The Evidence-Based Medicine Working Group. JAMA. 1995 Aug 16;274(7):570–574. [PubMed]
  • Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet. 1998 Aug 22;352(9128):609–613. [PubMed]
  • Cluzeau FA, Littlejohns P. Appraising clinical practice guidelines in England and Wales: the development of a methodologic framework and its application to policy. Jt Comm J Qual Improv. 1999 Oct;25(10):514–521. [PubMed]
  • Guideline development in Europe. An international comparison. Int J Technol Assess Health Care. 2000 Autumn;16(4):1039–1049. [PubMed]
  • Moher D, Schulz KF, Altman D. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001 Apr 18;285(15):1987–1991. [PubMed]
  • Moher D, Jones A, Lepage L. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA. 2001 Apr 18;285(15):1992–1995. [PubMed]

Articles from Quality & Safety in Health Care are provided here courtesy of BMJ Group

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • Cited in Books
    Cited in Books
    PubMed Central articles cited in books
  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...