• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of qualsafetyQuality and Safety in Health CareCurrent TOCInstructions for authors
Qual Saf Health Care. Oct 2005; 14(5): 347–351.
PMCID: PMC1744077

Handling over-dispersion of performance indicators

Abstract

Objectives: A problem can arise when a performance indicator shows substantially more variability than would be expected by chance alone, since ignoring such "over-dispersion" could lead to a large number of institutions being inappropriately classified as "abnormal". A number of options for handling this phenomenon are investigated, ranging from improved risk stratification to fitting a statistical model that robustly estimates the degree of over-dispersion.

Design: Retrospective analysis of publicly available data on survival following coronary artery bypass grafts, emergency readmission rates, and teenage pregnancies.

Setting: NHS trusts in England.

Results: Funnel plots clearly show the influence of the method chosen for dealing with over-dispersion on the "banding" a trust receives. Both multiplicative and additive approaches are feasible and give intuitively reasonable results, but the additive random effects formulation appears to have a stronger conceptual foundation.

Conclusion: A random effects model may offer a reasonable solution. This method has now been adopted by the UK Healthcare Commission in their derivation of star ratings.

Full Text

The Full Text of this article is available as a PDF (199K).

Supplementary Material

[Web-only appendix]

Selected References

These references are in PubMed. This may not be the complete list of references from this article.
  • Stark J, Gallivan S, Lovegrove J, Hamilton JR, Monro JL, Pollock JC, Watterson KG. Mortality rates after surgery for congenital heart defects in children and surgeons' performance. Lancet. 2000 Mar 18;355(9208):1004–1007. [PubMed]
  • Stark JF, Gallivan S, Davis K, Hamilton JR, Monro JL, Pollock JC, Watterson KG. Assessment of mortality rates for congenital heart defects and surgeons' performance. Ann Thorac Surg. 2001 Jul;72(1):169–175. [PubMed]
  • Spiegelhalter David J. Mortality and volume of cases in paediatric cardiac surgery: retrospective study based on routinely collected data. BMJ. 2002 Feb 2;324(7332):261–263. [PMC free article] [PubMed]
  • Spiegelhalter D. Funnel plots for institutional comparison. Qual Saf Health Care. 2002 Dec;11(4):390–391. [PMC free article] [PubMed]
  • Tekkis Paris P, McCulloch Peter, Steger Adrian C, Benjamin Irving S, Poloniecki Jan D. Mortality control charts for comparing performance of surgical units: validation study using hospital mortality data. BMJ. 2003 Apr 12;326(7393):786–788. [PMC free article] [PubMed]
  • Spiegelhalter David J. Funnel plots for comparing institutional performance. Stat Med. 2005 Apr 30;24(8):1185–1202. [PubMed]
  • Mohammed MA, Cheng KK, Rouse A, Marshall T. Bristol, Shipman, and clinical governance: Shewhart's forgotten lessons. Lancet. 2001 Feb 10;357(9254):463–467. [PubMed]
  • Aylin Paul, Best Nicky, Bottle Alex, Marshall Clare. Following Shipman: a pilot system for monitoring mortality rates in primary care. Lancet. 2003 Aug 9;362(9382):485–491. [PubMed]
  • DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986 Sep;7(3):177–188. [PubMed]
  • Christiansen CL, Morris CN. Improving the statistical approach to health care provider profiling. Ann Intern Med. 1997 Oct 15;127(8 Pt 2):764–768. [PubMed]

Articles from Quality & Safety in Health Care are provided here courtesy of BMJ Group

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • Cited in Books
    Cited in Books
    PubMed Central articles cited in books
  • MedGen
    MedGen
    Related information in MedGen
  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...