Format

Send to

Choose Destination
Med Care. 2011 Aug;49(8):724-33. doi: 10.1097/MLR.0b013e31821b3482.

How can health care organizations be reliably compared?: Lessons from a national survey of patient experience.

Author information

1
Cambridge Centre for Health Services Research, Department of Public Health and Primary Care, Institute of Public Health, Cambridge, CB, UK. gl290@medschl.cam.ac.uk

Abstract

BACKGROUND:

Patient experience is increasingly used to assess organizational performance, for example in public reporting or pay-for-performance schemes. Conventional approaches using 95% confidence intervals are commonly used to determine required survey samples or to report performance but these may result in unreliable organizational comparisons.

METHODS:

We analyzed data from 2.2 million patients who responded to the English 2009 General Practice Patient Survey, which included 45 patient experience questions nested within 6 different care domains (access, continuity of care, communication, anticipatory care planning, out-of-hours care, and overall care satisfaction). For each question, unadjusted and case-mix adjusted (for age, sex, and ethnicity) organization-level reliability, and intraclass correlation coefficients were calculated.

RESULTS:

Mean responses per organization ranged from 23 to 256 for questions evaluating primary care practices, and from 1454 to 2758 for questions evaluating out-of-hours care organizations. Adjusted and unadjusted reliability values were similar. Twenty-six questions had excellent reliability (≥0.90). Seven nurse communication questions had very good reliability (≥0.85), but 3 anticipatory care planning questions had lower reliability (<0.70). Reliability was typically <0.70 for questions with <100 mean responses per practice, usually indicating questions which only a subset of patients were eligible to answer. Nine questions had both excellent reliability and high intraclass correlation coefficients (≥0.10) indicating both reliable measurement and substantial performance variability.

CONCLUSIONS:

High reliability is a necessary property of indicators used to compare health care organizations. Using the English General Practice Patient Survey as a case study, we show how reliability and intraclass correlation coefficients can be used to select measures to support robust organizational comparisons, and to design surveys that will both provide high-quality measurement and optimize survey costs.

PMID:
21610543
DOI:
10.1097/MLR.0b013e31821b3482
[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for Wolters Kluwer
Loading ...
Support Center