Send to

Choose Destination
Perm J. 2017;21:16-084. doi: 10.7812/TPP/16-084.

Comparing Hospital Processes and Outcomes in California Medicare Beneficiaries: Simulation Prompts Reconsideration.

Author information

Regional Director for Hospital Operations Research for The Permanente Medical Group, Inc, at the Division of Research in Oakland, CA.
Public Health Program Specialist for Contra Costa Public Health Clinic Services in Martinez, CA.
Data Analyst for the Division of Research in Oakland, CA.
Professor of Applied Mathematics and Statistics at the University of California, Santa Cruz.
Regional Director for Hospital Advanced Analytics for The Permanente Medical Group, Inc, at the Division of Research in Oakland, CA.
Principal Statistician for Decision Support at Kaiser Foundation Health Plan.



This article is not a traditional research report. It describes how conducting a specific set of benchmarking analyses led us to broader reflections on hospital benchmarking. We reexamined an issue that has received far less attention from researchers than in the past: How variations in the hospital admission threshold might affect hospital rankings. Considering this threshold made us reconsider what benchmarking is and what future benchmarking studies might be like. Although we recognize that some of our assertions are speculative, they are based on our reading of the literature and previous and ongoing data analyses being conducted in our research unit. We describe the benchmarking analyses that led to these reflections.


The Centers for Medicare and Medicaid Services' Hospital Compare Web site includes data on fee-for-service Medicare beneficiaries but does not control for severity of illness, which requires physiologic data now available in most electronic medical records.To address this limitation, we compared hospital processes and outcomes among Kaiser Permanente Northern California's (KPNC) Medicare Advantage beneficiaries and non-KPNC California Medicare beneficiaries between 2009 and 2010.


We assigned a simulated severity of illness measure to each record and explored the effect of having the additional information on outcomes.


We found that if the admission severity of illness in non-KPNC hospitals increased, KPNC hospitals' mortality performance would appear worse; conversely, if admission severity at non-KPNC hospitals' decreased, KPNC hospitals' performance would appear better.


Future hospital benchmarking should consider the impact of variation in admission thresholds.

[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for PubMed Central
Loading ...
Support Center