• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Totten AM, Miake-Lye IM, Vaiana ME, et al. Public Presentation of Health System or Facility Data about Quality and Safety: A Systematic Review [Internet]. Washington (DC): Department of Veterans Affairs (US); 2011 Oct.

Cover of Public Presentation of Health System or Facility Data about Quality and Safety: A Systematic Review

Public Presentation of Health System or Facility Data about Quality and Safety: A Systematic Review [Internet].

Show details

APPENDIX EEVIDENCE TABLES FOR KEY QUESTION #4

Author, Year (ID)ObjectiveSubject of public reporting;
Hospital/Health plan;
Location
SampleDesign TypeDesign Rating; Global RatingKey Findings
Impact on Quality Improvement Activity Articles from Fung
Chassin, 200236To examine relationship between mortality rate outlier status and hospital CABG volume/quality improvement activity following the implementation of the CSRSNYS CSRS;
Hospital;
New York State
Key informants at four hospitals and state officials directly involved in efforts to quality improvement efforts at the hospitalsCase Series1;1Increase in quality improvement activity (e.g., staffing policy changes, multidisciplinary approach to examining care processes, changes in operating room schedule)
Dziuban, 199459To document a hospital's response to being identified as a high risk-adjusted mortality outlier in the CSRSNYS CSRS;
Hospital;
New York State
One outlier hospitalCase Study1;1Quality improvement activity increased (change in timing & technique used for patients undergoing emergent CABG, change in hospital policies)
Bentley, 199860To determine whether Pennsylvania Health Care Cost Containment Council's Consumer Guide to CABG, which compared in-hospital mortality rates, led to more changes in Pennsylvania hospitals' CABG policies/practices than in New Jersey hospitals, which were not required to publicly-report performance resultsPennsylvania consumer guide;
Hospital;
Pennsylvania and New Jersey
Key informants at the hospitals identified by the chief executive officers of these hospitals; Hospitals providing CABG surgerySurvey (Descriptive)1;1Response in Pennsylvania hospitals (e.g., recruited staff, started continuous quality improvement program to improve CABG procedures). More changes in Pennsylvania than New Jersey hospitals (no formal statistical testing because small sample size)
Hibbard, 200358To compare the effects of public reporting (QualityCounts) to confidential reporting and no reporting, on quality improvement activity, market share (hospital discharges), and risk-adjusted performance (two summary indices of adverse events and indices in three clinical areas--hip/knee surgery, cardiac care, and obstetric care)QualityCounts;
Hospital;
South central Wisconsin
Hospitals participating in Quality Counts (n=24)Controlled Before/After Trial3;1Compared to hospitals that received confidential reports or no reports, QualityCounts hospital did not engage in more quality improvements overall, but they did engage in a statistically higher number of quality improvement efforts specific to the areas included in the reports.
Hibbard, 20054To compare the impact of public (QualityCounts), internal (private) and no reporting, on quality improvement activity, market share (hospital discharges), and risk-adjusted performance (two summary indices of adverse events and indices in three clinical areas--hip/knee surgery, cardiac care, and obstetric care)QualityCounts;
Hospital;
South central Wisconsin
Hospitals participating in Quality Counts (n=24)Descriptive (survey) (for quality improvement analysis)1;1Out of seven possible activities, mean number of quality improvement activities was 4.1 overall; 5.7 for hospitals with improved ratings; 2.6 with no change in ratings; 4 with decrease in ratings (no formal statistical testing)
Rosenthal, 199862To study quality improvement activities following release of CHQC reports of mortality rates, length of stay, and cesarean section rates (all measures severity-adjusted)CHQC;
Hospital;
Cleveland
One academic and three community hospitals of varying size in the Cleveland areaCase Series1;1Quality improvement activities increased (e.g., interdisciplinary process improvement teams, detailed review of processes of care, development of practice guidelines)
Tu, 200361To study the impact of the “Cardiovascular Health and Services in Ontario: AN ICES Atlas,” which reports hospital-specific acute myocardial infarction performance measures, on quality improvement activityICES;
Hospital;
Ontario, Canada
All Ontario hospitals providing acute myocardial infarction care; Physicians working in Ontario hospitals representing 62 of 121 eligible hospitals (52% overall hospital response rate)Descriptive (survey)1;154% of respondents indicated that one or more changes were made at their hospital
Longo, 199763To examine the impact of Missouri Department of Health's obstetrics consumer report, which provides structure, process, and outcomes measures, on quality improvement activity and clinical outcomesMO Dept. Health obstetrics consumer report;
Hospital;
Missouri
All hospitals providing obstetric care; Key informant designated by hospital administrators at 82 hospitals (93% response rate)Descriptive (survey)1;1Hospitals instituted services (e.g., hospital policy for that infants ride in car seats upon discharge, formal neonatal transfer agreements) after the reports were published
Luce, 199665To describe quality improvement activity following the California OSHPD's CHOP report featuring risk-adjusted outcomesOSHPD CHOP;
Hospital;
California
All California non-federal hospitals; 17 out of 22 public hospitals that are members of the California Association of Public Hospitals and Health SystemsDescriptive (survey)1;1Minimal impact on quality improvement activity
Rainwater, 199866To describe the impact of publicly reporting California's CHOP risk-adjusted 30-day inpatient mortality rates for patients with acute myocardial infarction, on quality improvement activityOSHPD CHOP;
Hospital;
California
California non-federal acute care hospitals; 39 key informants at a sample of acute care hospitals in CaliforniaInterviews1;1Minimal impact on quality improvement activity (2/3 respondents indicated no specific QI activity)
Mannion, 200564To describe impact of the National Health Service (NHS) star performance ratings on quality improvement effortsNHS;
Hospital;
United Kingdom
All hospital trusts; Staff at four low performing hospital trusts and two high performing hospital trustsCase series1;1Ratings transmitted important priorities from central government and helped direct and concentrate front-line resources. Public reporting led to tunnel vision and distortion of clinical priorities and disincentive to improve performance among high-rated organizations.
Impact on Quality Improvement Articles, not in Fung
Wang, 201057To examine the impact of report cards on provider volume (hospital and surgeons) and on patient matching with surgeons.Hospital CABG VolumeHospitals in PA who perform 30 or more CABG per year between 3rd Q 1998 and 1st Q 2006Observational Cohort3; 2Report cards have no significant impact on hospital surgical volume and do not change the population of patients who have CABG. Report cards have a larger impact on the distribution of healthier patients as opposed to sicker across hospitals. Bad rating takes a year to have an effect on volume which was estimated as a decrease in quarterly CABG cases of about 15%. These were almost all among low severity CABG cases. This effect did not persist past one year.
Rainwater, 200556To evaluate the use and impact of California's Quality of Care Report Card (QRC), based on three questions: 1. Do consumers use the QRC? 2. How useful to consumers are the quality measures included in the QRC? 3. What is the impact of the QRC on quality improvement and other activities in the participating HMOs and medical groups?California's Quality of Care Report Card (QRC); Health Plan; California6 consumer focus groups, 2,341 respondents to mail and internet surveys, 56 key informantsMixed methods: focus groups, surveys, interviews3;3Use is reported at over 28,000 visitors to the QRC website annually, and over 100,000 booklets distributed. Users are most interested in comparing HMOs in the plan service domain, and find features like the specialty care information, specific measures such as mental health care, and comparative performance information by health topic or disease most helpful.
Impact on Clinical Outcomes Articles From Fung
Hannan, 199471To assess changes in in-hospital mortality rates of CABG patients following the publication of mortality data in the CSRSNYS CSRS;
Hospital;
New York
All New York hospitals performing CABG; 57187 patients undergoing CABG (1989-1992)Analysis of Time Trend2;2RAMR decreased from 4.17% to 2.45%.
Dziuban, 199459To document a hospital's response to being identified as a high risk-adjusted mortality outlier in the CSRSNYS CSRS;
Hospital;
New York
One poor performing hospitalCase Study1;1Excess mortality was localized to high-acuity patients undergoing emergent CABG. Mortality decreased to zero following focused effort to optimize management of these patients.
Hannan, 199454To determine if mortality rate outlier status was associated with changes in CABG-related in-hospital risk-adjusted mortality rates following the implementation of the CSRSNYS CSRS;
Hospital;
New York
All New York hospitals performing CABG; All New York patients discharged after CABG (1989 to 1992)Analysis of Time Trend2;2Reductions in RAMR, especially among hospitals that had highest initial mortality rates. Convergence in risk-adjusted mortality rates among hospitals initially identified as high, medium, and low performers.
Peterson, 199872To examine the impact of the CSRS on in-hospital mortality rates by comparing unadjusted mortality rates in New York to other states. To examine the impact of the CSRS on in-state access to CABG and referral out-of-state of patients in need of CABGNYS CSRS;
Hospital;
New York
All hospitals performing CABG; Medicare patients 65 or older who underwent CABG in a U.S. hospital (1987 to 1992)Observational cohort3;3Both unadjusted and risk-adjusted mortality rates in New York declined more than in other states.
NY MI patients were less likely to receive CABG, but the overall percentage of NY MI patients receiving CABG rose, paralleling national trends, even among higher risk elderly subsets; out-of-state CABG rates declined
Ghali, 199773To compare trends in CABG-related mortality in Massachusetts (a state without statewide public reporting of CABG outcomes) to New York (a state with public reporting) and northern New EnglandNYS CSRS;
Hospitals;
New York and Massachusetts
All NY hospitals performing CABG; 12 Massachusetts hospitals performing cardiac surgery (except Veterans Affairs hospitals) and hospitals contained in the HCFA hospital 30-day unadjusted mortality dataset (1990, 1992, and 1994)Observational cohort3;2RAMR reductions in Massachusetts were comparable to mortality reduction in New York and northern New England; unadjusted mortality trends were similar in Massachusetts, New York, northern New England, and the United States
Rosenthal, 199777To measure changes in hospital mortality that occurred following the implementation of the CHQC reporting initiative, which publicly-released in-hospital mortality ratesCHQC;
Hospital;
Cleveland
Hospitals in the Cleveland area; 101,060 consecutive eligible discharges with eight diagnoses (acute myocardial infarction, heart failure, obstructive airway disease, gastrointestinal hemorrhage, pneumonia, stroke, CABG, and lower bowel resection) from 30 northeastern Ohio hospitals (1992 to 1993)Time Series3;1Risk-adjusted mortality for most conditions declined from 7.5% to 6.8%, 6.8%, and 6.5% for 3 periods following publication. Declines in mortality rates were significant in weighted linear regression analyses for heart failure (0.50% per period) and pneumonia (0.38% per period)
Baker, 200334To examine hospitals' market share and 30-day risk-adjusted mortality at hospitals participating in CHQCCHQC;
Hospital;
Cleveland
Medicare patients receiving care at these Cleveland-area hospitals (1991 to 1997)Time Series3;2Hospital outlier status was not significantly related to changes in risk-adjusted 30-day mortality between 1991 and 1997.
Clough, 200278To measure changes in in-hospital mortality rates associated with the implementation of the CHQC reporting initiativeCHQC;
Hospital;
Cleveland
Hospitals included in the Ohio Hospital Association's inpatient discharge data (1992 to 1995)Observational cohort3;2No statistical difference in rate of decline in combined mortality in Cleveland compared to the rest of the Ohio
Longo, 199763To examine the impact of Missouri Department of Health's obstetrics consumer report, which provides structure, process, and outcomes measuresMO Dept. Health obstetrics consumer report;
Hospital;
Missouri
All Missouri hospitals providing obstetrics care (1989 to 1993)Observational cohort3;2Improvements in ultrasound rates, vaginal birth after cesarean rates, and cesarean rates were noted among outlier hospitals
Hibbard, 20054To compare the impact of public (QualityCounts), internal (private) and no reporting, on quality improvement activity, market share (hospital discharges), and risk-adjusted performance (two summary indices of adverse events and indices in three clinical areas--hip/knee surgery, cardiac care, and obstetric care)QualityCounts;
Hospital;
South central Wisconsin
Hospitals participating in Quality Counts (2001 to 2003, n=24)Controlled Before/After Trial (for outcomes analysis)3;2Performance feedback, whether public or private, was associated with improved performance
Moscucci, 200574To measure the effect of the New York State PCI report on case selection for percutaneous coronary intervention (PCI) by comparing Michigan's and New York's adjusted and unadjusted in-hospital mortality ratesNYS PCI (CSRS);
Hospital;
New York and Michigan
All New York hospitals performing CABG; 11,374 patients in a multicenter (eight hospital) PCI database in Michigan and 69,048 patients in a statewide (34 hospital) PCI database in New York (1998 to 1999)Observational cohort3;2Unadjusted mortality rates were significantly lower in New York than Michigan, but adjusted mortality rates were not statistically different.
Omoigui, 199675To determine if dissemination of CSRS mortality data was associated with outmigration of high-risk patients to undergo treatment at the Cleveland ClinicNYS CSRS;
Hospital;
New York and Cleveland
All hospital performing CABG in New York State; 9,442 patients receiving CABG at the Cleveland Clinic (1989 to 1993)Observational cohort3;2Patients from New York State receiving CABG at the Cleveland Clinic had higher RAMR than patients from Ohio, other states, and other countries
Dranove, 200376To study the effects of public reporting in New York and PennsylvaniaNYS CSRS and Pennsylvania public reporting system;
Hospital;
New York and Pennsylvania
All New York and Pennsylvania hospitals performing CABG; Medicare beneficiaries and hospitals found in a Medicare claims data set (not specified) and hospitals participating in the American Hospital Association annual survey (1987 to 1994)Observational cohort3;2Report cards shifted CABG use to healthier patients, leading to worse outcomes, especially among sicker patients (defined as higher hospital expenditures and days in hospital)
Baker, 200279To examine mortality trends associated with the CHQC programCHQC;
Hospital;
Cleveland
Hospitals in the Cleveland area; Medicare patients hospitalized with acute myocardial infarction, heart failure, gastrointestinal hemorrhage, obstructive pulmonary disease, pneumonia, or stroke (1991 to 1999)Time Series3;2Risk-adjusted in-hospital mortality declined significantly for most conditions, but the mortality rate in the early post discharge period rose significantly for most conditions and the 30-day mortality rate declined significantly for only heart failure and obstructive pulmonary disease
Bost, 200180To compare HEDIS and CAHPS results for plans that publicly report data with those who do not, over a three-year periodHEDIS and CAHPS;
Health plan
U.S.
Commercial health plans (1997-1999)Observational cohort2;1Technical performance measures and patient experience measures (except communication) were higher for health plans that publicly report data.
McCormick, 200281To assess the relationship between health plan performance and participation in public reporting programsHMO commercial health plans;
Health plan;
U.S.
HMO health plans (1997 to 1999)Observational cohort2;2Lower-scoring plans are significantly more likely than plans with higher-scoring plans to stop disclosing publicly their quality data
Impact on Clinical Outcomes Articles, not in Fung
Bevan, 200970To assess the impact of public reporting on the performance of ambulance servicesAmbulance service response times;
UK
Yearly data from 2000 to 2005“natural experiment’ Comparison of UK countries with the same target but one had reporting and the others did not.3;2Response times improved in the countries with public reporting and did not in the others. Examination of potential harms found evidence that some types of gaming occurred (data was changed) but that others types that were suspected (changes in the classification of the event) did not.
Cutler, 200432To examine whether medical quality among hospitals are affected by report cardsNYS CSRS;
Hospital;
New York State
All hospitals performing bypass surgery in New York (3,406 patients in the baseline year)Observational, time series--across hospital rather than statewide trends.3;2Hospitals identified as high mortality improve performance in terms of decreased risk-adjusted mortality rates: mortality declined 1.2 percentage points (significant at the 0.01 level) in these low quality hospitals during the 12 months after the reporting.
Elliott, 201068To determine if hospitals improved in terms of patient experience over the initial 2 years of public reporting of HCAHPS resultsHCAHPS;
Hospital;
US
Hospital, National CAHPS US 61% of hospitals in 3/08 3,864; 84 % of hospitals in 3/09 3,863 Patient response rate averaged 34%--patients are a random sample of dischargesObservational, Time series, no comparison group3;2Hospitals improved in 8 of 9 domains as measured by percent of positive responses (MD communication did not improve). Magnitude of changes was small, but would result in change in ranking. Hospital size and original (both years) vs. later (2nd year only) participation were examined and smaller hospitals who participated later performed better.
Hendriks, 200969To determine if managed competition and public reporting of quality information is associated with quality improvement in health plans.National health plans;
Health plan;
Netherlands
Dutch Health Plans, and Health Plans on a National Level; Random sample of health Plan Members; CQI--based on CAHPS;Observational, time series, no comparison group3;1Plans improved in some domains (health plan information and transparency of copayment, conduct of employees, and general rating and requirements, but not others(access to call center, getting needed help from call center and reimbursement of claims) from 2005 to 2008. Identification of selected domains as areas in need of improvement did not seem to affect whether there was improvement or not.
Kim, 200567To assess the impact of public release of hospital caesarean rates.Caesarean Section Rates;
Hospital;
South Korea
263 hospitalsObservational, time series, no comparison group2;1Caesarean rates were 43.0% in 1999. Hospital data for 1999 were published in 2000 and rates declined to 38.6% in 2000 and 39.6% in 2001, which are lower than predicted based on rates for 1985 to 1999 and the first years with any decline. Multiple regression results found that hospitals with higher with higher baseline caesarean rates and higher volume were more likely to decline, while market share and financial incentives were not significantly associated with decline in rates.

Design ratings: 4 stars indicate a strong study design rating; while 1 star indicates a weaker study design rating.

Global ratings: 3 indicates great weight in the stratum's body of evidence; and 1 indicates little weight.

PubReader format: click here to try

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (892K)

Other titles in this collection

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...