Table 7Summary of evidence: long-term care services

Key Question 1: Does public reporting result in improvements in the quality of health care (improvements in health care delivery structures, processes, or patient outcomes?

Key Question 2: What harms result from public reporting?

Key Question 3: Does public reporting lead to change in health care delivery structures or processes?

Key Question 4: Does public reporting lead to change in the behavior of patients, their representatives, or organizations that purchase care?

Key Question 5: What characteristics of public reporting increase its impact on quality of care?

Key Question 6: What contextual factors (population characteristics, decision type, and environmental) increase the impact of public reporting on quality of care?

Author
Year
(QA)
Public ReportStudy OverviewKey QuestionResults
↑ Improvement; ↓ Worse; ↔ No difference
Cai
201081
(Fair)
NH CompareCompared State vaccination rates for three flu seasons (2005–2006, 2006–2007, 2007–2008) after the publication of vaccination rates in NH Compare.
Rates for NH residents compared with rates for community dwelling elderly.
N=51 (all States and DC).
3State vaccination rates change with NH Compare
↑ Vaccination rate: Short-stay and long-stay residents
↔ Larger increase in community-dwelling elderly than in NH residents
6↑ More improvement among NHs with lower baseline rate
↓Slight decline among NHs with higher baseline rate
Castle
2007115
(Fair)
NH CompareCompared publicly reported QMs for U.S. NHs in markets with high competition and low occupancy rates to NHs in markets with low competition and high occupancy rates in 2003 and 2004.

N=14,554
6Higher Competition
↑ 5 out of 14 QMs improved and overall improvement
Long Stay: ADLs, low risk pressure sores; short stay: delirium, pain, pressure sores
↔no significant effect: 9 out of 14 QMs
Lower Occupancy
↑ 8 out 14 QMs improved and overall improvement
Long stay: ADLs, low risk pressure sores, catheter, ability to move around
Short stay: delirium, pain, pressure sores
↔no significant effect: 7 out of 14 QMs
Castle
200860
(Fair)
NH CompareExamined trend in improvement post public reporting adjusted for regression to the mean for U.S. NHs from 2004 to 2006. Subgroup comparisons by market characteristics.

N=14,224
1↑ 9 of 15 QMs
Long stay: pain, high risk PU, low risk PU, restraints, depressed, catheters
Short stay: delirium, pain, pressure sores
↓ 5 of 15 QMs
Long stay: ADLs, incontinence, move about, UTI, lose too much weight
↔ 1 of 15 QMs
Long stay: mostly in chair or bed
6Higher Competition
↑ 8 out of 15 QMs and overall
Long stay: ADLs, high risk pressure sores, depressed, most time in bed or chair, UTI, lost too much weight
Short stay: delirium, pain
↔no significant effect: 7 out of 15
Lower Occupancy
↑ 10 out 15 QMs and overall
Long stay: ADLs, low risk pressure sores, restraints, depressed, incontinence, UTI, ability to move around, lost too much weight
Short stay: delirium, pressure sores
↔no significant effect: 7 out of 14 QMs
Castle
2010224
(Fair)
Special Focus Facility designation by CMS (on Nursing Home Compare)Compared all U.S. NHs divided by whether they are in counties that had one or more special focus facility in 2007 (n=135) compared with NHs in counties where none had this designation.

(N=14,1553)
6Impact on quality measure of SFF in same county
↑4 out of 22 QMs
High-risk PU, low-risk PU, UTI, short-stay PU
↓2 out of 22 QMs
Any deficiency, quality citations
↔ 16 out of 22 QMs
↑8 out of 22 QMs when only facilities below the median level of quality are analyzed
Gaudet
201162
(Good)
NH CompareExamined how NH performance changed in response to public reporting and how this varies across market and facility characteristics, particularly the proportion of black residents in NHs.

N=over 14,500 NHs (exact n varies for each quarter)
1Change in NH Compare QMs
↑Restraints, pressure ulcers, pain
↓ADLs
6↔ Percent no significant effect of Medicare residents, nonprofit ownership, market competition on QMs
↔ Percent Black residents had no significant effect overall; NH with higher percent Black residents started a higher quality pre public reporting on some QMs. NH Compare had less of an impact (slope of change was less) on facilities with higher percent Black residents compared to facilities with lower levels of Black residents
Grabowski
201161
(Good)
NH CompareEvaluated the effect of NH Compare on facility performance and consumer demand for services in pilot and on pilot States.

N=15,553 NHs
1QMs post NH Compare; comparison of pilot and non pilot States
↔ No impact on 5 of 5 QMs
4↔ No impact of 5 publicly reported QMs on market share
6Higher competition
↑2 of 5 QMs
High-risk PU, low-risk PU
↔ 3 of 5 QMs
Jung
201014
(Fair)
Home Health CompareDescribed change in quality measures from 2003 to 2007 (yearly measures) and change by Home Health Agency Characteristics.

N=8,679 agency with at least 2 years of data.
1Change in QMs post HH Compare
↑7 of 7 functional measures
Number of QM for which agencies changed quality indicator scores
↑ 6 of 7 more agencies improved
↓ 1 of 7 more agencies worsened
6↑ Nonprofit started lower than for profits on some QM, but had greater improvement and ended with higher scores on all QMs
↑ hospital-based had greater improvement
↑ longer Medicare tenure had greater improvement
↑ lower baseline QMs increased more
Konetzka
201275
(Good)
NH CompareAnalyzed if NHs responded to public reporting by rehospitalizing postacute care patients who might have a negative impact on their NH Compare scores before they are assessed (Day 14) for NH Compare scores.

N=8,139 NHs
2↓ 1.2% increase in discretionary rehospitalizations. 0.5% after controlling for secular trends by comparing pilot and non pilot States.
Increase greater in patients at higher risk of poor scores on NH Compare QMs
Mukamel
200863
(Good)
NH CompareCompared quality scores for all U.S. NHs.
Pre Public Reporting(4th Q 2001 to 4th Q 2002) and Post Public Reporting: (1st Q 2003 to 4th Q 2003).

Merged with survey responds for 10% sample of administrators. 724 completed survey (48.2%)
1↔0 of 5 for time trend
↑2 of 5 for change in level after public report: physical restraints, short-stay pain
↓ 1 out of 5: pressure ulcers (in non demonstration States)
↔2 out of five: ADLs, infection and PU in demo States.
3Change in QMs with number of actions taken
↑ With increase in actions: Physical restraints, short-stay pain
↓ With increase in actions: Pressure Ulcers
↔ With increase in actions: ADL and Infections
Mukamel
200972
(Fair)
NH CompareCompared NH admission cohorts for all U.S. nursing homes for periods pre and post reporting as well as after changes in 1st Q 2004. Pre Reporting: 1st Q 2001 to 4th Q 2002.
Post Reporting: 1st Q 2003 to 4th Q 2005.

N=16,745
2↔ No significant change in admission cohorts indicating no cream skimming
ADL, diabetes, incontinence, PU stage 2 or higher
↓ Decrease indicating cream skimming
Pain and memory loss
6Change in admission cohorts by NH characteristics
↔ADL, diabetes, incontinence, PU stage 2 or higher
Reduced admissions
↓ Pain: for profit and nonprofit reduced admissions, government NH did not
Memory loss: for profit and chain reduced admissions
Mukamel
201080
(Fair)
NH CompareCompared ratio of clinical to hotel expenses by NHs for 2 pre report-card years and 4 post public report years including 10,022 NHs over 6 years from 2001 to 2006 (54,235 observations).3↑ by 5% in the ratio of clinical to hotel expenditures post public report
Magnitude of effect reduced significantly by controlling for differential growth in costs.
6↑ Ratio for NH with:
Lower-quality scores
Lower occupancy
For profit
Chain owned
More competitive markets
Park
(13080)
2011a108
(Good)
NH CompareExamined if high quality NHs or NHs that improve on publicly reported quality scores receive a return in terms of financial performance by increasing admissions by comparing 1999–2002 to 2003–2006.

N=6,286 NHs
4Improvement in NH Compare QMs leads to
↑ Market share, specifically increased Medicare admissions leading to better financial performance (higher revenues)

If NHs is High Quality based on NH Compare QMs than
↔ Market share and financial performance relationship do
Park
(12601)
2011b109
(Good)
NH CompareExplored if public reporting changes the relationship between financial performance and quality of care in NHs prior to NH Compare (1997–2002) vs after NH Compare (2003–2006).

N=9,444 NHs
4Interaction between profit margin and QMs
↑ 3 of 4 QMs the association between profit margin and QMs was stronger after public reporting (total staff hours per resident day, incidence of pressure ulcers, number of deficiency citations)
↔ 1 o4 QMS the association between profit margin and restraint use was not significantly different after public reporting
6For profit vs. nonprofit
↑ For profit: 3 of 4 QMs stronger association between profit margins and QMs after public reporting
↑ Nonprofit: 1 of 4 QMs stronger association between profit margins and QMs after public reporting
Competitive Markets
↑ greater increased association between profit and quality in competitive markets after public reporting
Stevenson
2006106
(Poor)
Nationally posted Deficiencies and Staffing Levels for NHsCompared Pre Reporting: 1996 - Oct. 15, 1998 (1996, 1997, 1998) to Post Reporting Years: (1999, 2000, 2001, 2002).4Change in occupancy rate as measure for patient selection
Post quality rating:
↑ Increase in occupancy with fewer prior deficiencies, with fewer prior serious deficiencies with more LPN/RN staff
↓Decrease in occupancy with more aide staff (contrary to hypothesis)
Werner
2009a13
(Good)
NH Compare for Post Acute careCompared all NHs with residents with postacute stays of at least 14 days pre 2002 NH Compare launch vs. post NH Compare and compared these to small nursing homes not included in NHC.

N= 8,137 in NH Compare; 2,777 small NHs
1↑ 3 of 4 QMs
Pain, delirium, walking
↓1 of 4 QMs
Preventable rehospitalization
Incorporation of secular trend
↑3 of 4 QMs
Pain, smaller magnitude
Delirium: no change in magnitude
Walking: slight increase in magnitude
↓1 of 4 QMs
Preventable rehospitalization
Slight worsening, then stable but did not improve.
Werner
2009b74
(Good)
NH Compare for Post Acute CareCompared all U.S. NHs using MDS data pre NH Compare and post NH Compare on postacute care measures on NH Compare.

N=13,683
2Change After NH Compare

↑3 of 3 publicly reported QMs
Pain, delirium, walking

Not publicly reported QMs for same period
↑5 of 9 QMs
Pain, locomotion, shortness of breath, incontinence, respiratory infection
↓4 of 9
UTI, ADLs, mid-loss ADLs, early-loss ADLs

Non publicly reported for NHs with high score on publicly reported
↑6 of 9 QMs
Pain, locomotion, shortness of breath, incontinence, respiratory infection, UTI
↓3 of 9 QMs
ADLs, mid-loss ADLs, early-loss ADLs

↓Nurse staffing
decline less for high score than low score on reported measures
Werner
201058
(Good)
NH Compare for Post Acute CareCompared all NHs reporting postacute measures twelve months before Public Report to twelve months after launch of NH Compare. Disaggregates change into portions due to QI, market share and residual

N=8,137
1Post acute care measure change post NH Compare
↑ Pain overall
↑Pain due to QI
↑Pain due to market share
↓Pain due to residual

↔Delirium overall
↔Delirium due to QI
↑Delirium due to market share
↓Delirium due to residual

↔walking overall
↑Walking due to QI
↑Walking due to market share
↓Walking due to residual
Werner
201173
(Good)
NH Compare for Post Acute CareCompared pilot and non pilot States prior to and after NH Compare to determine if public reporting results in changes in the types of people choosing high and low quality providers (patient sorting) occurred for postacute care.

N=8,139 NHs
2Cream skimming
↔No evidence NHs admitted lower risk patients in order to improve NH Compare scores
Down coding
↓Change in admission levels of pain (lower after public reporting) suggests facility may be down coding high risk patients
4Patient sorting; high risk patients admitted to higher quality NHs
↑ 1 of 3 QMs.
Pain (correlation between higher risk on admission and high quality increased after NH Compare.
10 point higher NH Compare score associated with 1% point increase in admission pain level for following quarter
↔ 2 of 3 QMs Delirium and difficultly walking. No change
Werner
2012107
(Good)
NH Compare for postacute careTo determine if public reporting influences patients’ selection of NHs for postacute care.4Selection of NHs (market share)
↑ 1 of 3 QMs; Pain
Change in a Pain score from 25th to 74th percentile (fewer patients with pain) increases market share 1.3%

↔ 1 of 3 QMs; Delirium near zero
↓ 1 of 3 QMs; Walking
Counter intuitive result: improvement in score associated with decline in market share
6Patient Education Level
↑ 3 of 3 QMs larger response to public reporting by patients with higher (High school or more) education level

NH Occupancy/Capacity Constraints
↑ Greater impact on selection in markets with lower occupancy (lower capacity constraints

Not reporting in NH Compare
↓ Smaller NHs not required to publicly report lose market share after public reporting, suggesting patients interpret the lack of data as a sign of poor quality.
Zinn
200559
(Fair)
NH CompareAssessed quality improvement using NH Compare quarterly reports from November 2002 (first publication) through January 2004 for all NHs reporting.

(N=over 13,00 for long-stay resident measures, over 9,000 for short-stay resident measures)
1Post NH Compare:
↑Long stay: pain, physical restraints
Short stay: delirium, pain
↔Long stay: daily tasks, PU, PU risk adjusted, infection
Short stay: delirium risk adjusted, walking
6Characteristics compared on rate of improvement. End level was still higher even though improvement is faster for NH with characteristics (the trend lines do not cross)
Long Stay Residents
↑ Pain higher rate of improvement in hospital-based vs. not hospital-based
Short Stay Residents
↑ Delirium higher rate of improvement with low occupancy rate vs. high
↑ Pain higher rate of improvement in non chain vs. chain NH
Zinn
200879
(Good)
NH CompareCross-sectional comparison of response to NH Compare by different types of strategic orientation:
Prospectors changed frequently and valued innovation and flexibility.
Defenders focused on core services and emphasize operating efficiencies.
Analyzers blended characteristics of the first two.
Reactors lacked a strategy.

Survey responds for 10% sample of administrators. 724 completed survey (48.2%)
337% took immediate action due to NH Compare; 30% took no action

Found differences in responses by strategic type of administrator
  • Respond immediately: Prospectors
  • Take no action: Defenders
  • Communicate with families about public report: No strategic type
  • Investigate reasons for scores: Prospectors and analyzers
  • Revise job descriptions: Prospectors
  • Invest in equipment of technology: No strategic type
637% took immediate action due to NH Compare; 30% took no action
Characteristics of NH more like to take these actions:
  • Respond immediately: Nonprofits, high competition
  • Take no action: Poor initial quality, low competition
  • Communicate with families about public report: High competition, chain
  • Investigate reasons for scores: Poor initial scores
  • Revise job descriptions: Poor initial scores
  • Invest in equipment of technology: different by no NH characteristics
Zinn
201078
(Fair)
NH CompareLikelihood of investing resources to response to NH compared by administrator perceptions and NH characteristics. 10% random sample of NH administrators at all facilities with at least one quality measure reported on NH Compare in 2006.

538 responses from1407 contacted (38.3%)
3Likelihood of resource intensive changes in response to perceptions of NH Compare influence
Believe NH Compare Influences Referrals
↑4 out of 6 actions
↔ 2 out 6
Believe NH Compare Influences Choice of NH
↑1 out of 6 actions
↔5 out 6
Believe NH Compare Influence State Survey
↑5 out of 6 actions
↔1 out 6
Have Managed Care Contract
↓ 3 out of 6 actions
↔ 3 out 5
6↑3 out of 6 actions
More likely if NH had low-quality scores as opposed to high-quality scores and is in a highly competitive market

Notes: ADL = activities of daily living; CMS = Centers for Medicare and Medicaid Services; NH = nursing home; PU = pressure sores or ulcers; QI = quality improvement; QM = quality measure; SFF = special focus facility; U.S. = United States; UTI = urinary tract infection

From: Results

Cover of Closing the Quality Gap: Revisiting the State of the Science (Vol. 5: Public Reporting as a Quality Improvement Strategy)
Closing the Quality Gap: Revisiting the State of the Science (Vol. 5: Public Reporting as a Quality Improvement Strategy).
Evidence Reports/Technology Assessments, No. 208.5.
Totten AM, Wagner J, Tiwari A, et al.

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.