• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of clinjasnLink to Publisher's site
Clin J Am Soc Nephrol. Nov 2010; 5(11): 2024–2033.
PMCID: PMC3001761
Original Articles

Dialysis Practices That Distinguish Facilities with Below- versus Above-Expected Mortality

Abstract

Background and objectives: Mortality rates vary widely among dialysis facilities even after adjustment with standardized mortality ratios (SMRs). This variation may occur because top-performing facilities use practices not shared by others, because the SMR fails to capture key patient characteristics, or both. Practices were identified that distinguish top- from bottom-performing facilities by SMR.

Design, setting, participants, & measurements: A cross-sectional survey was performed of staff across three organizations. Staff members rated the perceived quality of their units' patient-, provider-, and facility-level practices using a six-point Likert scale. Facilities were divided into those with above- versus below-expected mortality on the basis of SMRs from U.S. Renal Data Service facility reports. Mean Likert scores were computed for each practice using t tests. Practices that were statistically significant (P ≤ 0.05) and achieved at least a medium effect size of ≥0.4 were reported. Significant predictors were entered into a linear regression model.

Results: Dialysis facilities with below-expected mortality reported that patients in their unit were more activated and engaged, physician communication and interpersonal relationships were stronger, dieticians were more resourceful and knowledgeable, and overall coordination and staff management were superior versus facilities with above-expected mortality. Staff ratings of these practices explained 31% of the variance in SMRs.

Conclusions: Patient-, provider-, and facility-level practices partly explain SMR variation among facilities. Improving SMRs may require processes that reflect a coordinated, multidisciplinary environment (i.e., no one group, practice, or characteristic will drive facility-level SMRs). Understanding and improving SMRs will require a holistic view of the facility.

Since 1995, the U.S. Renal Data System (USRDS) has calculated standardized mortality ratios (SMRs) to estimate patient survival in dialysis facilities. Because unadjusted mortality rates cannot capture the inevitable case-mix variations among facilities, the SMR is adjusted for a range of variables including age, sex, race, disease duration, comorbidities, nursing home status, and body mass index (1,2). SMR further adjusts for underlying mortality rates by scaling observed mortality by expected mortality on the basis of national data. Individual facilities are classified as having lower (SMR < 1.0), higher (SMR > 1.0), or as-expected (SMR = 1.0) mortality compared with the national average (1,2). This information is publicly available (3) and allows payers and consumers to monitor quality within individual facilities and to compare quality among facilities.

Despite attempts to adjust for case-mix variations, patient survival varies among dialysis facilities even after adjustment. McClellan and colleagues evaluated mortality rates in a cohort of dialysis units and showed that facility-specific mortality ranged between 2.0 and 10.5 deaths per 10,000 patient days; variations remained after extensive adjustment (4). Goodkin and colleagues found that intra- and international mortality variations also remained after adjusting for a range of demographic characteristics and comorbidities (5). In short, mortality varies from center to center, and this variation is not explained by case-mix adjustment alone.

If case-mix adjustment is inadequate to fully explain mortality variations, then what else might explain the differences? One possibility is that there may be important differences in facility-level factors that affect SMRs. These factors may include procedural (e.g., implementation patterns of policies in a facility), attitudinal (e.g., staff morale), interpersonal (e.g., staff communication), or structural (e.g., layout of facilities) practices associated with reduced SMRs (6). There may also be additional patient-level characteristics that are unaccounted for with standard case-mix adjustment. These factors may be incremental to standard demographics and comorbidities or may even supersede those factors. For example, it is possible that top-performing units have more compliant, more motivated, or more disciplined patients compared with bottom-performing units—all characteristics that elude standardized adjustments.

To test the hypothesis that there are patient-, provider-, and facility-level factors that are associated with SMRs, and to further elucidate predictors of SMRs, we conducted the Identifying Best Practices in Dialysis (IBPiD) study; this study aims to catalogue best practices associated with improved patient outcomes in dialysis, including SMRs (6). In this study, we identify predictors of mortality that are not traditionally included in case-mix adjustments.

Materials and Methods

IBPiD Overview

IBPiD is a three-phase multidisciplinary study to identify best practices associated with facility-level achievement of dialysis outcomes, including SMRs (6). The study presented here presents data from Phase III of IBPiD. Phases I and II have been described in a previous publication (6), which culminated in the development of 155 candidate best practices for dialysis facilities. These candidate practices were identified through a multistage process, including systematic literature review, cognitive interviews of dialysis staff members, and an online “virtual focus group” of nurse managers, community nephrologists, and academic opinion leaders (6). The resulting practices were qualitatively catalogued (see Appendix) and operationalized in a questionnaire designed for use by dialysis organizations to measure adherence with these practices in their facilities. The primary objective of Phase III, described here, was to test whether the candidate best practices resulting from Phases I and II predict SMR. The study was approved by the University of California at Los Angeles Institutional Review Board and was conducted in accordance with the institutional guidelines regulating human subject research.

Questionnaire Administration

Questionnaire Content.

We administered confidential online questionnaires to staff in 90 randomly selected dialysis facilities across three organizations, including one not-for-profit small dialysis organization (Satellite Healthcare), one not-for-profit medium dialysis organization (Dialysis Clinic, Inc [DCI]), and one for-profit small dialysis organization (Renal Ventures Management, LLC). The questionnaire elicited staff perceptions regarding dialysis practices and other facility-level factors developed from our previous work (6). The questionnaire had eight sections regarding facility and staff practices, including (1) staff working environment, (2) facility characteristics and amenities, (3) facility-based health maintenance practices, (4) technician practices, (5) nursing practices, (6) social work practices, (7) dietician practices, and (8) multidisciplinary conference practices. In addition to facility-level characteristics, personnel rated characteristics of physicians who work in their facility, including the perceived quality of physician-staff interactions, physician-patient interactions, treatment practices, and documentation practices. For each dialysis practice within these sections, respondents indicated their level of agreement that the practice occurs in their facility using a six-point Likert scale (1 = “do not agree at all”; 6 = “agree completely”). Frequency ratings were also rendered on a six-point scale (1 = “never”; 6 = “always”). Dichotomous policies and procedures were answered with a binary “yes” versus “no” response. Finally, staff reported characteristics of the patients in their facility, including their perceived willingness to learn, compliance with dietary and medical advice, self-efficacy, and motivation for self-care.

Validating Online Questionnaire Results.

Although provider surveys are often a cost-effective way to assess process of care, they are potentially limited because responses may be incongruent with the reality on the ground. On-site assessments can help to verify the responses of provider surveys. We conducted on-site assessments in a sample of 15 dialysis units from our sample. A trained observer visited the centers and abstracted site data while blinded to the performance outcomes of the facilities and the results of online staff surveys. For purposes of inter-rater reliability, a second trained rater was present during four of the visits, and agreement was evaluated with kappa statistic for dichotomized assessments. To test the reliability of the online instrument, we measured correlation coefficients across a core set of items embedded in both types of data collection. We measured cross-instrument correlations for various facility physical characteristics (e.g., layout of chairs and pods, proportion of nursing stations with direct view of patients, use of computers on wheels), staffing characteristics (e.g., staff to patient ratios), and processes of care (e.g., hand washing frequency, glove changing frequency, use of aseptic technique, etc.). We found that inter-rater agreement was excellent, with kappa scores between 0.82 and 0.89 for the dual-rater visits. Furthermore, correlations between on-site assessments and staff reporting were positive and achieved expected directionality with most r values of 0.5 to 0.7.

Sampling.

Within each facility, we administered online surveys to staff in five roles, including (1) medical director, (2) nurse manager/supervisor, (3) floor nurse, (4) dialysis patient care technician, and (5) dietician. We developed separate surveys for each staff position that reflected the responsibilities and perspectives of the respondent's role, but also included overlapping questions to help evaluate multiple perspectives on variables common to all roles. To ensure that no one staff member's personal bias skewed results, each question was independently asked of staff from at least two different positions within the facility. Additionally, for facilities that were large enough, we surveyed two nurses and two technicians in lieu of focusing on only one individual's perception.

Calculation of Facility Item Scores.

To minimize the influence of social desirability in the response, a subset of items were asked in the negative. The first step in processing item scores was to reverse the scaling on all of these items. To obtain the best, unbiased estimate for each item at a facility level, all available responses, independent of the staff position, were averaged to arrive at a composite-scale score for each question. In instances in which staff skipped items, we imputed data using the mean of available nonmissing items. In instances in which no staff members in a particular role responded to the survey, we were unable to assign item scores to facilities lacking the data.

Statistical Analyses

We sought to identify individual predictors of facility-level SMRs as measuring by the 2007 USRDS Dialysis Facility Report. Because SMRs may vary from month to month in an individual facility, we obtained the 12-month running average for each facility (January through December 2007) in the period directly preceding administration of the staff surveys, which occurred between December 2007 and January 2008.

Because we tested over 150 predictors of facility-level outcomes, we performed sequential analytic steps to focus our analyses on the most important predictors. First, we divided the participating facilities into high versus low mortality groups, stratified by USRDS SMR of <1.0 (below expected mortality) versus >1.0 (above expected mortality). We then performed bivariate analyses to compare mean facility-level scores for each survey item between groups using the t test. To generate a parsimonious list of variables, and in acknowledgment of the fact that statistical significance does not always correspond with clinical significance, we limited our subsequent analyses to predictors fulfilling explicit criteria, including (1) the difference in scores between high versus low mortality facilities met or exceeded an effect size of 0.4 SD (calculated as the difference in the means in the respective groups divided by the pooled SD), which is considered to be a “medium effect size” (7); and (2) the P value for the bivariate relationship was significant at the P ≤ 0.05 level. By limiting our analysis to parameters that were statistically significant and achieved a minimum effect size threshold, we attempted to guard against spurious and potentially noninformative results.

We next evaluated the linear relationship between each predictor selected from our initial screen and facility-level SMRs. We calculated a Spearman correlation coefficient between each facility-level item score and facility-level SMR. For each analysis, we calculated a P value for the linear relationship on the basis of a bivariate ordinary least-squares regression analysis. We then selected predictors that met criteria for statistical significance and arranged them within conceptual categories.

Finally, the predictors selected from bivariate analyses were entered as independent variables into an exploratory multiple linear regression model predicting facility SMR as a continuous variable. We entered variables in a forward stepwise regression model while adjusting for organization to identify factors that independently predicted SMR after adjustment. To support a regression model with up to six independent predictors, assuming a minimum of 15 observations per predictor, we required a sample size of 90 total facilities. We used SAS v9.0 for all analyses.

Results

Sample Characteristics

There were 90 participating facilities in which 423 of 508 personnel completed the surveys (80% response rate; mean age 44.8 ±10 years; 19% men; tenure = 6.6 ± 6.3 years). Table 1 provides descriptive information about the participating units. The mean 12-month SMR across facilities was 0.87 ± 0.4. The SMR followed a normal distribution (Kolmogrov–Smironov test for normality: P = 0.10; 25th, 50th, 75th percentile = 0.61, 0.81, and 1.1, respectively; range = 0.2 to 2.5).

Table 1.
Characteristics of participating centers

Sixty-six percent of the facilities achieved an SMR < 1.0; the remainder had above-expected mortality (SMR > 1.0). None of the facilities in this sample achieved an SMR of exactly 1.0. The SMR mean and range for the top- and bottom-performing groups were 0.66 (range = 0.2 to 0.99) and 1.2 (range = 1.01 to 1.8), respectively. The percentage of patients per month with Kt/V ≥1.2, hemoglobin of 11 to 12 g/dl, albumin ≥3.5 g/dl, and parathyroid hormone concentrations of 150 to 300 pg/ml was 91%, 46%, 81%, and 38%, respectively. Among staff that completed the survey, only 4% of items were skipped and required imputation. Of the 90 participating facilities, 73 had complete case ascertainment from all five staff roles. The remaining had data from no fewer than four of five staff roles. There was no difference in staff nonresponse between top- versus bottom-performing groups (P = 0.5). Of note, there also were no differences in mean facility size or years in operation between facilities in the top- versus bottom-performing groups.

Predictors of Facility-Level SMR

We identified 19 predictors of facility-level SMR that met criteria for effect size and statistical significance, including nine patient characteristics, two physician characteristics, three dietician characteristics, and five facility characteristics, policies, and practices. Table 2 lists these predictors, arranges them by category, and provides the relationship between each predictor and SMR. When these were entered into a multivariable linear regression analysis there were five additive factors (r2 = 0.31) that independently predicted facility-level SMR (Table 3). The factors with the highest independent predictive value were (1) rapidity that multidisciplinary care conferences are convened after hospital discharge of a patient returning to the dialysis facility (β = −0.05; P = 0.01), (2) ability of dieticians to effectively address cultural issues in developing plan (β = −0.05; P = 0.01), (3) perceived quality of continuing medical education programs (β = −0.07; P = 0.05), (4) willingness of patients to learn from staff and physicians about self-care (β = −0.10; P = 0.02), and (5) patient discipline in following medical advice (β = −0.08; P = 0.05). We evaluated for evidence of an organization-level effect on the observed relationships. However, organization was not a significant predictor of SMR in the final regression model.

Table 2.
Predictors of facility-level SMRs stratified by domain
Table 3.
Independent predictors of SMRs across cohort of 90 participating facilities

Discussion

USRDS uses the SMR to adjust for variations in patient populations among dialysis facilities (1,2). This helps to “level the playing field” because facilities with highly morbid populations will yield higher unadjusted mortality rates. Yet there are variations in mortality despite attempts to adjust for case-mix variations. In this study, we identified patient-, provider-, and facility-level predictors of mortality that are not traditionally included in case-mix adjustments.

On the basis of reporting from a range of dialysis personnel, we identified 19 factors that distinguished high versus low mortality units by SMRs, of which 5 remained independently associated after multivariable adjustment. The factors cover a range of domains, including patient characteristics, physician practices, dietician practices, staff working climate, and facility policies and practices. This suggests that improving SMRs may require processes that reflect a coordinated, multidisciplinary environment (i.e., no one group, practice, or characteristic will drive facility-level SMRs). Understanding and improving SMRs will require a holistic view of the facility.

These data suggest that the USRDS formula for calculating SMR may be necessary but insufficient to fully adjust for case-mix variations among facilities. It is notable that the predictive factors identified in this hypothesis-generating study explained nearly one-third of the variance in SMR—an endpoint that is itself already adjusted. Because some of these factors pertain to patient characteristics, this suggests that further case-mix adjustment is possible. In our sample of 90 facilities including three participating organizations, we explained 31% of the variation in adjusted mortality among facilities on the basis of empirical ratings from dialysis staff alone. It appears that dialysis personnel have vital knowledge about their facility and its patients that the SMRs may not reflect.

In particular, the SMRs may be incapable of adjusting for various patient characteristics that are incremental to standard demographics and comorbidities. We found that personnel in facilities with low mortality rates perceive their patients to be more cooperative, willing to learn, compliant, self-efficacious, independent, and accountable compared with patients in high mortality units. To the degree that staff perception is a reliable surrogate for true patient characteristics, these data indicate that standard SMR adjustments may not fully reflect attributes of patient populations that matter for mortality. It is possible that the “institutional personality” of a dialysis patient population makes a difference. This finding may help dialysis managers gain better insight regarding how to measure population attributes of patients under their care.

Moreover, measurement of patient attribute scores might help managers in low-performing units identify patient-level barriers to quality performance that the SMR cannot. Managers could use this information to identify strategies to boost the institutional morale of its patients. That is, it may be possible that institutional personalities are malleable in the right setting. Potential interventions might include motivational events, enhanced and tailored educational techniques, or efforts to personalize the oftentimes complex science of dialysis in ways that motivate, inspire, and activate patients. This “patient activation model,” as originally expressed by Kaplan and Greenfield (812), has been successful in other areas of health care. The model posits that patients who are informed and actively participate in their health care have better outcomes than patients who are inadequately informed, unmotivated, or passive (812). Moreover, effective personal care marked by participatory decision-making is linked not only to improved psychosocial and functional outcomes, but also to physiologic outcomes in various chronic diseases (10,11). This suggests that dialysis units with low patient activation scores might consider participatory decision-making approaches to potentially enhance their SMRs.

In addition to patient-level characteristics, we identified physician-level factors that were associated with facility-level SMRs. Compared with high-mortality units, staff in low-mortality units rated their physicians as better able to interact with their patients with care and respect and more willing to spend time with the dialysis staff without feeling “bugged.” Of note, the staff ratings of physician technical skill or knowledge were not predictive of SMRs. Instead, characteristics relating to interpersonal relationships and communication were predictive of SMRs, indicating that communication, shared expectations, and plan coordination among staff may be important to enhance outcomes. However, it is notable that these physician-level factors did not remain predictive after adjustment in our hypothesis-generating exploratory regression model. This suggests that although physician factors are likely important, they may be of relatively lower importance in driving SMRs compared with other factors, including patient characteristics, dietician characteristics, and facility practices.

The importance of coordinated care is further emphasized by our finding that low-mortality units report more frequent multidisciplinary care conferences. In addition, low-mortality units convene multidisciplinary conference sooner after dialysis patients return to the facility after hospitalization and perform medication reconciliation more frequently than high-mortality units. Multidisciplinary conferences are an essential component of patient care and are required for dialysis facility reimbursement by the Centers for Medicare and Medicaid Services. Although the enhanced coordination of care is a touted benefit of these conferences, we have previously documented uncertainty among dialysis providers regarding the true effect of these conferences on outcomes (6). Our results here suggest that frequent and timely conferences may be associated with improved mortality. Although this link may not be causal, the relationship suggests that efforts to enhance coordination, such as timely and frequent multidisciplinary conferences, may improve survival.

We found that personnel in top-performing facilities perceive a higher quality of staff management and education. Specifically, top units are more likely to call for per diem help if there is unexpected short staffing and have a higher perceived quality of continuing medical education programs for staff. When coupled with the finding that physicians and staff are perceived to get along better in the top-performing units, these data suggest that low-mortality facilities have a more staff-oriented and friendly environment marked by better perceived staffing, a more communal and respectful work place, and a stronger emphasis on quality educational programs. This suggests that dialysis managers should aim to formally identify and correct nonadherence with interpersonal and attitudinal best practices. Doing so may not only improve morale, but may in fact improve outcomes, although this will require empirical testing in a prospective interventional study.

Our analysis has important limitations. First, this is a cross-sectional exploratory study, so we can only comment on associations, not causations. Future research should measure the longitudinal predictive value of the factors identified in this hypothesis-generating study. Second, although we used a comprehensive approach to identify candidate best practices in Phase I to II of IBPiD (6), it is likely that key clinical elements are nonetheless missing. Third, we relied on staff reporting about practices in their facility. It is possible that staff perceptions may not be wholly accurate. However, we sought data from multiple perspectives, so the principle of regression to the mean should tend to stabilize biases from any particular group. Fourth, we did not directly collect data from patients. Although our study did not aim to perform patient-level interviews, we did draw inferences from patient populations from staff reports; it would be ideal to also directly consider patient data. Future research should also incorporate patient-reported data to more optimally predict SMRs and should consider use of hierarchical models to establish how much each level of analysis (e.g., patient, staff, physicians, facility-level) affects overall SMRs. This would require a larger sample size of units.

In conclusion, we found that dialysis facilities with below-expected mortality report that patients in their unit are more highly activated and engaged, physician communication and interpersonal relationships are stronger, dieticians are more resourceful and knowledgeable, and coordination and staff management are superior compared with facilities with above-expected mortality. This suggests that SMR outcomes are enhanced by processes reflecting a coordinated, multidisciplinary environment. Future research should measure the prospective predictive validity of these factors in a larger sample of dialysis facilities.

Appendix.Appendix.Appendix.Appendix.
Candidate best practices perceived to improve outcomes in dialysis facilities, arranged according to major and minor domains.

Disclosures

None.

Acknowledgments

Support for this investigator-initiated study was provided by a research grant from Amgen, Inc. Dr. Gitlin is an employee of Amgen, Inc. The principal investigator, Dr. Spiegel, maintained full control over all aspects of the study design, implementation, data collection, data analysis, data interpretation, and manuscript preparation. Dr. Spiegel had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. The contributions of individual authors is as follows: A.D.—study design, study implementation, data collection, data interpretation, and manuscript preparation; R.B.—study design, study implementation, data collection, data analysis, and manuscript review; A.N.—study design, data interpretation, and manuscript review; P.Z.—data collection/provision, data interpretation, and manuscript review; T.P.—data collection/provision, data interpretation, and manuscript review; J.M.—data collection/provision, data interpretation, and manuscript review; S.B.—study design, data interpretation, and manuscript review; M.S.—study design, data interpretation, and manuscript review; O.K.—study design, data interpretation, and manuscript review; M.G.—data interpretation and manuscript preparation; J.T.—study implementation and data collection; B.S.—study design, study implementation, data collection, data analysis, data interpretation, manuscript preparation, manuscript approval, and guarantor of article. The opinions and assertions contained herein are the sole views of the authors and are not to be construed as official or as reflecting the views of the Department of Veteran Affairs. This study endorsed and supported by the Renal Physicians Association and the American Nephrology Nurses Association. Special thanks to the participating dialysis organizations—DCI; Renal Ventures Management LLC, and Satellite Healthcare.

Footnotes

Published online ahead of print. Publication date available at www.cjasn.org.

See related editorial, “Processes of Care and Reduced Mortality among Hemodialysis Patients in the United States,” on pages 1905–1907.

Access to UpToDate on-line is available for additional clinical information at http://www.cjasn.org/

References

1. Wolfe RA.: The standardized mortality ratio revisited: Improvements, innovations, and limitations. Am J Kidney Dis 24: 290–297, 1994. [PubMed]
2. Wolfe RA, Held PJ, Port FK.: Calculation and public use of the unit-specific standardized mortality ratio. Am J Kidney Dis 38: 212–213, 2001. [PubMed]
4. McClellan WM, Flanders WD, Gutman RA.: Variable mortality rates among dialysis treatment centers. Ann Intern Med 117: 332–336, 1992. [PubMed]
5. Goodkin D.: Does case mix explain the differences in dialysis mortality rates around the world? A report from the Dialysis Outcomes and Practice Patterns Study (DOPPS). IKidney—Nephrology Incite. Spring: 2005. (13)
6. Desai AA, Bolus R, Nissenson A, Bolus S, Solomon MD, Khawar O, Gitlin M, Talley J, Spiegel BM.: Identifying best practices in dialysis care: Results of cognitive interviews and a national survey of dialysis providers. Clin J Am Soc Nephrol 3: 1066–1076, 2008. [PMC free article] [PubMed]
7. Cohen J.: Statistical Power Analysis for the Behavioral Sciences, London, Academic Press, 1969
8. Greenfield S, Kaplan S, Ware JE., Jr: Expanding patient involvement in care. Effects on patient outcomes. Ann Intern Med 102: 520–528, 1985. [PubMed]
9. Greenfield S, Kaplan SH, Ware JE, Jr, Yano EM, Frank HJ.: Patients' participation in medical care: Effects on blood sugar control and quality of life in diabetes. J Gen Intern Med 3: 448–457, 1988. [PubMed]
10. Kaplan SH, Gandek B, Greenfield S, Rogers W, Ware JE.: Patient and visit characteristics related to physicians' participatory decision-making style. Results from the Medical Outcomes Study. Med Care 33: 1176–1187, 1995. [PubMed]
11. Kaplan SH, Greenfield S, Gandek B, Rogers WH, Ware JE., Jr: Characteristics of physicians with participatory decision-making styles. Ann Intern Med 124: 497–504, 1996. [PubMed]
12. Kaplan SH, Greenfield S, Ware JE., Jr: Assessing the effects of physician-patient interactions on the outcomes of chronic disease. Med Care 27[Suppl 3]: S110–S127, 1989. [PubMed]

Articles from Clinical Journal of the American Society of Nephrology : CJASN are provided here courtesy of American Society of Nephrology
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...