Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Int J Nurs Stud. Author manuscript; available in PMC 2010 Jul 1.
Published in final edited form as:
PMCID: PMC2700208

Nurse staffing and patient outcomes in Belgian acute hospitals

Cross-sectional analysis of administrative data



Studies have linked nurse staffing levels (number and skill mix) to several nurse-sensitive patient outcomes. However, evidence from European countries has been limited.


This study examines the association between nurse staffing levels (i.e. acuity-adjusted Nursing Hours per Patient Day, the proportion of registered nurses with a Bachelor’s degree) and 10 different patient outcomes potentially sensitive to nursing care.


Cross-sectional analyses of linked data from the Belgian Nursing Minimum Dataset (general acute care and intensive care nursing units: n = 1403) and Belgian Hospital Discharge Dataset (general, orthopedic and vascular surgery patients: n = 260,923) of the year 2003 from all acute hospitals (n = 115).


Logistic regression analyses, estimated by using a Generalized Estimation Equation Model, were used to study the association between nurse staffing and patient outcomes.


The mean acuity-adjusted Nursing Hours per Patient Day in Belgian hospitals was 2.62 (S.D. = 0.29). The variability in patient outcome rates between hospitals is considerable. The inter-quartile ranges for the 10 patient outcomes go from 0.35 for Deep Venous Thrombosis to 3.77 for failure-to-rescue. No significant association was found between the acuity-adjusted Nursing Hours per Patient Day, proportion of registered nurses with a Bachelor’s degree and the selected patient outcomes.


The absence of associations between hospital-level nurse staffing measures and patient outcomes should not be inferred as implying that nurse staffing does not have an impact on patient outcomes in Belgian hospitals. To better understand the dynamics of the nurse staffing and patient outcomes relationship in acute hospitals, further analyses (i.e. nursing unit level analyses) of these and other outcomes are recommended, in addition to inclusion of other study variables, including data about nursing practice environments in hospitals.

Keywords: Nurse staffing, Patient outcomes, Intensity of nursing care

What is already known about the topic?

  • A vast number of pre-dominantly North-American studies have found that nurse staffing levels are often, but not always, tied to patient outcomes in acute care hospitals.
  • Evidence for European health care systems is limited. Minimum staffing ratios and a mechanism to correct for differences in the intensity of nursing care are imbedded in the Belgian hospital financing system.

What this paper adds?

  • Substantial variation in patient outcomes exists. The variation in nurse staffing levels in Belgian hospitals is larger within, than between hospitals.
  • This nationwide study in Belgium does not confirm the North-American findings that acute care hospitals with the most (or best trained) nursing staff have better patient outcomes than those with less (or worst trained) nursing staff.
  • The findings draw attention to the importance of further and increased efforts to study the link between nurse staffing and patient outcomes in different health care settings and in different ways.

1. Background

Research over the past two decades has indicated that healthcare systems worldwide show wide variations in practice that clearly jeopardize patient safety (Brennan et al., 1991; Vincent et al., 2001; Wilson et al., 1995). The availability of nurses, the organization of the nursing workforce, and the work environment in which nurses practice have gained increasing attention, particularly since nurses represent the single largest group of providers in hospital settings (Needleman et al., 2007). A critical mass of literature has indicated that higher nurse staffing levels in North-American hospitals are associated with decreases in mortality and adverse patient events (Aiken et al., 2002a; Estabrooks et al., 2005; Kane et al., 2007; Needleman et al., 2002; Tourangeau et al., 2006). European countries have differently organized health systems (i.e. more homogenously organized healthcare systems that are less sensitive to free market forces). Therefore, it can be questioned that North-American evidence is applicable to the European setting. While nurse survey data from across Europe are suggestive of work environment problems leading to difficulties for hospitals in recruiting and retaining highly qualified nurses (Aiken et al., 2002b; Estryn-Behar et al., 2007; Gunnarsdottir et al., 2007), very little is known about the relationship between staffing levels and patient outcomes in European hospitals. Two research reports examining the association between nurse staffing levels and patient outcomes in Europe are consistent with the North-American evidence (Hugonnet et al., 2007; Rafferty et al., 2007). The U.K.-study partially replicated a U.S. reference study (Aiken et al., 2002a) in 30 NHS-trusts. Hospitals with the most favourable staffing levels had consistently lower mortality and failure-to-rescue rates than those hospitals with less favourable staffing levels. Nurses employed in high staffed hospitals reported less job dissatisfaction, nurse burnout and problems with the quality of patient care. These associations were in contrast to the U.S.-study not clearly linear. Rafferty et al. (2007) found the clearest association between the staffing levels and patient and nurse outcomes when hospitals from the upper quartile (“worst” staffed hospitals) were compared with hospitals from the lowest quartile (“best” staffed hospitals). The associations were not always significant when the hospitals from the middle quartiles were compared with the “best” staffed hospitals. Hugonnet et al. (2007) showed in a single-centre study in Switzerland, that nurse staffing levels were systematically lower 2–4 days before nosocomial infections on intensive care units. The authors dichotomized the nurse-to-patient ratio at the upper quartile (“worst” staffing) to illustrate this association. Nevertheless, more multi-center research in different countries is desirable to strengthen the European evidence.

This study addresses the gap by examining data from Belgium. Staffing levels in Belgian hospitals are influenced by at least two factors inherent in the hospital financing system. A first is a base level of hospital funding that sets nurse staffing ratios at certain minima (e.g. 12 nurse full-time-equivalents per 30 surgical beds). A second mechanism relates to the allocation of a part (6.5%) of the budget on the basis of nursing acuity (differences in the intensity of nursing care activities). The intensity of nursing services is measured by a validated tool, the Belgian Nursing Minimum Data Set (B-NMDS) for medical, surgical, pediatric and intensive care units (Sermeus et al., 2007a).

A recent Belgian survey (Milisen et al., 2006), found that nurses experience the insufficient availability of nursing staff as an obstacle for providing good quality of nursing care, as they do in other countries (Aiken et al., 2002b). Furthermore, Van den Heede et al. (2006) found a high prevalence of adverse outcomes with a substantial variation across hospitals based on an analysis of the Belgian Hospital Discharge Data Set (B-HDDS). In this context, worried by a deepening nursing shortage, the Belgian Ministry of Science Policy in collaboration with the Ministry of Public Health commissioned a study to examine the relationship between nurse staffing levels across hospitals and the outcome rates among patients, using administrative data representing all acute care hospitals in the country.

2. Methods

2.1. Data sources

Data sources analyzed here were the B-NMDS and the B-HDDS from the year 2003. Contribution of data to the B-NMDS and B-HDDS has been required from all acute hospitals since 1988 and 1990, respectively. Both databases were stripped of both hospital and patient identifiers in order to assure anonymity of hospitals and patients.

In the B-HDDS, data regarding each hospitalization (whether a 1-day or conventional stay) are summarized in a single episode record. The dataset includes facility identifiers indicating where the hospitalization occurred, patient demographics, characteristics of the admission, principal and secondary diagnosis and procedure codes (International Classification of Diseases, 9th Version or ICD9-CM), indication of a stay on intensive care, length of stay, discharge status and destination, and DRG assignment (All Patients Refined Diagnosis Related Groups v15.0 or APR-DRG v15.0) (Van den Heede et al., 2006).

The B-NMDS contains data regarding staffing patterns and the nature of nursing care provided in each Belgian acute care hospital. The Ministry of Public Health randomly samples 5 days from the first to the 15 days of March, June, September, and December and instructs hospitals to collect data of two types and relay them to a national database. Firstly, for every patient admitted in 1-day units or inpatient hospitalization units a list of nursing interventions (i.e. 23 items: care relating to hygiene, care relating to mobility, care relating to elimination, care relating to feeding, tube feeding, mouth care, prevention of pressure sores: changing position, assistance in getting dressed, care of patient with tracheotomy or endotracheal tube, nursing admission assessment, training in activities of daily living, emotional support, care of a disoriented patient, isolation for preventing contamination, monitoring vital signs, monitoring clinical signs, cast care, taking blood samples, intramuscular or subcutaneous medication management, intravenous medication management, infusion therapy, surgical wound care, traumatic wound care) is scored. The 23 nursing interventions can be summarized, by means of non-linear principal component analysis, in one validated measure of the intensity of nursing care (Sermeus et al., 2008). Secondly, for every nursing unit where those patients are treated the number of nursing staff that is directly involved in patient care is recorded. In Belgium two educational pathways exist to enter the nursing profession: diploma level and bachelor level. Both degrees meet the criteria to practice nursing as a registered nurse and limited differentiation between those two types of nurses is made in the execution of one’s professional duties. On every B-NMDS registration day the hours worked by nursing staff on each nursing unit are registered. The hours of registered nurses are categorized as diploma level or bachelor level nurses. In addition, also the hours of nursing aides are registered. Additional details regarding the content of the database and the wording of questions used in the data collection can be found in Sermeus et al. (2008).

Entries in both datasets are systematically audited on a regular basis by the Ministry of Public Health before release to policy and researchers (Sermeus et al., 2008; Van den Heede et al., 2006).

2.2. Study population

We obtained data on all 115 Belgian acute hospitals for the year 2003. From the B-HDSS, we selected, using the APR-DRG system (see Box 1), all patients 20–85 years old hospitalized in these institutions for a sample of general, orthopedic and vascular surgeries using a list mirroring that from prior international studies (Aiken et al., 2002a, 2003; Rafferty et al., 2007). 43 specific APR-DRG were included(see Box 1). A surgical patient population was selected because previous research illustrated that nurse staffing is more consistently tied to surgical outcomes (Kane et al., 2007) and risk-adjustment procedures are superior for the surgical patient population (Iezzoni, 2003). The final sample was restricted to acute in-patient hospitalizations and included 260,923 patients.

Box 1APR-DRG included in the analysis

168 Major Thoracic Vascular Procedures; 169 Major Abdominal Vascular Procedures; 172 Amputation for Circulatory System Disorder Except Upper Limb & Toe; 178 Upper Limb & Toe Amputation for Circulatory System Disorders; 180 Other Circulatory System Disorders Procedures; 220 Major Stomach, Esophegeal & Duodenal Procedures; 221 Major Small & Large Bowel Procedures; 223 Minor Small & Large Bowel Procedures; 224 Peritoneal Adheolysis; 225 Appendectomy; 226 Anal & Stomal Procedures; 227 Hernia Procedures Except Inguinal & Femoral; 228 Inguinal & Femoral Hernia Procedures; 229 Other Digestive System Procedures; 260 Pancreas, Liver & Shunt Procedures; 261 Major Biliary Tract Procedures; 262 Cholestyctectomy Except Laparoscopic; 263 Laparoscopic Cholestyctectomy; 264 Other Hepatobiliary & Pancreas Procedures; 300 Bilateral & Multiple Major Joint Procedures of Lower Extremity; 301 Major Joint & Limb Reattachment Procedures of Lower Extremity for Trauma; 302 Major Joint & Limb Reattachment Procedures of Lower Extremity Except for Trauma; 303 Dorsal & Lumbar Fusion Procedure for Curvature of Back; 304 Dorsal & Lumbar Fusion Procedure for Except for Curvature of Back; 305 Amputation for Musculoskeletal System & Connective Tissue Disorder; 308 Hip & Femur Procedures Except Major Joint for Trauma; 309 Hip & Femur Procedures Except Major Joint for Non-Trauma; 310 Back & Neck Procedures Except Dorsal & Lumbal Fusion; 312 Skin Graft & Wound Debridement Except for Open Wound, Ms & Connective Tissue Disorder, Except Hand; 313 Knee & lower Leg Procedures Except Foot; 314 Foot Procedures; 315 Shoulder, Elbow & Forearm procedures; 317 Soft Tissue Procedures; 318 Removal of Internal Fixation Device; 320 Other Musculoskeletal System and Connective Tissue Procedures; 360 Skin Graft & Wound Debridement for Skin Ulcer & Cellulitis; 361 Skin Graft & Wound Debridement Except for Skin Ulcer & Cellulitis; 362 Mastectomy Procedures; 400 Amputation Lower Limb for Endocrine Nutritional and Metabolic Disorders; 401 Adrenal & Pituitary Procedures; 402 Skin Graft and Wound Debridement for Endocrine Nutritional and Metabolic Disorders; 403 Procedures for Obesitas; 405 Other Endocrine Nutritional and Metabolic Procedures.

2.3. Measures

To select variables for this study a literature review and international Delphi survey were carried out (Van den Heede et al., 2007). This resulted in a final list of three variable types: 32 patient outcomes, 10 nurse staffing variables and 29 background variables (patient, nurse and institutional characteristics that are potential confounders of relationships between staffing and outcomes). It was evaluated which of the variables from this list could be derived from the Belgian datasets on hand. Internationally available algorithms (Department of Health and Human Services, 2007;Needleman et al., 2002; Silber et al., 1997) were used as a starting point to develop the technical specifications of these variables (algorithms) (Sermeus et al., 2007b). The research team, with the support of seven experts on the B-HDDS made adjustments to these international algorithms for use in the present study (Sermeus et al., 2007b).

2.4. Patient outcomes

Ten of the 32 indicators suggested by the Delphi-panel (Van den Heede et al., 2007) could be derived from the B-HDDS. We included one safety/integrity measure (Pressure Ulcer), three complication measures (Deep Venous Thrombosis, Shock or Cardiac Arrest, Postoperative Respiratory Failure), four infection measures (Postoperative complications & infections, Urinary Tract Infections, Hospital-acquired Pneumonia, Ventilator-associated Pneumonia, Hospital-acquired sepsis), ‘In-hospital Mortality’ and ‘failure-to-rescue’. Failure-to-rescue is defined as the probability of death after a complication. The original definition of failure-to-rescue was used (Silber et al., 1992, 1997). Relevant ICD9-CM codes and APR-DRG, used in the patient outcomes algorithms, are provided in Appendix 1.

2.5. Nurse staffing

Two nurse staffing variables are calculated from the B-NMDS (Appendix A): ‘Nursing Hours per Patient Day’ (NHPPD) and ‘Proportion of registered nurses with a Bachelor’s degree’. NHPPD is the hours of care provided by registered nurses divided by the number of patients being cared for. The ‘Proportion of registered nurses with a Bachelor’s degree’ is obtained by calculating the proportion of staffed registered nurses hours performed by nurses with at least a Bachelor’s degree. From the B-NMDS only information from acute care units (internal medicine units: n = 525, general surgical units: n = 511, units with internal medicine and surgical patients: n = 140, intensive care units: n = 227) was retained resulting in a sample that contains 583,429 inpatient days for 267,398 patients and 1,403 acute care nursing units.

Appendix A
Definition and data sources used for the studied measures

Consistently collected data about nursing care intensity in Belgian hospitals allowed us to compute uniquely adjusted staffing levels. Staffing needs to vary, not only in the number of patients being cared for, but also in the type of care provided for each of those patients. As nursing care intensity increases, the number of nursing staff required to care for patients will increase. However, it is not common to integrate data about workload or patient acuity into the nurse staffing and patient safety research as this requires hospitals to use uniform workload measurement systems. Belgium can deliver a unique contribution to this research area by introducing the B-NMDS in the nurse staffing and patient outcomes research. Sermeus et al. (2008) showed how to derive a valid measure of the intensity of nursing care from the B-NMDS. Together with information about the hospital type (general vs. academic), service type (intensive care vs. acute general care), type of day (week vs. weekend), the intensity of care measure was used to calculate the expected number of nursing staff per nursing unit per B-NMDS registration day (Van den Heede et al., 2008). These expected hours of nursing staff are, like the observed hours of nursing staff, aggregated (i.e. summed up) on the hospital level. For each hospital the observed nursing staff hours are divided by the expected nursing staff hours. These ratios are multiplied by the national mean NHPPD to obtain an acuity-adjusted NHPPD per hospital (see Appendix A). In other words, we standardized the number of nursing staff for differences in patient acuity.

2.6. Background variables

Background variables for use in adjusting associations between staffing and outcomes for possible confounders, were drawn from both major data sources to the fullest extent possible. The B-HDDS contains information about patient-clinical characteristics (age, gender, type of illness, severity of illness, co-morbidities and admission type) and structural characteristics of organizations (institution type, hospital size, technological sophistication). However, nurse characteristics (experience and employment status) and indicators on the organizational process and work environment are not available in the databases under study.

Patient characteristics available in the B-HDDS were used to develop risk-adjustment models to correct for the influence of the differences across hospitals in the risks of patients for each type of adverse events. To identify model variables, we drew heavily upon existing literature. The variables included age, sex, surgery type and dummy variables indicating the presence of chronic pre-existing health conditions (co-morbidities) reflected in the ICD-9 codes of the B-HDDS. For failure-to-rescue and mortality the co-morbidities defined by Silber (1997) were used. For the other outcomes the co-morbidities defined by Elixhauser et al. (1998) were applied. The final set of variables was determined by a selection process similar to that used by Silber et al. (1997). The c-statistic, a measure that evaluates how well the risk-adjustment model predicts which patient experience adverse events, were 0.96 and 0.88 for patient mortality and failure-to-rescue, respectively. The c-statistics for the other patient outcomes were ranging from 0.80 to 0.92 (the highest possible value is 1.00) (Iezzoni, 2003).

We controlled the results also for three hospital characteristics: hospital type (i.e. academic or general hospitals); hospital size (i.e. number of beds) and technology level of the hospital (i.e. performing cardiac procedures). In the analysis that studies the relationship between acuity-adjusted NHPPD and the different patient outcomes only the hospital size and technology level of the hospital were taken into account as the hospital type is already taken into account in the calculation of acuity-adjusted NHPPD (Van den Heede et al., 2008).

2.7. Statistical analyses

Acuity-adjusted NHPPD and percentage of registered nurses with at least a Bachelor’s degree were measured at the hospital level. The patient outcomes, together with the risk-adjustment variables, were measured at the patient level. A two-level data structure (i.e. hospitals and patients) was used in the analyses because we could not exactly link the patient level data (derived from the B-HDDS) to the nursing units (derived from the B-NMDS), where those patients stayed for the patient population under study. Logistic regression analyses were used to investigate the association between nurse staffing variables and patient outcomes. All logistic regression models were estimated by using a Generalized Estimation Equation Model (GEE) to adjust standard errors of the parameter estimates for the clustering of patients within hospitals. All analyses were performed using SAS v9.1 (SAS Institute, 2001).

3. Results

3.1. Descriptive results

Tables Tables11 through through44 provide summary statistics for the variables in our study. In terms of hospital characteristics it is illustrated that more than half of the patients (55.3%) were treated in a hospital with 450 beds or more, 36.1% of the patients were treated in high technology hospitals and 10.8% of the patients were treated in one of the country’s 7 (6.1%) academic medical centers (Table 1).

Table 1
Hospital characteristics study sample
Table 4
Nurse staffing variables

The median age of patients in this study was 56 years (inter-quartile range or IQR = 28). In Table 2 it is illustrated that 46.6% of patients were male. The median length of stay was 3 days (IQR = 6). The emergency admission rate was 22.5%. The In-hospital Mortality for patients admitted through emergency was 3.5% compared to 0.4% for patients not admitted through the emergency department. A total of 30,885 patients (11.8%) stayed on an intensive care nursing unit. The In-hospital Mortality of patients with a stay on intensive care nursing units is 6.5%, compared to 0.4% among patients without a stay on intensive care nursing units. More than half of the patients (54.7%) were classified in an orthopedic surgery DRG. The second largest group (35.4%) is that of patients that undergo surgery for the digestive or hepatobiliary system. The vascular surgery patients have the highest in-hospital mortality rate (8.9%).

Table 2
Characteristics of surgical patients included in staffing and outcome analysis

The incidence of 10 adverse outcomes is used to measure the impact of nurse staffing on patients. Table 3 presents the overall percentage, the number of cases in numerator and denominator and the median and IQR to indicate the spread across hospitals for each patient outcome. Among the adverse outcomes, the composite measure ‘Postoperative complications and infections’, Hospital-acquired Pneumonia and Pressure Ulcers were the most frequent. Failure-to-rescue (deaths among the subgroup of patients with complications) had the highest rate. The percentage of patients dying in the hospital was 1.13%. All other outcomes had rates of below 1%. The IQR for the 10 patient outcomes go from 0.35 for Deep Venous Thrombosis to 3.77 for failure-to-rescue.

Table 3
Descriptive statistics patient outcomes

In terms of staffing, hospitals recorded a mean of 2.73 (S.D. = 0.40) NHPPD (Table 4). The mean acuity-adjusted NHPPD was 2.62 (S.D. = 0.29). The descriptive statistics (i.e. standard deviation and IQR) demonstrate that the variability in nurse staffing, measured at the hospital level, decreases when the number of nursing staff is adjusted for intensity of nursing care. When the hospital nurse staffing levels are calculated with data from intensive care nursing units only the NHPPD is 4.6 times more (i.e. 10.39 vs. 2.27 NHPPD) than when this is calculated for all acute nursing units (except intensive care). Further, more nurses (1.7 times more) working in intensive care units held at least a Bachelor’s degree compared to other acute nursing units. In Appendix B it is illustrated that the variability in nurse staffing measured at the hospital level, was lower than the variability per nursing unit and per nursing unit per registration day.

Appendix B
Nurse staffing variables, measured on the level of the nursing unit

3.2. Association nurse staffing and patient outcomes

Table 5 presents the results of the impact of average nurse staffing levels per hospital on adverse outcomes, corrected for differences in intensity of nursing care, patient as well as hospital characteristics. No significant relationship was found between the acuity-adjusted NHPPD, proportions of registered nurses with at least a Bachelor’s degree and the 10 patient outcomes.

Table 5
Relationship between nurse staffing and patient outcomes, using logistic regression analyses

4. Discussion

Our study documented substantial differences in patient outcomes across Belgian hospitals, possibly the result of differences in quality of care. The study has a number of strengths including carefully cleaned and validated hospital outcomes data, a nationwide study population, a database with systematically collected data about nursing staff involved in direct care, a solid risk-adjustment model and a unique validated control measure for the intensity of nursing care. However, we were not able to show that these differences in outcomes were associated with variation across hospitals in nurse staffing or nurse education measured at the hospital level.

We tested the accuracy of our findings by some re-analyses and explored a number of strategies/reasons to explain why the null hypotheses could not be rejected. We repeated the analyses using a two-level structure multi-level model including the hospital and patient level to fully account for the fact that patients are clustered in a hospital and thus violating the assumption of independence (e.g. patients within a particular hospital are cared for by the same nurses, the same hospital policy). This multi-level approach untangles sources of variation in patient outcomes and partitions it into two distinct levels: variation attributable to patient and hospital characteristics (Luke, 2004).

One possible explanation for the ‘null result’ in the Belgian setting is that the variation in nurse staffing among hospitals is smaller than in other countries. After all, two other reports in a European setting (Hugonnet et al., 2007; Rafferty et al., 2007) did found an association between nurse staffing and patient outcomes. In the Swiss study (Hugonnet et al., 2007) the nurse staffing levels were dichotomized. The 75th percentile of nurse-to-patient ratios was used to distinguish high from low nurse-to-patient ratios. However, the restriction of the latter study to intensive care units from one hospital hampers the comparison with our study results. The U.K.-study, on the other hand, allows better comparison. Like in our study the nurse staffing levels from different specialties were aggregated on the hospital level and compared to patient outcomes from a (highly similar) selection of surgical procedures. The association found in the U.K.-study was only consistent when hospitals from the lowest quartile (“best” staffed) were compared to hospitals from the upper quartile (“worst” staffed). This indicates that the variation in nurse staffing levels in U.K. hospitals (in contrast with the U.S.-study) is mainly attributed to the staffing levels from hospital from the upper quartile. However, using four categories of nurse staffing (i.e. quartiles of NHPPD) in a re-analysis of the study results did not change our findings. The limited variation in Belgian nurse staffing levels (measured at the hospital level) can be attributed to the financing mechanism. In Belgium a system of minimum nurse staffing ratios at baseline is integrated in the Belgian hospital financing system. Furthermore, when variation in NHPPD exists, it is most likely to be attributed to differences in intensity of nursing care and thus justified. Hospitals with a high intensity of nursing care get a higher budget for nurse staffing. By correcting our staffing measure for the intensity of care (i.e. acuity-adjusted NHPPD) the minimal variation that exists in the uncorrected NHPPD is leveled out further.

Furthermore it can be hypothesized that the variation in nurse staffing within hospitals in Belgium is bigger than in other countries. It is shown that in Belgium the IQR of the acuity-adjusted NHPPD, when measured per nursing unit, is twice the IQR of the acuity-adjusted NHPPD, when measured on the hospital level (Table 4 and Appendix B). Nevertheless, staffing data were aggregated across observation days and specialties at the level of the hospital, as in most (if not all) multi-center studies, because we do not know what trajectory patients, whose outcomes were studied, followed during their hospital episode. This methodology requires substantial variation (which is the case in North-American countries) in nurse staffing on the hospital level. In healthcare systems with homogenous staffing levels per hospital (e.g. Belgium), however, it might be necessary to measure nurse staffing at the nursing unit level in order to be able to illustrate a relationship between nurse staffing and patient outcomes.

The finding of the great differences in nurse staffing levels between intensive care and other acute care nursing units resulted in two separate analyses. First the association between nurse staffing levels from intensive care units (aggregated on the hospital level) with patient outcomes from patients that stayed on intensive care were studied. In a second analysis the association between nurse staffing levels from other acute nursing care units (aggregated on the hospital level) and patient outcomes from patients that did not stay on intensive care was studied. These additional analyses did not change our primary findings.

Using only nursing care hours provided by registered nurses omits an important factor (i.e. nursing aides) from the analyses. Thus, nursing aides’ hours were introduced into the calculation of acuity-adjusted NHPPD. This re-analysis did not result in different findings.

It is recognized that the results could be highly influenced by problems inherent to the use of administrative databases, such as inadequate reporting and questions of data integrity (Iezzoni, 2003). The B-HDDS from which we derived our patient outcomes did not distinguish whether a complication was present on admission. Therefore, it is difficult to distinguish co-morbidities (present on admission) from adverse events (developed during the hospitalization period) as for both measures the same secondary diagnosis codes are used. The access to B-HDDS data from prior years could have helped to deal with this issue by excluding any co-morbid conditions and/or complications that appeared for each patient in prior year data. However, we did not have these data for this study. Furthermore, the use of both databases (B-NMDS and B-HDDS) for the hospital financing contains the risk of “upcoding” or “DRG creep” (a tendency to preferentially code secondary diagnoses and nursing activities perceived to make a case for increased hospital charges or budgets). The Belgian datasets used here contained a wealth of information about patients, nursing services and hospitals. However, not all factors relevant to the relationship between nurse staffing and patient outcomes (Van den Heede et al., 2007) could be taken into account. It can be concluded from the results of a study by Aiken et al. (2002b) that joint effects of staffing and various practice environment elements on patient outcomes exist. For that reason, the lack of a ‘nursing practice environment’ measure like the ‘Nursing Work Index’ (Lake, 2007) in the databases used for this study was an important shortcoming.

The APR-DRG and co-morbidities adjustment cannot be a complete control for all levels of risk leading to the development of adverse outcomes, nor are the studies available, or likely to tell us which risk adjuster is best (Iezzoni, 2003). Therefore, in a subsequent analysis round a combination of APR-DRG with the four risk-of-mortality categories (for In-hospital Mortality and failure-to-rescue) or severity-of-illness categories (for the other eight patient outcomes) was used as an alternative method of risk-adjustment. For these analyses the severity-of-illness and risk-of-mortality categories were re-calculated without the secondary diagnoses that are used in the patient outcomes algorithms. These extra analyses resulted in highly similar results.

Finally, the study population in Belgium differs from study populations in other countries. The In-hospital Mortality in this study (1.13%) is lower than the values of 2.0% and 2.3% reported by Aiken et al. (2002a, 2003) and Rafferty et al. (2007), respectively. The selection of orthopedic, vascular and general surgical patients may be a selection of patients with a relative low risk of dying in the Belgian context due to a different organization of healthcare. ‘Shoulder, Elbow and Forearm Procedures’, for example, have a 0.2% mortality rate in this study, as well as in the U.S.-study of Aiken et al. (2002a). Compared to the U.S.-study, where this patient group accounts for 3.4% of the sample, the figure in the reported study is 9.3%. This is an indication that substitution of classic hospital stays by 1-day clinics in Belgian hospitals is not so pronounced as in the U.S. Therefore, the analyses were repeated for a population more at risk. Patients undergoing surgical procedures classified in an APR-DRG that is present in at least 50% of the hospitals and has nationwide at least 10 fatalities, at least 100 cases and a mortality percentage of at least 0.5% were taken into account. Analyzing the relationship between nurse staffing and patient outcomes for this high volume and high risk population did not yield other conclusions.

5. Conclusion

This study incorporated two important corrections in the analysis of the relationship between nurse staffing and patient outcomes. The correction for the risk of patient outcomes, typically done by researchers, resulted in risk-adjusted patient outcome rates, which illustrates substantial differences between Belgian hospitals, possibly due to differences in care quality. A second correction included the adjustment of nurse staffing for the intensity of nursing care. The latter makes this study a unique contribution to the international body of knowledge. Despite these methodological improvements, our findings do not indicate an association between nurse staffing and the selected patient outcomes at the hospital level.

The absence of relationships between hospital nurse staffing levels and patient outcomes should not be construed as implying that nurse staffing does not affect patient outcomes in Belgian hospitals. The findings do suggest that a potential relationship in Belgium is more complex and needs further study because of greater uniformity in staffing levels across hospitals throughout the country due to a strong federal influence in hospital finance. To gain a better insight into the relationship between nurse staffing and patient outcomes it is recommended that this relationship is studied on the nursing unit level. The effect of nurse staffing, after all, is most direct at the patient care unit level (especially in homogenously organized healthcare systems like it is the case in Belgium). Furthermore, administrative databases can be improved. For instance, experts have made an increasingly strong case for including fields in secondary datasets that distinguish between secondary diagnoses that are present on admission from those that develop during hospitalizations to enable better risk-adjustment and more accurate ascertainment of complications (Pine et al., 2007).


This research was commissioned by the Belgian Federal Science Policy Office, in collaboration with the Belgian Federal Ministry of Public Health, in the framework of the research programme Agora.



Conflict of interest statement

None declared.

Funding source

The funding source had no active role in the study.

Ethical approval

The study committee appointed by the Ministry of Science Policy gave ethics clearance for us to carry out this study. The data provided by the Ministry of Public Health of Belgium did not contain information about the identity of patients or hospitals.


  • Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. The Journal of the American Medical Association. 2002a;288(16):1987–1993. [PubMed]
  • Aiken LH, Clarke SP, Sloane DM. Hospital staffing, organization, and quality of care: cross-national findings. International Journal for Quality in Health Care. 2002b;14(1):5–13. [PubMed]
  • Aiken LH, Clarke SP, Cheung RB, Sloane DM, Silber JH. Educational levels of hospital nurses and surgical patient mortality. The Journal of the American Medical Association. 2003;290(12):1617–1623. [PMC free article] [PubMed]
  • Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. The New England Journal of Medicine. 1991;324(6):370–376. [PubMed]
  • Department of Health and Human Services . Guide to Patient Safety Indicators. Agency for Healthcare Research and Quality; Rockville: 2007.
  • Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Medical Care. 1998;36(1):8–27. [PubMed]
  • Estabrooks CA, Midodzi WK, Cummings GG, Ricker KL, Giovannetti P. The Impact of hospital nursing characteristics on 30-day mortality. Nursing Research. 2005;54(2):74–84. [PubMed]
  • Estryn-Behar M, Van der Heijden BI, Oginska H, Camerino D, Le Nezet O, Conway PM, et al. The impact of social work environment, teamwork characteristics, burnout, and personal factors upon intent to leave among European nurses. Medical Care. 2007;45(10):939–950. [PubMed]
  • Gunnarsdottir S, Clarke SP, Rafferty AM, Nutbeam D. Front-line management, staffing and nurse-doctor relationships as predictors of nurse and patient outcomes. A survey of Icelandic hospital nurses International Journal of Nursing Studies 2007. (Epub ahead of print). [PubMed]
  • Hugonnet S, Villaveces A, Pittet D. Nurse staffing level and nosocomial infections: empirical evaluation of the case-crossover and case-time-control designs. American Journal of Epidemiology. 2007;165(11):1321–1327. [PubMed]
  • Iezzoni LI. Risk Adjustment for Measuring Health Care Outcomes. 3rd Health Administration Press; Chicago: 2003.
  • Kane RL, Shamliyan TA, Mueller C, Duval S, Wilt TJ. Nursing staffing and quality of patient care. Medical Care. 2007;45(12):1195–1204. [PubMed]
  • Lake ET. The nursing practice environment: measurement and evidence. Medical Care Research and Review. 2007;64(Suppl 2):104S–122S. [PubMed]
  • Luke DA. Multilevel Modeling. Sage, Thousand Oaks; California: 2004.
  • Milisen K, Abraham I, Siebens K, Darras E, Dierckx-de-Casterlé B. Work environment and workforce problems: a cross-sectional questionnaire survey of hospital nurses in Belgium. International Journal of Nursing Studies. 2006;43(6):745–754. [PubMed]
  • Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. The New England Journal of Medicine. 2002;346(22):1715–1722. [PubMed]
  • Needleman J, Kurtzman ET, Kizer KW. Performance measurement of nursing care: state of the science and the current consensus. Medical Care Research and Review. 2007;64(Suppl 2):10S–43S. [PubMed]
  • Pine M, Jordan HS, Elixhauser A, Fry DE, Hoaglin DC, Jones B, et al. Enhancement of claims data to improve risk adjustment of hospital mortality. The Journal of the American Medical Association. 2007;297(1):71–76. [PubMed]
  • Rafferty AM, Clarke SP, Coles J, Ball J, James P, McKee M, et al. Outcomes of variation in hospital nurse staffing in English hospitals: cross-sectional analysis of survey data and discharge records. International Journal of Nursing Studies. 2007;44(2):175–182. [PMC free article] [PubMed]
  • SAS Institute . The SAS System for Windows. SAS Institute; Cary: 2001.
  • Sermeus W, Gillet P, Tambeur W, Gillain D, Laport N, Michiels D, et al. KCE reports. 53A. Federaal Kenniscentrum voor de gezondheidszorg (KCE); Brussels: 2007a. Financing Nursing Care in Hospitals.
  • Sermeus W, Delesie L, Van den Heede K, Diya L, Lesaffre E. Measuring the intensity of nursing care: making use of the Belgian Nursing Minimum Data Set. International Journal of Nursing Studies. 2008;45(7):1011–1021. [PubMed]
  • Sermeus W, Van den Heede K, Vleugels A. Database Quality of Nursing Care AG/01/123. 2007b. 1–193.193Katholieke Universiteit Leuven; Leuven
  • Silber JH, Williams SV, Krakauer H, Schwartz JS. Hospital and patient characteristics associated with death after surgery. A study of adverse occurrence and failure to rescue. Medical Care. 1992;30(7):615–629. [PubMed]
  • Silber JH, Rosenbaum PR, Williams SV, Ross RN, Schwartz JS. The relationship between choice of outcome measure and hospital rank in general surgical procedures: implications for quality assessment. International Journal for Quality in Health Care. 1997;9(3):193–200. [PubMed]
  • Tourangeau AE, Cranley LA, Jeffs L. Impact of nursing on hospital patient mortality: a focused review and related policy implications. Quality and Safety in Health Care. 2006;15(1):4–8. [PMC free article] [PubMed]
  • Van den Heede K, Sermeus W, Diya L, Lesaffre E, Vleugels A. Adverse outcomes in Belgian acute hospitals: retrospective analysis of the national hospital discharge dataset. International Journal for Quality in Health Care. 2006;18(3):211–219. [PubMed]
  • Van den Heede K, Clarke SP, Sermeus W, Vleugels A, Aiken LH. International experts’ perspectives on the state of the nurse staffing and patient outcomes literature. Journal of Nursing Scholarship. 2007;39(4):290–297. [PubMed]
  • Van den Heede K, Diya L, Lesaffre E, Vleugels A, Sermeus W. Benchmarking nurse-staffing levels: applicability of the Belgian nursing minimum dataset for the development of a nationwide feedback tool Journal of Advanced Nursing 2008, in press. [PubMed]
  • Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. British Medical Journal. 2001;322(7285):517–519. [PMC free article] [PubMed]
  • Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The Quality in Australian Health Care Study. Medical Journal of Australia. 1995;163(9):458–471. [PubMed]
PubReader format: click here to try


Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...


  • Cited in Books
    Cited in Books
    PubMed Central articles cited in books
  • MedGen
    Related information in MedGen
  • PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...