Objective: To evaluate the performance of a scoring system (NOSEP) to predict nosocomial sepsis in neonates at the hospital where the score was developed (internal validation) and in an independent data set from other centers (external validation).
Design: Multiple center prospective cohort study.
Setting: Six neonatal intensive care units from the Flanders in Belgium.
Patients: We analyzed two groups of patients: 62 episodes of presumed nosocomial sepsis in the internal validation cohort and 93 episodes of presumed nosocomial sepsis in a multiple center external validation cohort.
Interventions: Assessment of the predictive power of the NOSEP score 24 hrs preceding sepsis workup and the patients' basic demographic characteristics and co-morbidity was performed. Diagnosis of nosocomial sepsis and the microbiology results were registered.
Main results: The NOSEP score's discriminative capability was very good in the internal validation (area under receiver operating characteristic curve = 0.73 +/- 0.08 [sem]). The NOSEP score performed satisfactory in the external validation (area under receiver operating characteristic curve = 0.66 +/- 0.06). The calibration capability in both validation sets as measured by goodness-of-fit tests (internal validation, p =.56; external validation, p =.48) was good. An improvement of the NOSEP score was obtained for the external centers by redefining the cut-off of the items of the NOSEP score (area under receiver operating characteristic curve for NOSEP-NEW-I = 0.71 +/- 0.05) or adding co-morbidity factors (area under receiver operating characteristic curve for NOSEP-NEW-II = 0.82 +/- 0.04), with good calibration performance (goodness-of-fit test, p >.50). Finally, the fit of the NOSEP score demonstrated no significant variation across subgroups of patients.
Conclusions: The predictive power of the original NOSEP score is very good in neonates at the original neonatal intensive care unit. In other neonatal intensive care units, its discriminatory performance is satisfactory but could be improved after modification of the variables in the model or adding additional variables. To use such a NOSEP score in other neonatal intensive care units, its accuracy has to be validated and adjusted if necessary.