Format

Send to

Choose Destination
Prehosp Emerg Care. 2014 Oct-Dec;18(4):495-504. doi: 10.3109/10903127.2014.916022. Epub 2014 May 30.

A comparative assessment of adverse event classification in the out-of-hospital setting.

Abstract

OBJECTIVES:

We sought to test reliability of two approaches to classify adverse events (AEs) associated with helicopter EMS (HEMS) transport.

METHODS:

The first approach for AE classification involved flight nurses and paramedics (RN/Medics) and mid-career emergency physicians (MC-EMPs) independently reviewing 50 randomly selected HEMS medical records. The second approach involved RN/Medics and MC-EMPs meeting as a group to openly discuss 20 additional medical records and reach consensus-based AE decision. We compared all AE decisions to a reference criterion based on the decision of three senior emergency physicians (Sr-EMPs). We designed a study to detect an improvement in agreement (reliability) from fair (kappa = 0.2) to moderate (kappa = 0.5). We calculated sensitivity, specificity, percent agreement, and positive and negative predictive values (PPV/NPV).

RESULTS:

For the independent reviews, the Sr-EMP group identified 26 AEs while individual clinician reviewers identified between 19 and 50 AEs. Agreement on the presence/absence of an AE between Sr-EMPs and three MC-EMPs ranged from κ = 0.20 to κ = 0.25. Agreement between Sr-EMPs and three RN/Medics ranged from κ = 0.11 to κ = 0.19. For the consensus/open-discussion approach, the Sr-EMPs identified 13 AEs, the MC-EMP group identified 18 AEs, and RN/medic group identified 36 AEs. Agreement between Sr-EMPs and MC-EMP group was (κ = 0.30 95%CI -0.12, 0.72), whereas agreement between Sr-EMPs and RN/medic group was (κ = 0.40 95%CI 0.01, 0.79). Agreement between all three groups was fair (κ = 0.33, 95%CI 0.06, 0.66). Percent agreement (58-68%) and NPV (63-76%) was moderately dissimilar between clinicians, while sensitivity (25-80%), specificity (43-97%), and PPV (48-83%) varied.

CONCLUSIONS:

We identified a higher level of agreement/reliability in AE decisions utilizing a consensus-based approach for review rather than independent reviews.

KEYWORDS:

adverse events; measurement; reliability; safety

PMID:
24878451
PMCID:
PMC4410777
DOI:
10.3109/10903127.2014.916022
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Taylor & Francis Icon for PubMed Central
Loading ...
Support Center