Send to

Choose Destination
J Emerg Nurs. 2007 Aug;33(4):319-23. Epub 2007 Jun 21.

Triage tool inter-rater reliability: a comparison of live versus paper case scenarios.

Author information

Division of Emergency Medicine, McMaster University and Hamilton Health Sciences, Department of Emergency Medicine, Hamilton, Ontario, Canada.



Published studies of triage scale inter-rater reliability assessment have been conducted mostly using paper case scenarios.


To determine if this method of inter-rater reliability assessment generated significantly different measures from those generated from live triage cases.


This was a multicenter, prospective, observational cohort study of a population-based random sample of patients triaged at 2 emergency departments during a period of 4 months. All patients presenting to the emergency department within the study periods were simultaneously and independently triaged using a 5-level triage acuity scale by 2 to 3 research triage nurses blinded to each other's assessment and to the study objective. After 6 months, the same nurses were asked to assign triage scores to paper case scenarios of the same patients that they had each previously triaged.


Each of the 9 research nurses triaged approximately 90 cases. The inter-rater reliabilities as measured by an intraclass correlation coefficient were 0.9 (95% CI = 0.87 to 93) for the live triage assessments and 0.76 (95% CI = 0.73 to 0.79) for the paper case scenarios. The mean triage score assigned to the live cases was significantly less than that assigned to the paper-based cases (3.17; 95% CI = 3.08 to 3.26) (p < 0.001).


There is moderate to high agreement between live cases and paper case scenarios, and the inter-rater reliabilities, although significantly different, are acceptable in both cases. It is impossible to determine which triage setting provides a more accurate triage score but paper case scenarios generally receive lower triage scores than live cases.

[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for Elsevier Science
Loading ...
Support Center