Send to

Choose Destination
Ann Allergy Asthma Immunol. 2002 Feb;88(2):209-14.

Inter-rater evaluation of a clinical scoring system in children with asthma.

Author information

Wayne State University School of Medicine, Pediatrics, Detroit, Michigan, USA.



Many clinical scores that measure the degree of asthma are used without adequate evaluation of inter-rater reliability. When reliability is tested, most often the Cohen K statistic is used, which limits the comparative results of only two raters at a time.


To evaluate inter-rater agreement of a clinical asthma score using a multi-rater K statistic.


Four raters administered a clinical asthma score to 17 children with clinical asthma. Five items were evaluated: O2 requirement, inspiratory breath sounds, accessory muscle use, expiratory wheeze, and cerebral function. For each, a score of zero indicated a normal state; one, moderate impairment; two, severe impairment. A multi-rater kappa statistic was used as a measure of agreement among all four raters simultaneously. This was applied using hand calculations then cross-checked by using a standard statistical syntax, a component of the Statistical Package for Social Sciences (SPSS 9.0).


Application of the multi-rater K statistic revealed strong agreement among raters on oxygenation (K = 0.759), moderate agreement for expiratory wheeze and cerebral function (K = 0.698), and poor agreement for accessory muscle use (K = 0.528) and inspiratory breath sounds (K = 0.316).


The level of agreement varied by item with the least subjective item, O2 requirement, demonstrating the highest inter-rater correlation. A multi-rater kappa statistic can be applied to data obtained from a clinical scoring instrument either manually or by using statistical syntax provided by SPSS.

[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for Elsevier Science
Loading ...
Support Center