Background: Growth-friendly surgery has high complication rates. The Complication Severity Score for growth-friendly surgery was developed to maintain consistency while reporting complications as part of research in this rapidly evolving field. This study evaluates the interrater and intrarater reliability of this complication classification system.
Methods: After Institutional Review Board approval, complications during treatment for early onset scoliosis were identified from a prospectively collected database. Previous validation studies and a 10-case pilot survey determined that 60 cases were needed to represent a minimum of substantial agreement. In total, 63 of 496 cases were selected randomly to evenly represent each severity classification. The cases comprised an internet survey for classification sent to faculty and research coordinators involved in early onset scoliosis research, 3 weeks apart, with questions shuffled between iterations. Fleiss Kappa and Cohen Kappa were used to assess interrater and intrarater agreement, respectively.
Results: A total of 20 participants, 12 faculty and 8 research assistants, completed the survey twice. The overall Fleiss Kappa coefficient for interrater agreement from the second round of the survey was 0.86 (95% confidence interval, 0.86-87), which represents substantial agreement. Reviewers agreed almost perfectly on categorizing complications as Device I (0.85), Disease I (0.91), Disease II (0.94), Device IIB (0.92), and Disease IV (0.98). There was substantial agreement for categorizing Device IIA (0.73) and Device III (0.76) complications. Disease III and Device IV were not evaluated in this survey since none of these occurred in the database. There was almost perfect intrarater agreement among faculty (0.87), research coordinators (0.85), and overall (0.86).
Conclusions: There is strong interrater and intrarater agreement for the published complications classification scheme for growing spine surgery. The complication classification system is a reliable tool for standardizing reports of complications with growth-friendly surgery. Adoption of this classification when reporting on growth-friendly surgery is recommended to allow for comparison of complications between treatment modalities.
Level of evidence: Level I-diagnostic study.