Display Settings:

Format

Send to:

Choose Destination

    J Speech Lang Hear Res. 2006 Oct;49(5):1058-71.

    Reliability issues and solutions for coding social communication performance in classroom settings.

    Olswang LB, Svensson L, Coggins TE, Beilinson JS, Donaldson AL.

    Department of Speech and Hearing Sciences, University of Washington, 1417 Northeast 42nd Street, Seattle, WA 98105-6246l, USA. lolswang@u.washington.edu

    PURPOSE: To explore the utility of time-interval analysis for documenting the reliability of coding social communication performance of children in classroom settings. Of particular interest was finding a method for determining whether independent observers could reliably judge both occurrence and duration of ongoing behavioral dimensions for describing social communication performance. METHOD: Four coders participated in this study. They observed and independently coded 6 social communication behavioral dimensions using handheld computers. The dimensions were mutually exclusive and accounted for all verbal and nonverbal productions during a specified time frame. The technology allowed for coding frequency and duration for each entered code. Data were collected from 20 different 2-min video segments of children in kindergarten through 3rd-grade classrooms. Data were analyzed for interobserver and intraobserver agreements using time-interval sorting and Cohen's kappa. Further, interval size and total observation length were manipulated to determine their influence on reliability. RESULTS: The data revealed interval sorting and kappa to be a suitable method for examining reliability of occurrence and duration of ongoing social communication behavioral dimensions. Nearly all comparisons yielded medium to large kappa values; interval size and length of observation minimally affected results. Implications The analysis procedure described in this research solves a challenge in reliability: comparing coding by independent observers of both occurrence and duration of behaviors. Results indicate the utility of a new coding taxonomy and technology for application in online observations of social communication in a classroom setting.

    PMID: 17077214 [PubMed - indexed for MEDLINE]

    Supplemental Content

    Click here to read