Format

Send to

Choose Destination
Aust Occup Ther J. 2015 Dec;62(6):420-7. doi: 10.1111/1440-1630.12218. Epub 2015 Aug 19.

The development, content validity and inter-rater reliability of the SMART-Goal Evaluation Method: A standardised method for evaluating clinical goals.

Author information

1
Research and Evaluation Service, Justice Health and Forensic Mental Health Network, Malabar, New South Wales, Australia.
2
Occupational Therapy Department, Faculty of Health Sciences, La Trobe University, Melbourne, Victoria, Australia.
3
School of Medicine, University of Western Sydney, Campbelltown, New South Wales, Australia.
4
School of Science and Health, University of Western Sydney, Penrith South, New South Wales, Australia.
5
Occupational Therapy Department, Alfred Health, Melbourne, Australia.
6
John Walsh Centre of Rehabilitation Research, Northern Clinical School, Sydney Medical School, University of Sydney, St Leonards, New South Wales, Australia.

Abstract

INTRODUCTION:

Goal setting is a complex skill. The use of formal goal writing procedures (including the use of the SMART goal model) has been advocated. However, a standardised method of writing and evaluating SMART goals is currently lacking. This study comprised of two phases. The aims of phase one was to (i) develop the SMART Goal Evaluation Method (SMART-GEM) based on a SMART goal model; and (ii) investigate the content validity of the SMART-GEM. The aim of phase two of the study was to test the inter-rater reliability of the SMART-GEM.

METHODS:

Development of the SMART- GEM involved defining and constructing evaluation criteria suitable for auditing goal statements. A content validity assessment was conducted using an expert panel of Occupational Therapists (n = 10). Inter-rater reliability of the SMART-GEM was examined using a purposive sample of multiple raters (n = 24).

RESULTS:

The SMART- GEM was rated as having good content validity (individual items CVI ranged from 0.90 to 1.00; total SMART- GEM CVI = 0.99, ρ = 0.05). Agreement between raters on individual items ranged from poor (κ = 0.254) to excellent (κ = 0.965) and agreement of overall grades was fair to good (κ = 0.582). Inter-rater agreement on total scores was found to be very good (ICC = 0.895, 95% CI = 0.743 to 0.986, ρ = 0.001) with excellent internal consistency (α = 0.995).

CONCLUSION:

The SMART-GEM demonstrated good construct validity and very good inter-rater reliability on total score and shows promise as a standardised method to writing and evaluating clinical goals.

KEYWORDS:

goal; occupational therapy; outcome assessment (health care); rehabilitation

PMID:
26286379
DOI:
10.1111/1440-1630.12218
[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for Wiley
Loading ...
Support Center