Format

Send to

Choose Destination
Diagnostics (Basel). 2018 Sep 24;8(4). pii: E69. doi: 10.3390/diagnostics8040069.

Experience Applying the Guidelines for Reporting Reliability and Agreement Studies (GRRAS) Indicated Five Questions Should Be Addressed in the Planning Phase from a Statistical Point of View.

Author information

1
Department of Nuclear Medicine, Odense University Hospital, J.B. Winsløws Vej 4, 5000 Odense C, Denmark. oke.gerke@rsyd.dk.
2
Department of Clinical Research, University of Southern Denmark, Winsløwparken 19, 3. sal, 5000 Odense C, Denmark. moeller@health.sdu.dk.
3
Odense Patient data Explorative Network, Odense University Hospital, J.B. Winsløws Vej 9a, 5000 Odense C, Denmark. moeller@health.sdu.dk.
4
Department of Public Health, Epidemiology and Biostatistics, University of Southern Denmark, J.B. Winsløws Vej 9b, 5000 Odense C, Denmark. bdebrabant@health.sdu.dk.
5
Department of Public Health, Epidemiology and Biostatistics, University of Southern Denmark, J.B. Winsløws Vej 9b, 5000 Odense C, Denmark. uhalekoh@health.sdu.dk.

Abstract

The Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed in 2011 to support transparent and accurate reporting. These studies may be conducted with the primary aim of estimating reliability and/or agreement itself, but are more often than not part of larger diagnostic accuracy studies, clinical trials, or epidemiological studies. As such, the study design may be compromised in terms of practicability issues, preventing the collection of sufficient results. We presented an example from a consultancy with a difficult mission and discussed five questions that concern the very nature of such a study (agreement vs. reliability; intra- vs. interrater), the rater population, explanatory factors in a multivariable model, and the statistical analysis strategy. Discussion of such basic methodological and statistical questions must take place before an investigation is started in order to ensure adequate data collection, to predict possible complications in the study, to plan sufficient statistical analyses, and to request timely assistance from an experienced statistician. GRRAS and its accompanying checklist of 15 items proved to be most helpful. Hopefully, our commentary will help improve the planning of agreement and reliability studies, which, in turn, will then be more focused, more appropriate, and more easily reported using GRRAS.

KEYWORDS:

agreement; guideline; interrater; intrarater; reliability; reproducibility

Supplemental Content

Full text links

Icon for Multidisciplinary Digital Publishing Institute (MDPI) Icon for PubMed Central
Loading ...
Support Center