J Nerv Ment Dis. 1976 Nov;163(5):307-17.

On the methods and theory of reliability.

Abstract

This paper reviews the most frequently used and misused reliability measures appearing in the mental health literature. We illustrate the various types of data sets on which reliability is assessed (i.e., two raters, more than two raters, and varying numbers of raters with dichotomous, polychotomous, and quantitative data). Reliability statistics appropriate for each data format are presented, and their pros and cons illustrated. Inadequancies of some methods are highlighted. The meaning of different levels of reliability obtained with various statistics is discussed. This critique is intended for the reading professional and the investigator who has an occasional need for reliability assessment. Statistical expertise is not required and theoretical material is referenced for the interested reader. Necessary formulas for computations are presented in the appendices. A summary table of some suitable reliability measures is presented.

PMID:: 978187; [PubMed - indexed for MEDLINE]

MeSH Terms

LinkOut - more resources

Full Text Sources

Ovid Technologies, Inc.

Medical

Health Statistics - MedlinePlus Health Information

Supplemental Content

Save items

Related citations in PubMed

Review Systematic reviews of the effectiveness of day care for people with severe mental disorders: (1) acute day hospital versus admission; (2) vocational rehabilitation; (3) day hospital versus outpatient care. [Health Technol Assess. 2001]
Review Systematic reviews of the effectiveness of day care for people with severe mental disorders: (1) acute day hospital versus admission; (2) vocational rehabilitation; (3) day hospital versus outpatient care.
Marshall M, Crowther R, Almaraz-Serrano A, Creed F, Sledge W, Kluiter H, Roberts C, Hill E, Wiersma D, Bond GR, et al. Health Technol Assess. 2001; 5(21):1-75.
Review [Diagnostic structured interviews in child and adolescent's psychiatry]. [Encephale. 2004]
Review [Diagnostic structured interviews in child and adolescent's psychiatry].
Renou S, Hergueta T, Flament M, Mouren-Simeoni MC, Lecrubier Y. Encephale. 2004 Mar-Apr; 30(2):122-34.
Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. [Value Health. 2009]
Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report.
Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA, Lenderking WR, Cella D, Basch E, ISPOR ePRO Task Force. Value Health. 2009 Jun; 12(4):419-29. Epub 2008 Nov 11.
Learning how to differ: agreement and reliability statistics in psychiatry. [Can J Psychiatry. 1995]
Learning how to differ: agreement and reliability statistics in psychiatry.
Streiner DL. Can J Psychiatry. 1995 Mar; 40(2):60-6.
Clinical trials in psychosocial medicine: methodologic and statistical considerations. Part II. Assessing reliability with equal-appearing interval scales. [Cancer Treat Rep. 1980]
Clinical trials in psychosocial medicine: methodologic and statistical considerations. Part II. Assessing reliability with equal-appearing interval scales.
Feldstein ML. Cancer Treat Rep. 1980 Feb-Mar; 64(2-3):445-50.

See reviews... See all...

Cited by 17 PubMed Central articles

The genetic and environmental determinants of the association between brain abnormalities and schizophrenia: the schizophrenia twins and relatives consortium. [Biol Psychiatry. 2012]
The genetic and environmental determinants of the association between brain abnormalities and schizophrenia: the schizophrenia twins and relatives consortium.
van Haren NE, Rijsdijk F, Schnack HG, Picchioni MM, Toulopoulou T, Weisbrod M, Sauer H, van Erp TG, Cannon TD, Huttunen MO, et al. Biol Psychiatry. 2012 May 15; 71(10):915-21. Epub 2012 Feb 16.
Iranian version of a face version of the Modified Child Dental Anxiety Scale: Transcultural adaptation and reliability analysis. [J Res Med Sci. 2011]
Iranian version of a face version of the Modified Child Dental Anxiety Scale: Transcultural adaptation and reliability analysis.
Javadinejad S, Farajzadegan Z, Madahain M. J Res Med Sci. 2011 Jul; 16(7):872-7.
The stability of symptoms and syndromes in chronic schizophrenic patients. [Indian J Psychiatry. 1992]
The stability of symptoms and syndromes in chronic schizophrenic patients.
Borde M, Davis EJ, Sharma LN. Indian J Psychiatry. 1992 Apr; 34(2):133-9.

See all...

Related information

Related Citations
Calculated set of PubMed citations closely related to the selected article(s) retrieved using a word weight algorithm. Related articles are displayed in ranked order from most to least relevant, with the “linked from” citation displayed first.
Cited in PMC
Full-text articles in the PubMed Central Database that cite the current articles.

Recent activity

Clear Turn Off Turn On

On the methods and theory of reliability.
On the methods and theory of reliability.
J Nerv Ment Dis. 1976 Nov ;163(5):307-17.

PubMed
The results of CAPRIE, IST and CAST. Clopidogrel vs. Aspirin in Patients at Risk...
The results of CAPRIE, IST and CAST. Clopidogrel vs. Aspirin in Patients at Risk of Ischaemic Events. International Stroke Trial. Chinese Acute Stroke Trial.
Thromb Res. 1998 Sep 15 ;92(1 Suppl 1):S13-6.

PubMed
Pulmonary Artery Catheter Consensus Conference.
Pulmonary Artery Catheter Consensus Conference.
Crit Care Med. 1998 Oct ;26(10):1760-1.

PubMed
Muscle satellite cells in urodele amphibians: faciliatated identification of sat...
Muscle satellite cells in urodele amphibians: faciliatated identification of satellite cells using ruthenium red staining.
J Exp Zool. 1976 Oct ;198(1):57-64.

PubMed
Endocrine and metabolic aberrations in men with abdominal obesity in relation to...
Endocrine and metabolic aberrations in men with abdominal obesity in relation to anxio-depressive infirmity.
Metabolism. 1998 Oct ;47(10):1187-93.

PubMed

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

PubMed

Result Filters

Display Settings:

Send to:

On the methods and theory of reliability.

Abstract

MeSH Terms

MeSH Terms

LinkOut - more resources

Full Text Sources

Medical

Supplemental Content

Save items

Related citations in PubMed

Cited by 17 PubMed Central articles

Related information

Recent activity

Simple NCBI Directory

Getting Started

Resources

Popular

Featured

NCBI Information