Box 6.1Threats to the Internal Validity of Evaluations

  1. "History, the specific events occurring between the first and second measurement in addition to the experimental measurement."
    Example: During the course of a telepsychiatry project in a poor rural area, a public clinic adds a psychiatric social worker to its staff and thereby makes access to on-site mental health services easier.
  2. "Maturation, processes within the respondents [those being studied] operating as a function of the passage of time per se (not specific to the particular events), including growing older, hungrier, more tired, and the like."
    Example: In a long-term monitoring program for seriously ill, homebound elderly patients, an unrecognized decrease in functional abilities may limit patients' capacity to carry out instructions successfully, potentially compromising evaluators' ability to assess the program and suggest ways it might be redesigned.
  3. “Testing, the effects of taking a test upon the scores of a second testing."
    Example: As primary care physicians participate in a series of teleconsultations for a particular clinical problem, they gain sufficient expertise in diagnosis and management that they no longer seek consultations for the problem.
  4. "Instrumentation, in which changes in the calibration of a measuring instrument or changes in the observers or scorers used may produce changes in the obtained measurements."
    Example: In the midst of a test of digital radiography, a new radiologist, who replaces a more experienced radiologist, takes over the comparison of digitally transmitted images against original films.
  5. "Statistical regression [regression to the mean], operating where groups have been selected on the basis of their extreme scores."
    Example: Of diabetic patients who have been treated for hypoglycemia, those who test lowest on their understanding of appropriate dietary practices are called weekly by nurses or nutritionists.
  6. "Biases resulting in differential selection of respondents for the comparison groups."
    Example: In a telepsychiatry evaluation that involved telemedicine and control sites, the control sites include patients with greater experience with psychiatric intervention.
  7. "Experimental mortality, or differential loss of respondents for the comparison groups."
    Example: In a home care evaluation, sicker patients drop out of the comparison group that was not receiving special services.

SOURCE: Quoted material excerpted from Campbell and Stanley, 1963, pp. 5-6.

From: 6, A Framework for Planning and Improving Evaluations of Telemedicine

Cover of Telemedicine
Telemedicine: A Guide to Assessing Telecommunications in Health Care.
Institute of Medicine (US) Committee on Evaluating Clinical Applications of Telemedicine; Field MJ, editor.
Washington (DC): National Academies Press (US); 1996.
Copyright © 1996, National Academy of Sciences.

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.