Format

Send to

Choose Destination
J Glob Health. 2018 Dec;8(2):020804. doi: 10.7189/jogh.08.020804.

Validation studies for population-based intervention coverage indicators: design, analysis, and interpretation.

Author information

1
Institute for International Programs, Johns Hopkins University Bloomberg School of Public Health, Baltimore, Maryland, USA.
2
Population Council, New York, New York, USA.
3
Center for Applied Malaria Research and Evaluation, Tulane University School of Public Health and Tropical Medicine, New Orleans, Lousiana, USA.
4
Independent consultant, Bend, Oregon, USA.
5
Department of International Health, Johns Hopkins University Bloomberg School of Public Health, Baltimore, Maryland, USA.
6
Department of Disease Control, London School of Hygiene & Tropical Medicine, London, UK.
7
Stanton-Hill Research, LLC, Moultonborough, North Hampshire, USA.
8
Centre for Population Health Sciences, University of Edinburgh, Edinburgh, Scotland, UK.

Abstract

Background:

Population-based intervention coverage indicators are widely used to track country and program progress in improving health and to evaluate health programs. Indicator validation studies that compare survey responses to a "gold standard" measure are useful to understand whether the indicator provides accurate information. The Improving Coverage Measurement (ICM) Core Group has developed and implemented a standard approach to validating coverage indicators measured in household surveys, described in this paper.

Methods:

The general design of these studies includes measurement of true health status and intervention receipt (gold standard), followed by interviews with the individuals observed, and a comparison of the observations (gold standard) to the responses to survey questions. The gold standard should use a data source external to the respondent to document need for and receipt of an intervention. Most frequently, this is accomplished through direct observation of clinical care, and/or use of a study-trained clinician to obtain a gold standard diagnosis. Follow-up interviews with respondents should employ standard survey questions, where they exist, as well as alternative or additional questions that can be compared against the standard household survey questions.

Results:

Indicator validation studies should report on participation at every stage, and provide data on reasons for non-participation. Metrics of individual validity (sensitivity, specificity, area under the receiver operating characteristic curve) and population-level validity (inflation factor) should be reported, as well as the percent of survey responses that are "don't know" or missing. Associations between interviewer and participant characteristics and measures of validity should be assessed and reported.

Conclusions:

These methods allow respondent-reported coverage measures to be validated against more objective measures of need for and receipt of an intervention, and should be considered together with cognitive interviewing, discriminative validity, or reliability testing to inform decisions about which indicators to include in household surveys. Public health researchers should assess the evidence for validity of existing and proposed household survey coverage indicators and consider validation studies to fill evidence gaps.

PMID:
30202519
PMCID:
PMC6126515
DOI:
10.7189/jogh.08.020804
[Indexed for MEDLINE]
Free PMC Article

Conflict of interest statement

Competing interests: Harry Campbell is the Co-Editor in Chief of the Journal of Global Health. To ensure that any possible conflict of interest relevant to the journal has been addressed, this article was reviewed according to best practice guidelines of international editorial organisations. All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf (available upon request from the corresponding author) and declare no conflicts of interest.

Supplemental Content

Full text links

Icon for Edinburgh University Global Health Society Icon for PubMed Central
Loading ...
Support Center