• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of jepicomhInstructions for authorsCurrent TOCJournal of Epidemiology and Community Health
J Epidemiol Community Health. Feb 2002; 56(2): 119–127.
PMCID: PMC1732065

Criteria for evaluating evidence on public health interventions

Abstract

Public health interventions tend to be complex, programmatic, and context dependent. The evidence for their effectiveness must be sufficiently comprehensive to encompass that complexity. This paper asks whether and to what extent evaluative research on public health interventions can be adequately appraised by applying well established criteria for judging the quality of evidence in clinical practice. It is adduced that these criteria are useful in evaluating some aspects of evidence. However, there are other important aspects of evidence on public health interventions that are not covered by the established criteria. The evaluation of evidence must distinguish between the fidelity of the evaluation process in detecting the success or failure of an intervention, and the success or failure of the intervention itself. Moreover, if an intervention is unsuccessful, the evidence should help to determine whether the intervention was inherently faulty (that is, failure of intervention concept or theory), or just badly delivered (failure of implementation). Furthermore, proper interpretation of the evidence depends upon the availability of descriptive information on the intervention and its context, so that the transferability of the evidence can be determined. Study design alone is an inadequate marker of evidence quality in public health intervention evaluation.

Full Text

The Full Text of this article is available as a PDF (147K).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.
  • Oxman AD, Sackett DL, Guyatt GH. Users' guides to the medical literature. I. How to get started. The Evidence-Based Medicine Working Group. JAMA. 1993 Nov 3;270(17):2093–2095. [PubMed]
  • Woolf SH, Battista RN, Anderson GM, Logan AG, Wang E. Assessing the clinical effectiveness of preventive maneuvers: analytic principles and systematic methods in reviewing evidence and developing clinical practice recommendations. A report by the Canadian Task Force on the Periodic Health Examination. J Clin Epidemiol. 1990;43(9):891–905. [PubMed]
  • Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1993 Dec 1;270(21):2598–2601. [PubMed]
  • Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. JAMA. 1994 Jan 5;271(1):59–63. [PubMed]
  • Briss PA, Zaza S, Pappaioanou M, Fielding J, Wright-De Agüero L, Truman BI, Hopkins DP, Mullen PD, Thompson RS, Woolf SH, et al. Developing an evidence-based Guide to Community Preventive Services--methods. The Task Force on Community Preventive Services. Am J Prev Med. 2000 Jan;18(1 Suppl):35–43. [PubMed]
  • Mosteller F, Colditz GA. Understanding research synthesis (meta-analysis). Annu Rev Public Health. 1996;17:1–23. [PubMed]
  • Glasziou P, Longbottom H. Evidence-based public health practice. Aust N Z J Public Health. 1999 Aug;23(4):436–440. [PubMed]
  • Carande-Kulis VG, Maciosek MV, Briss PA, Teutsch SM, Zaza S, Truman BI, Messonnier ML, Pappaioanou M, Harris JR, Fielding J. Methods for systematic reviews of economic evaluations for the Guide to Community Preventive Services. Task Force on Community Preventive Services. Am J Prev Med. 2000 Jan;18(1 Suppl):75–91. [PubMed]
  • McLeroy KR, Bibeau D, Steckler A, Glanz K. An ecological perspective on health promotion programs. Health Educ Q. 1988 Winter;15(4):351–377. [PubMed]
  • Ziglio E. How to move towards evidence-based health promotion interventions. Promot Educ. 1997 Jun;4(2):29–33. [PubMed]
  • Oakley A. Experimentation and social interventions: a forgotten but important history. BMJ. 1998 Oct 31;317(7167):1239–1242. [PMC free article] [PubMed]
  • Smith PJ, Moffatt ME, Gelskey SC, Hudson S, Kaita K. Are community health interventions evaluated appropriately? A review of six journals. J Clin Epidemiol. 1997 Feb;50(2):137–146. [PubMed]
  • Ebrahim S, Smith GD. Effects of the Heartbeat Wales programme. Effects of government policies on health behaviour must be studied. BMJ. 1998 Sep 26;317(7162):886–886. [PMC free article] [PubMed]
  • Harvey K. Effects of the Heartbeat Wales programme. Health promotion is a waste of time and money. BMJ. 1998 Sep 26;317(7162):887–887. [PubMed]
  • Shefer A, Briss P, Rodewald L, Bernier R, Strikas R, Yusuf H, Ndiaye S, Wiliams S, Pappaioanou M, Hinman AR. Improving immunization coverage rates: an evidence-based review of the literature. Epidemiol Rev. 1999;21(1):96–142. [PubMed]
  • Susser M. The tribulations of trials--intervention in communities. Am J Public Health. 1995 Feb;85(2):156–158. [PMC free article] [PubMed]
  • Winkleby MA. The future of community-based cardiovascular disease intervention studies. Am J Public Health. 1994 Sep;84(9):1369–1372. [PMC free article] [PubMed]
  • Tudor-Smith C, Nutbeam D, Moore L, Catford J. Effects of the Heartbeat Wales programme over five years on behavioural risks for cardiovascular disease: quasi-experimental comparison of results from Wales and a matched reference area. BMJ. 1998 Mar 14;316(7134):818–822. [PMC free article] [PubMed]
  • Sorensen G, Emmons K, Hunt MK, Johnston D. Implications of the results of community intervention trials. Annu Rev Public Health. 1998;19:379–416. [PubMed]
  • Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000 Sep 16;321(7262):694–696. [PMC free article] [PubMed]
  • Flay BR. Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. Prev Med. 1986 Sep;15(5):451–474. [PubMed]
  • Furberg CD. Challenges to the funding of prevention research. Prev Med. 1994 Sep;23(5):599–601. [PubMed]
  • Krieger N, Zierler S. What explains the public's health?--A call for epidemiologic theory. Epidemiology. 1996 Jan;7(1):107–109. [PubMed]
  • Krieger N, Zierler S. The need for epidemiologic theory. Epidemiology. 1997 Mar;8(2):212–214. [PubMed]
  • Diez-Roux AV. Bringing context back into epidemiology: variables and fallacies in multilevel analysis. Am J Public Health. 1998 Feb;88(2):216–222. [PMC free article] [PubMed]
  • Susser M, Susser E. Choosing a future for epidemiology: I. Eras and paradigms. Am J Public Health. 1996 May;86(5):668–673. [PMC free article] [PubMed]
  • Susser M, Susser E. Choosing a future for epidemiology: II. From black box to Chinese boxes and eco-epidemiology. Am J Public Health. 1996 May;86(5):674–677. [PMC free article] [PubMed]
  • Gyorkos TW, Tannenbaum TN, Abrahamowicz M, Oxman AD, Scott EA, Millson ME, Rasooly I, Frank JW, Riben PD, Mathias RG, et al. An approach to the development of practice guidelines for community health interventions. Can J Public Health. 1994 Jul-Aug;85 (Suppl 1):S8–13. [PubMed]
  • Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ. 1996 May 11;312(7040):1215–1218. [PMC free article] [PubMed]
  • Mackenbach JP. Public health epidemiology. J Epidemiol Community Health. 1995 Aug;49(4):333–334. [PMC free article] [PubMed]
  • Ioannidis JP, Haidich AB, Lau J. Any casualties in the clash of randomised and observational evidence? BMJ. 2001 Apr 14;322(7291):879–880. [PMC free article] [PubMed]
  • Lomas J. Research and evidence-based decision making. Aust N Z J Public Health. 1997 Aug;21(5):439–441. [PubMed]
  • Kerridge I, Lowe M, Henry D. Ethics and evidence based medicine. BMJ. 1998 Apr 11;316(7138):1151–1153. [PMC free article] [PubMed]
  • Little M. Assignments of meaning in epidemiology. Soc Sci Med. 1998 Nov;47(9):1135–1145. [PubMed]
  • Maynard A. Evidence-based medicine: an incomplete method for informing treatment choices. Lancet. 1997 Jan 11;349(9045):126–128. [PubMed]
  • Russell LB, Gold MR, Siegel JE, Daniels N, Weinstein MC. The role of cost-effectiveness analysis in health and medicine. Panel on Cost-Effectiveness in Health and Medicine. JAMA. 1996 Oct 9;276(14):1172–1177. [PubMed]
  • Weinstein MC, Siegel JE, Gold MR, Kamlet MS, Russell LB. Recommendations of the Panel on Cost-effectiveness in Health and Medicine. JAMA. 1996 Oct 16;276(15):1253–1258. [PubMed]
  • Drummond MF, Richardson WS, O'Brien BJ, Levine M, Heyland D. Users' guides to the medical literature. XIII. How to use an article on economic analysis of clinical practice. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1997 May 21;277(19):1552–1557. [PubMed]
  • O'Brien BJ, Heyland D, Richardson WS, Levine M, Drummond MF. Users' guides to the medical literature. XIII. How to use an article on economic analysis of clinical practice. B. What are the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. JAMA. 1997 Jun 11;277(22):1802–1806. [PubMed]
  • Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation Working Party. BMJ. 1996 Aug 3;313(7052):275–283. [PMC free article] [PubMed]
  • Mason J, Drummond M. Reporting guidelines for economic studies. Health Econ. 1995 Mar-Apr;4(2):85–94. [PubMed]
  • Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998 Mar;13(1):87–108. [PubMed]
  • Irwig L, Zwarenstein M, Zwi A, Chalmers I. A flow diagram to facilitate selection of interventions and research for health care. Bull World Health Organ. 1998;76(1):17–24. [PMC free article] [PubMed]
  • Feinstein AR, Horwitz RI. Problems in the "evidence" of "evidence-based medicine". Am J Med. 1997 Dec;103(6):529–535. [PubMed]
  • Birch S. As a matter of fact: evidence-based decision-making unplugged. Health Econ. 1997 Nov-Dec;6(6):547–559. [PubMed]
  • Watt D, Verma S, Flynn L. Wellness programs: a review of the evidence. CMAJ. 1998 Jan 27;158(2):224–230. [PMC free article] [PubMed]
  • Popay J, Williams G. Qualitative research and evidence-based healthcare. J R Soc Med. 1998;91 (Suppl 35):32–37. [PMC free article] [PubMed]
  • Mays N, Pope C. Rigour and qualitative research. BMJ. 1995 Jul 8;311(6997):109–112. [PMC free article] [PubMed]
  • Popay J, Rogers A, Williams G. Rationale and standards for the systematic review of qualitative literature in health services research. Qual Health Res. 1998 May;8(3):341–351. [PubMed]
  • Greenhalgh T, Taylor R. Papers that go beyond numbers (qualitative research) BMJ. 1997 Sep 20;315(7110):740–743. [PMC free article] [PubMed]
  • Mays N, Pope C. Qualitative research in health care. Assessing quality in qualitative research. BMJ. 2000 Jan 1;320(7226):50–52. [PMC free article] [PubMed]
  • Giacomini MK, Cook DJ. Users' guides to the medical literature: XXIII. Qualitative research in health care A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 2000 Jul 19;284(3):357–362. [PubMed]
  • Sauerborn R, Nitayarumphong S, Gerhardus A. Strategies to enhance the use of health systems research for health sector reform. Trop Med Int Health. 1999 Dec;4(12):827–835. [PubMed]
  • Blakely T, Thornley CN. Screening for hepatitis B carriers: evidence and policy development in New Zealand. N Z Med J. 1999 Nov 12;112(1099):431–433. [PubMed]
  • Murray CJ, Lopez AD. Evidence-based health policy--lessons from the Global Burden of Disease Study. Science. 1996 Nov 1;274(5288):740–743. [PubMed]

Articles from Journal of Epidemiology and Community Health are provided here courtesy of BMJ Group

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • Cited in Books
    Cited in Books
    PubMed Central articles cited in books
  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...