Table 4, Chapter 2Criteria for assigning strength of evidence for effectiveness/harms questions

What does the evidence show about the effectiveness of this PSP among those at risk?

Individual study risk-of-bias score: Low (+4); Moderate (+3); High (+2); for Cochrane/EPOC Risk of Bias instrument, suggest zero “No” answers = Low risk, one to two “No” answers = Moderate risk, and three or more “No” answers = High risk; suggest taking the median or average as the overall risk of bias for the evidence base.

Across all study types, decrease score if:
  • Important inconsistency across studies (-1)
  • Serious imprecision (-1)
  • High probability of reporting bias (-1)
  • No explanation in any of the studies of why the PSP might work, either in terms of theory, logic models, or prior success in other fields or in pilot studies (-1)
  • PSP not described in sufficient detail to permit replication (-1)
Across all study types, increase score if:
  • Very strong effect in majority of studies (+1)
  • All plausible residual confounding would reduce a demonstrated effect or would suggest a spurious effect if no effect was observed (+1)
  • Use of theory/logic models, assessment of contexts, reporting of implementation process, and fidelity of implementation (+1)
For observational studies, increase score if:
  • Use of observational study designs of stronger internal validity (controlled before-and after, time series, statistical process control) (+1)
If evidence allows a conclusion, then strength of evidence should be scored as follows:
  • ≥+4 = High
  • +3 = Moderate
  • +2 = Low
If evidence does not permit a conclusion then the strength of evidence = insufficient

From: Chapter 2, Methods

Cover of Making Health Care Safer II
Making Health Care Safer II: An Updated Critical Analysis of the Evidence for Patient Safety Practices.
Evidence Reports/Technology Assessments, No. 211.

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.