Table 1Inclusion and exclusion of constructs for risk-of-bias assessment, applicability, and strength of evidence

ConstructIncluded in appraisal of individual study risk of bias?Included in assessing applicability of studies and the body of evidence?Included in grading strength of the body of evidence?
Risk of bias (from selection bias and confounding, attrition, performance, detection, reporting, and other biases)YesNoYes (required domain of risk of bias)
PrecisionOnly when no quantitative pooling or presentation is possibleNoYes (required domain of precision)
Applicability/external validityOnly when components of applicability influence risk of bias (e.g., duration of follow-up varies across intervention arms)YesDepends on the SOE system. GRADE includes applicability as part of directness, AHRQ-EPC does not (with the exception of rating surrogate outcomes as indirect evidence)
Poor or inadequate reportingYes, studies may be rated as having unclear risk of biasNoNo
Selective outcome reportingYes, only when judgments can be made about the impact of differences between outcomes listed in full protocol and published materialsNoYes
Outcome measuresYes (potential for outcome measurement bias, specifically validity, reliability, variation across study arms)Yes (applicability of outcomes measures)Yes (directness of outcome measures)
Study designAssessment should evaluate the relevant sources of risk of bias by study design rather than rate the study risk of bias by design labels aloneNoYes (overall risk of bias is rated separately for randomized and nonrandomized studies)
Fidelity to protocolYesYesNo
Conflict of interest from sponsor biasIndirectly (sponsor bias may influence one or more sources of bias)Indirectly (sponsor bias may limit applicability)Indirectly (sponsor bias may influence domains of risk of bias, directness, and publication bias)

Abbreviations: GRADE=Grading of Recommendations Assessment, Development and Evaluation; SOE=strength of evidence.

From: Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions

Cover of Methods Guide for Effectiveness and Comparative Effectiveness Reviews
Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet].

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.