Appendix HThresholds for Quality Assessment

Publication Details

Quality assessment thresholds for Cochrane Risk of Bias (RoB) Tool: There are three categories for describing the quality of studies: “Good”, “Fair”, and “Poor”. In order to assign a study to a category, we need to establish the threshold between good and fair quality studies and between fair and poor quality studies. Cochrane Collaboration uses strict criteria for the quality ratings.

Cochrane Collaboration criteria for quality ratings:

  • A good quality study must meet all criteria (Low RoB).
  • A fair quality study does not meet, or it is not clear that it meets, at least one criterion, but it has no known important limitation that could invalidate its results (Moderate RoB).
  • A poor quality study has important limitations and/or at least one criterion is not met (High RoB).

Modifications of criteria for quality ratings:

  • If all criteria are rated as “low” = Low RoB = Good Quality
  • If one criterion is rated as “high” or 1-2 criteria are “unclear”, and the assessment is that this was unlikely to have biased the outcome, and there is no known important limitation that could invalidate the results = Low RoB = Good Quality
    • Example: not blinded, but blinding was not possible and based upon design and outcomes, it is unlikely that the lack of blinding could have affected the outcome measure or other factor that would introduce bias
  • If one criterion is rated as “high” or 1-2 criteria are “unclear”, and the assessment is that this was likely to have biased the outcome, and there is no known important limitation that could invalidate the results = Moderate RoB = Fair Quality
    • Could be poor, if the factors were considered to combine to important limitations
  • If one criterion is rated as “high” and 3 are “unclear = Moderate RoB = Fair Quality
    • Could be poor, if the factors were considered to combine to important limitations
  • If two criteria are rated as “high”, and all other criteria are ‘low’, and the assessment that this was unlikely to have biased the outcome, and there are no known important limitations that could invalidate the results = Moderate RoB = Fair Quality
  • If two criteria are rated as “high”, and all other criteria are ‘low’, and the assessment that this was likely to have biased the outcome, and there are important limitations that could invalidate the results = High RoB =Poor Quality
  • If three or more criteria are rated as “high” = High RoB = Poor Quality
  • If four or more criteria are rated as “unclear”= High RoB = Poor Quality