U.S. Preventive Services Task Force Quality Rating Criteria for Randomized Controlled Trials and Observational Studies

Roger Chou; Tracy Dana; Christina Bougatsos

Appendix A5U.S. Preventive Services Task Force Quality Rating Criteria for Randomized Controlled Trials and Observational Studies

Publication Details

Diagnostic Accuracy Studies

Criteria

Screening test relevant, available for primary care, adequately described
Study uses a credible reference standard, performed regardless of test results
Reference standard interpreted independently of screening test
Handles indeterminate results in a reasonable manner
Spectrum of patients included in study
Sample size
Administration of reliable screening test
Random or consecutive selection of patients⁴⁴
Screening cutoff predetermined⁴⁴
All patients undergo the reference standard⁴⁴

Definition of ratings based on above criteria

Randomized Controlled Trials (RCTs) and Cohort Studies

Criteria

Initial assembly of comparable groups: RCTs—adequate randomization, including concealment and whether potential confounders were distributed equally among groups; cohort studies—consideration of potential confounders with either restriction or measurement for adjustment in the analysis; consideration of inception cohorts
Maintenance of comparable groups (includes attrition, cross-over, adherence, contamination)
Important differential loss to follow-up or overall high loss to follow-up
Measurements: equal, reliable, and valid (includes masking of outcome assessment)
Clear definition of interventions
Important outcomes considered
Analysis: adjustment for potential confounders for cohort studies, or intention-to-treat analysis for RCTs; for cluster RCTs, correction for correlation coefficient

Definition of ratings based on above criteria

Case Control Studies

Criteria

Accurate ascertainment of cases
Nonbiased selection of cases/controls with exclusion criteria applied equally to both
Response rate
Diagnostic testing procedures applied equally to each group
Measurement of exposure accurate and applied equally to each group
Appropriate attention to potential confounding variable

Definition of ratings based on criteria above

Publication Details

Copyright

Publisher

Agency for Healthcare Research and Quality (US), Rockville (MD)

NLM Citation

Chou R, Dana T, Bougatsos C. Screening for Visual Impairment in Children Ages 1-5 Years: Systematic Review to Update the 2004 U.S. Preventive Services Task Force Recommendation [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2011 Feb. (Evidence Syntheses, No. 81.) Appendix A5, U.S. Preventive Services Task Force Quality Rating Criteria for Randomized Controlled Trials and Observational Studies.

Good:	Evaluates relevant available screening test; uses a credible reference standard; interprets reference standard independently of screening test; assesses reliability of test; has few or handles indeterminate results in a reasonable manner; includes a large number (>100) of broad-spectrum patients with and without disease; study attempts to enroll a random or consecutive sample of patients who meet inclusion criteria⁴⁴; screening cutoffs are prestated.⁴⁴
Fair:	Evaluates relevant available screening test; uses reasonable although not best standard; interprets reference standard independent of screening test; includes a moderate sample size (50 to 100 subjects) and a “medium” spectrum of patients (i.e., applicable to most screening settings).
Poor:	Has important limitations, such as: uses inappropriate reference standard; screening test improperly administered; biased ascertainment of reference standard; very small sample size of very narrowly selected spectrum of patients.

Good:	Meets all criteria: comparable groups are assembled initially and maintained throughout the study (follow-up at least 80%); reliable and valid measurement instruments are used and applied equally to groups; interventions are spelled out clearly; important outcomes are considered; and appropriate attention to confounders in analysis.
Fair:	Studies will be graded “fair” if any or all of the following problems occur, without the important limitations noted in the “poor” category below: generally comparable groups are assembled initially but some question remains whether some (although not major) differences occurred in follow-up; measurement instruments are acceptable (although not the best) and generally applied equally; some but not all important outcomes are considered; and some but not all potential confounders are accounted for.
Poor:	Studies will be graded “poor” if any of the following major limitations exists: groups assembled initially are not close to being comparable or maintained throughout the study; unreliable or invalid measurement instruments are used or not applied at all equally among groups (including not masking outcome assessment); and key confounders are given little or no attention.

Good:	Appropriate ascertainment of cases and nonbiased selection of case and control participants; exclusion criteria applied equally to cases and controls; response rate equal to or greater than 80%; diagnostic procedures and measurements accurate and applied equally to cases and controls; and appropriate attention to confounding variables.
Fair:	Recent, relevant, without major apparent selection or diagnostic work-up bias but with response rate less than 80% or attention to some but not all important confounding variables.
Poor:	Major selection or diagnostic work-up biases, response rates less than 50%, or inattention to confounding variables.

Appendix A5U.S. Preventive Services Task Force Quality Rating Criteria for Randomized Controlled Trials and Observational Studies

Diagnostic Accuracy Studies

Criteria

Definition of ratings based on above criteria

Table

Randomized Controlled Trials (RCTs) and Cohort Studies

Criteria

Definition of ratings based on above criteria

Table

Case Control Studies

Criteria

Definition of ratings based on criteria above

Table

Publication Details

Copyright

Publisher

NLM Citation