NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Whitlock EP, O'Connor EA, Williams SB, et al. Effectiveness of Weight Management Programs in Children and Adolescents. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008 Sep. (Evidence Reports/Technology Assessments, No. 170.)

  • This publication is provided for historical reference only and the information may be out of date.

This publication is provided for historical reference only and the information may be out of date.

Cover of Effectiveness of Weight Management Programs in Children and Adolescents

Effectiveness of Weight Management Programs in Children and Adolescents.

Show details

Appendix H. Quality rating criteria

DesignUnited States Preventive Services Task Force quality rating criteria1National Institute for Health and Clinical Excellence methodology checklists2
Systematic reviews and meta-analyses
  • Comprehensiveness of sources considered/search strategy used
  • Standard appraisal of included studies
  • Validity of conclusions
  • Recency and relevance are especially important for systematic reviews
  • The study addresses an appropriate and clearly focused question
  • A description of the methodology used is included
  • The literature search is sufficiently rigorous to identify all the relevant studies
  • Study quality is assessed and taken into account
  • There are enough similarities between the studies selected to make combining them reasonable
Case-control studies
  • Accurate ascertainment of cases
  • Nonbiased selection of cases/controls with exclusion criteria applied equally to both
  • Response rate
  • Diagnostic testing procedures applied equally to each group
  • Measurement of exposure accurate and applied equally to each group
  • Appropriate attention to potential confounding variables
  • The study addresses an appropriate and clearly focused question
  • The cases and controls are taken from comparable populations
  • The same exclusion criteria are used for both cases and controls
  • What percentage of each group (cases and controls) participated in the study?
  • Comparison is made between participants and non-participants to establish their similarities or differences
  • Cases are clearly defined and differentiated from controls
  • Is it clearly established that controls are non-cases?
  • Measures have been taken to prevent knowledge of primary exposure influencing case ascertainment
  • Exposure status is measured in a standard, valid and reliable way
  • The main potential confounders are identified and taken into account in the design and analysis
  • Have confidence intervals been provided?
Randomized controlled trials (RCTs)
  • Initial assembly of comparable groups employs adequate randomization, including first concealment and whether potential confounders were distributed equally among groups.
  • Maintenance of comparable groups (includes attrition, crossovers, adherence, contamination)
  • Important differential loss to follow-up or overall high loss to follow-up
  • Measurements: equal, reliable, and valid (includes masking of outcome assessment)
  • Clear definition of the interventions
  • All important outcomes considered
  • The study addresses an appropriate and clearly focused question
  • The assignment of subjects to treatment groups is randomized
  • An adequate concealment method is used
  • Subjects and investigators are kept ‘blind’ about treatment allocation
  • The treatment and control groups are similar at the start of the trial
  • The only difference between groups is the treatment under investigation
  • All relevant outcomes are measured in a standard, valid and reliable way
  • What percentage of the individuals or clusters recruited into each treatment arm of the study dropped out before the study was completed?
  • All the subjects are analyzed in the groups to which they were randomly allocated (often referred to as intention-to-treat analysis)
  • Where the study is carried out at more than one site, results are comparable for all sites
Cohort studies
  • Initial assembly of comparable groups employs consideration of potential confounders with either restriction or measurement for adjustment in the analysis; consideration of inception cohorts
  • Maintenance of comparable groups (includes attrition, crossovers, adherence, contamination)
  • Important differential loss to follow-up or overall high loss to follow-up
  • Measurements: equal, reliable, and valid (includes masking of outcome assessment)
  • Clear definition of the interventions
  • All important outcomes considered
  • The study addresses an appropriate and clearly focused question
  • The two groups being studied are selected from source populations that are comparable in all respects other than the factor under investigation
  • The study indicates how many of the people asked to take part did so, in each of the groups being studied
  • The likelihood that some eligible subjects might have the outcome at the time of enrollment is assessed and taken into account in the analysis
  • What percentage of individuals or clusters recruited into each arm of the study dropped out before the study was completed?
  • Comparison is made between full participants and those lost to follow-up, by exposure status
  • The outcomes are clearly defined
  • The assessment of outcome is made blind to exposure status
  • Where blinding was not possible, there is some recognition that knowledge of exposure status could have influenced the assessment of outcome
  • The measure of assessment of exposure is reliable
  • Evidence from other sources is used to demonstrate that the method of outcome assessment is valid and reliable
  • Exposure level or prognostic factor is assessed more than once
  • The main potential confounders are identified and taken into account in the design and analysis
  • Have confidence intervals been provided?
Diagnostic accuracy studies
  • Screening test relevant, available for primary care, adequately described
  • Study uses a credible reference standard, performed regardless of test results
  • Reference standard interpreted independently of screening test
  • Handles indeterminate result in a reasonable manner
  • Spectrum of patients included in study
  • Sample size
  • Administration of reliable screening test
  • The nature of the test being studied is clearly specified
  • The test is compared with an appropriate gold standard
  • Where no gold standard exists, a validated reference standard is used as a comparator
  • Patients for testing are selected either as a consecutive series or randomly, from a clearly defined study population
  • The test and gold standard are measured independently (blind) of each other
  • The test and gold standard are applied as close together in time as possible
  • Results are reported for all patients that are entered into the study
  • A pre-diagnosis is made and reported

Hierarchy of research design1

  • I Properly conducted randomized controlled trial (RCT)
  • II-1: Well-designed controlled trial without randomization
  • II-2: Well-designed cohort or case-control analytic study
  • II-3: Multiple time series with or without the intervention; dramatic results from uncontrolled experiments
  • III: Opinions of respected authorities, based on clinical experience; descriptive studies or case reports; reports of expert committees

References for Appendix H

1.
Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM. et al. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med. 2001;20(3 Suppl):21–35. [PubMed: 11306229]
2.
National Institute for Health and Clinical Excellence. (April 2006). ‘The guidelines manual’. London: National Institute for Health and Clinical Excellence. Available from: www​.nice.org.uk.

Views

  • PubReader
  • Print View
  • Cite this Page

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...