Send to

Choose Destination
JAMA. 2002 Jun 12;287(22):2973-82.

Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials.

Author information

Evidence-based Practice Center, Division of Clinical Care Research, Tufts University School of Medicine, New England Medical Center, 750 Washington St, No. 63, Boston, MA 02111, USA.



Specific features of trial quality may be associated with exaggeration or shrinking of the observed treatment effect in randomized studies. Therefore, assessment of trial quality is often used in meta-analysis. However, the degree to which specific quality measures are associated with treatment effects has not been well established across a broad range of clinical areas.


To determine if quality measures are associated with treatment effect size in randomized controlled trials (RCTs).


Quality measures from published quality assessment scales were evaluated in RCTs included in meta-analyses from 4 medical areas (cardiovascular disease, infectious disease, pediatrics, and surgery). Included meta-analyses incorporated at least 6 RCTs, examined dichotomous outcomes, and demonstrated significant between-study heterogeneity in the odds ratio (OR) scale.


Relative ORs comparing overall treatment effect (summary OR) of high vs low-quality studies, as determined by each quality measure, with relative ORs less than 1 indicating larger treatment effect in low-quality studies.


Twenty-four quality measures were analyzed for 276 RCTs from 26 meta-analyses. Relative ORs of high vs low-quality studies for these quality measures ranged from 0.83 to 1.26; none was statistically significantly associated with treatment effect. The proportion of studies fulfilling specific quality measures varied widely in the 4 medical areas. In analyses limited to specific medical areas, placebo control, multicenter studies, study country, caregiver blinding, and statistical methods were significantly associated with treatment effect on 7 occasions. These relative ORs ranged from 0.40 to 1.74. However, the directions of these associations were not consistent.


Individual quality measures are not reliably associated with the strength of treatment effect across studies and medical areas. Although use of specific quality measures may be appropriate in specific well-defined areas in which there is pertinent evidence, findings of associations with treatment effect cannot be generalized to all clinical areas or meta-analyses.

[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for Silverchair Information Systems
Loading ...
Support Center