Literature Search

To identify relevant citations we searched Ovid MEDLINE®, Ovid MEDLINE® IN-Process (1950 to April Week 3, 2008), Cochrane Database of Systematic Reviews®, Cochrane Central Register of Controlled Trials®, and the Database of Abstracts of Reviews of Effects (3rd quarter 2007) using search terms for included drugs, indications, and study designs. Electronic database searches were supplemented by hand searches of reference lists of included studies and reviews. In addition, we searched the US Food and Drug Administration (FDA) Center for Drug Evaluation and Research, the Canadian Agency for Drugs and Technologies in Health, and the National Institute for Health and Clinical Excellence web sites for medical or statistical reviews and technology assessments. Finally, we searched dossiers of published and unpublished studies submitted by pharmaceutical companies. (See Appendix A for complete search strategies.) All citations were imported into an electronic database (Endnote® v.9.0).

Study Eligibility

All citations were reviewed for inclusion using the criteria shown in Table 2. Two reviewers independently assessed titles and abstracts of citations identified from literature searches. Full-text articles of potentially relevant citations were retrieved and assessed for inclusion by two reviewers, and disagreements were resolved by consensus. Results published only in abstract form (such as a conference proceeding) were not included, because they typically provided insufficient detail for adequate quality assessment.

Table 2

Table 2

Study inclusion and exclusion criteria

Data Abstraction

The following data were abstracted by one reviewer and reviewed by a second: study design; setting and population characteristics (including sex, age, ethnicity, and diagnosis); eligibility and exclusion criteria; interventions (dose and duration); comparisons; numbers screened, eligible, enrolled, and lost to follow-up; method of outcome ascertainment; and results for each outcome. We recorded intention-to-treat results when reported.

For included systematic reviews, we abstracted the searched databases, study eligibility criteria, numbers of studies and patients represented, characteristics of included studies, data synthesis methods, and main efficacy and safety results.

Validity Assessment

We assessed the internal validity (quality) of trials based on the predefined criteria listed in Appendix C. These criteria are based on the US Preventive Services Task Force and the National Health Service Centre for Reviews and Dissemination (UK) criteria.4, 5 We rated the internal validity of each trial based on the methods used for randomization, allocation concealment, and blinding; the similarity of compared groups at baseline; maintenance of comparable groups; adequate reporting of dropouts, attrition, crossover, adherence, and contamination; loss to follow-up; and the use of intention-to-treat analysis. We considered methods to meet criteria for intention-to-treat analysis if outcomes for at least 95% of participants were analyzed according to the group to which they were originally assigned. We considered total attrition of ≥15% in any of the treatment arms to be excessive.

Trials that had fatal flaws were rated poor quality. Trials that met all criteria were rated good quality and the remainder rated fair quality. As the fair-quality category is broad, studies with this rating vary in their strengths and weaknesses: The results of some fair-quality studies are likely to be valid, while others are only probably valid. A poor-quality trial is not valid; the results are at least as likely to reflect flaws in the study design as the true difference between the compared drugs. A fatal flaw is reflected by failing to meet combinations of items of the quality assessment checklist that work together to suggest a potential for bias.

We assessed the quality of systematic reviews using pre-defined criteria developed by Oxman and Guyatt (See Appendix C). These included adequacy of literature search and study selection methods, methods of assessing validity of included trials, methods used to combine studies, and validity of conclusions.

Data Synthesis

A qualitative analysis of the available evidence or lack of evidence was undertaken. We constructed evidence tables (included as a separate document) showing the study characteristics, quality rating, and results for all included studies.

Pooled estimates of effect sizes were estimated by meta-analysis using random-effects models.6 Results from each study were stratified by dose level of the drug intervention arms (high and low doses). Weighted mean differences between drug and control were calculated for outcomes (percent change in A1c, weight loss, fasting plasma glucose, and post-prandial glucose). Risk ratios between drug and control were pooled for withdrawals and adverse events. Forest plots for both weighted mean difference and risk ratio were created to visually inspect the data.7 The Q-statistic and the I2 statistic (the proportion of variation in study estimates due to heterogeneity) were calculated to assess heterogeneity between the effects from pooled studies.8,9 Publication bias was examined using funnel plots to check for asymmetry with respect to precision and magnitude of effect.10 All analysis was done using “R statistical environment” software and Forest plots were generated using RevMan.11, 12

Peer Review and Public Comment

Original Drug Effectiveness Review Project reports are independently reviewed and commented upon by three to five peer reviewers. Peer reviewers are identified through a number of sources, including but not limited to members of professional societies, acknowledged experts in a particular field, authors figuring prominently in the published literature, and persons recommended by the Drug Effectiveness Review Project participating organizations. A list of peer reviewers for Drug Effectiveness Review Project reports is available on the Drug Effectiveness Review Project website (

The Drug Effectiveness Review Project process allows for a two-week public comment period prior to finalization of the report. Draft reports are posted on the Drug Effectiveness Review Project website and interested individuals or organizations can submit comments. Comments received from peer reviewers are considered and revisions made accordingly. Public comments are discussed with the Drug Effectiveness Review Project participating organizations and then a determination is made as to what revisions are appropriate.