Topic Development

The topic for this report was nominated in a public process. At the beginning of the project, we recruited a panel of internal and external technical experts and key informants to give input on key steps including the selection and refinement of the questions to be examined. The panel included internal technical experts from the Johns Hopkins University having expertise in various aspects of the efficacy and/or safety of oral diabetes medications, and external experts who have expertise in diabetes research.

To understand some of the pressing issues concerning the use of oral diabetes medications, we analyzed the recommendations in published guidelines on the treatment of type 2 diabetes. We conducted a search of PubMed and the National Guideline Clearinghouse for all guidelines concerning oral diabetes medications published since completion of the 2007 review. Two investigators reviewed each guideline for inclusion in this process. Guidelines needed to have been written in English, published after July 2007, and included recommendations on the medical management of type 2 diabetes in nonpregnant adults. Additionally, the guideline had to have been sponsored by or authorized by an organization in the United States, United Kingdom, or Canada, and met the criteria for a guideline.²⁴ For each included guideline, two reviewers abstracted the recommendations on medical management and whether the recommendations agreed with the key findings from the 2007 review.

With the technical experts and representatives of AHRQ and the Scientific Resources Center, and with our understanding of the gaps in existing guidelines, we developed the Key Questions that are presented in the Scope and Key Questions section of the Introduction. The final Key Questions focus on the differences among oral diabetes medications, used as monotherapy and used in combination, in their ability to affect intermediate outcomes, long-term clinical outcomes, and their adverse effects.

Search Strategy

We searched the following databases for primary studies for the periods in parentheses: MEDLINE^® (1966 to April 2010), Embase^® (1974 to April 2010), and the Cochrane Central Register of Controlled Trials (1966 to April 2010). We updated the MEDLINE search to December 2010 for long-term clinical outcomes (i.e., all-cause mortality, cardiovascular morbidity and mortality, nephropathy and neuropathy). We developed a search strategy for MEDLINE, accessed via PubMed, based on an analysis of the medical subject headings (MeSH) terms and text words of key articles identified a priori. Our search strategy was similar to the one used for the initial 2007 review,²¹ but it included terms for the additional medications included in this review (Appendix B).

In addition, we received the following material from the Scientific Resource Center:

• Medical reviews of rosiglitazone, pioglitazone, sitagliptin, glyburide, and metformin, combination of metformin and glipizide, combination of metformin and sitagliptin, insulin detemir, exenatide and postmarketing drug safety information on pioglitazone and insulin glargine from the FDA Web site,
• The Scientific Discussion sections of the European Public Assessment Reports for rosiglitazone, pioglitazone, sitagliptin, combination rosiglitazone and metformin, exenatide, insulin detemir, and insulin glargine,
• Health Canada Product Monographs for rosiglitazone, pioglitazone, sitagliptin, combination rosiglitazone and metformin, insulin glargine, and insulin detemir,
• Public registries of clinical trials, such as Clinical Study Results Web site (available at: www.clinicalstudyresults.org) and ClinicalTrials.gov (available at: www.clinicaltrials.gov).

We hand searched 15 journals that most likely to publish articles on this topic (see Appendix C) by scanning the table of contents of each issue for relevant citations from February 2009 through September 2009. We also reviewed the reference lists of each included article and relevant review articles.

The results of the searches were downloaded and imported into ProCite^® version 5 (ISI ResearchSoft, Carlsbad, CA). We scanned for exact article duplicates, author/title duplicates, and title duplicates using the duplication check feature in ProCite.^® From ProCite, the articles were uploaded to DistillerSR (Evidence Partners, Ottawa, Ontario, Canada), a Web-based software package developed for systematic review data management. This database was used to track the search results at the levels of title review, abstract review, article inclusion/exclusion, and data abstraction.

Study Selection

Two independent reviewers conducted title scans in parallel. For a title to be eliminated at this level, both reviewers had to indicate that it was ineligible. If they disagreed, the article was promoted to the next level (Appendix D, Title Review Form). The title review was designed to capture as many studies as possible that reported on the efficacy or safety of oral diabetes medications. These titles were promoted to the abstract review phase.

The abstract review phase was designed to identify studies reporting on the effects of oral diabetes medications on intermediate outcomes, long-term clinical outcomes, or adverse events and side effects (Appendix D, Abstract Review Form). Abstracts were reviewed independently by two investigators, and were excluded if both investigators agreed that the article met one or more of the exclusion criteria (see inclusion and exclusion criteria listed in Table 3). Differences between investigators regarding abstract inclusion or exclusion were resolved through consensus adjudication.

Table 3

Inclusion and exclusion criteria.

Articles promoted on the basis of abstract review underwent another independent parallel review to determine if they should be included for data abstraction (Appendix D, Article Review Form). Differences regarding article inclusion were resolved through consensus adjudication.

During both the abstract review and article review, reviewers indicated if there was a monotherapy comparison or a combination comparison of interest. For studies that were excluded because they did not involve a comparison of interest, reviewers still noted the comparison (see Appendix E for a list of the comparisons that were tallied).

The inclusion and exclusion criteria for this review differed from the initial review. First, this review includes interventions that were excluded from the initial review: dipeptidyl peptidase-4 (DPP-4) inhibitor, glucagon-like peptide-1 (GLP-1) analogs, combination metformin plus DPP-4 inhibitor, combination metformin plus a meglitinide, combination metformin plus GLP-1 analogs, combination of metformin plus a basal insulin, combination of metformin plus a premixed insulin, and combination thiazolidinedione plus a meglitinide. This review includes studies with unambigous medication combinations but not studies in which participants were treated with unspecified adjunctive diabetes medications. We did not update the initial review on acarbose. Second, this review includes outcomes that were not included in the initial review: fractures, cholecystitis, and macular edema. We did not update the initial review on the outcomes of blood pressure, body mass index, 2-hour postprandial glucose, peripheral arterial disease, amputations, quality of life, functional status, anemia, thrombocytopenia, leucopenia, hypervolemia, and withdrawals due to adverse events.

Data Abstraction

We used a systematic approach for extracting data to minimize the risk of bias in this process. By creating standardized forms for data extraction, we sought to maximize consistency in identifying all pertinent data available for synthesis. If reviewers determined that an article addressed both efficacy and safety, multiple data abstraction forms were used.

Each article underwent double review by study investigators for data abstraction and assessment of study quality. The second reviewer confirmed the first reviewer’s data abstraction for completeness and accuracy. Reviewer pairs were formed to include personnel with both clinical and methodological expertise. A third reviewer rereviewed a random sample of articles by the first two reviewers to ensure consistency in the data abstraction of the articles. Reviewers were not masked to the articles’ authors, institution, or journal.²⁵ In most instances, data were directly abstracted from the article. If possible, relevant data were also abstracted from figures. Differences in opinion were resolved through consensus adjudication. For assessments of study quality, each reviewer independently judged study quality and rated items on quality assessment forms (Appendix D, Data Abstraction Review Forms).

For all articles, reviewers extracted information on general study characteristics (e.g., study design, study period, and followup), study participants (e.g., age, gender, race, weight/body mass index, hemoglobin A1c [HbA1c] levels, and duration of diabetes), eligibility criteria, interventions (e.g., initial, maximum, and mean doses, frequency of use, and duration of use), outcome measures and the method of ascertainment, and the results of each outcome, including measures of variability (Appendix D, Data Abstraction Review Forms).

All information from the article review process was entered into the DistillerSR database by the individual completing the review. Reviewers entered comments into the system whenever applicable. The DistillerSR database was used to maintain and clean the data, as well as to create detailed evidence tables and summary tables.

Quality Assessment

Article quality was assessed differently for randomized controlled trials (RCTs) and observational studies. For RCTs the dual, independent review of article quality was based on the Jadad criteria: (1) appropriateness of the randomization scheme, (2) appropriateness of the blinding, and (3) description of withdrawals and dropouts.²⁶ For the updated review, we also included a question to evaluate the overall quality of the study, as suggested by the Guide for Conducting Comparative Effectiveness Reviews.²⁷

We developed a quality assessment tool for observational studies based on the recommendations in the Guide for Conducting Comparative Effectiveness Reviews²⁷ and quality forms previously developed by our EPC.²⁸ The quality assessment included items about the study setting, inclusion and exclusion criteria, key characteristics of enrolled subjects, details about the treatments, details about the outcomes and how they were measured, statistical analysis, losses to followup, and the overall study quality. For both the RCTs and the observational studies, the overall study quality was assessed as:

• Good (low risk of bias). These studies had the least bias, and the results were considered valid. These studies adhered to the commonly held concepts of high quality, including the following: a formal randomized controlled design; a clear description of the population, setting, interventions, and comparison groups; appropriate measurement of outcomes; appropriate statistical and analytic methods and reporting; no reporting errors; a low dropout rate; and clear reporting of dropouts.
• Fair. These studies were susceptible to some bias, but not enough to invalidate the results. They did not meet all the criteria required for a rating of good quality because they had some deficiencies, but no flaw was likely to cause major bias. The study may have been missing information, making it difficult to assess limitations and potential problems.
• Poor (high risk of bias). These studies had significant flaws that might have invalidated the results. They had serious errors in design, analysis, or reporting; large amounts of missing information; or discrepancies in reporting.²⁷

In the initial 2007 review, we did not assess the quality of observational studies or nonrandomized trials.

We had high consistency between the primary and secondary reviewer; therefore, we report only the second reviewers’ quality scores (the second reviewers generally had more research experience than the primary reviewers). We used our study quality assessment to help us understand differences in results between studies.

Applicability

Throughout the report, we discuss the applicability of studies in terms of the degree to which the study population, interventions, outcomes, and settings are typical of the treatment of individuals with type 2 diabetes who are receiving treatment in a usual care setting (conceived as outpatient treatment by internists, family physicians, and endocrinologists).

Data Analysis and Synthesis

For each Key Question, we created a set of detailed evidence tables containing all information extracted from eligible studies. We conducted meta-analyses when there were sufficient data (at least three trials) and studies were sufficiently homogenous with respect to key variables (population characteristics, study duration, and drug dose). We combined medications by class, except for the thiazolidinediones, which were considered as individual drugs (rosiglitazone and pioglitazone) due to their differences in effects.

For continuous outcomes, we recorded the mean difference between groups along with its measure of dispersion. If this was not reported, we calculated the point estimate using the mean difference from baseline for each group. If the mean difference from baseline was not reported, we calculated this from the baseline and final values for each group.²⁹ If no measure of dispersion was reported for the between-group difference, we calculated it using the sum of the variances for the mean difference from baseline for each group. If there were no measures of dispersion for the mean difference from baseline for each group, we calculated the variance using the standard deviation of the baseline and final values, assuming a correlation between baseline and final values of 0.5. If data were only presented in graphical form, we abstracted data from the graphs. For trials that had more than one dosing arm, we chose the arm that was most consistent with dosing in the other trials. When more than one followup interval was reported, we used the data from the followup most similar to the other trials. We reported the rest of the results descriptively. When data were not sufficient to combine in a meta-analysis, we summarized the outcomes by reporting the ranges of values for mean differences from baseline or mean differences between groups (when possible).

For Key Questions 2 and 3, we were unable to conduct meta-analyses on most of the outcomes due to methodologic diversity among the trials such as differences in definitions of selected outcomes or lack of sufficient numbers of trials to combine. When there were sufficient data (at least three trials) and the studies were considered to be similar with respect to important variables (population characteristics, drug comparisons, drug dosage, definition of outcome, and followup time), we performed meta-analyses.

For the outcome of hypoglycemia, we needed to generate categories for the outcomes to match those in the 2007 review. The studies included in the 2007 review had hypoglycemia outcomes categorized as total, serious, and those which led to withdrawal from the study. In order to pool these with the new studies, we categorized those outcomes as: (a) serious hypoglycemia or hypoglycemia leading to withdrawal from the study, and (b) all other. These were then combined with events categorized as: (a) severe hypoglycemia and (b) mild or moderate hypoglycemia, which were the categories for the newly abstracted studies. The categories were based on the definitions of hypoglycemia provided in the studies. Usually, severe hypoglycemia was defined as requiring assistance. In previously included studies from the 2007 review, the hypoglycemia outcomes were reported as the number of people with hypoglycemic episodes (not the number of events). Therefore, in integrating the previously and newly identified studies, we pooled the number of people with events. The number of events is reported descriptively when available. Several studies reported only the rates of events per time of followup; these, too, are described in the text. The count of individuals upon enrollment was used as the denominator for the prevalence of hypoglycemic events. For trials not amenable to pooling, the Mantel-Haenszel risk ratios were calculated with 95 percent confidence intervals surrounding the estimate (STATA Intercooled, version 9.2, StataCorp, College Station, TX).

For continuous outcomes, we used a random-effects model with the DerSimonian and Laird formula to derive pooled posttreatment weighted mean differences.³⁰ For the outcome of hypoglycemia, we calculated pooled odds ratios using the Peto method because trial arms had balanced sample sizes.³¹ Because congestive heart failure and ischemic heart disease were rare events, we calculated pooled fixed-effects odds ratios using the treatment arm continuity correction (reciprocal of the sample size in the opposite treatment group in cells with 0 events).³² Heterogeneity among the trials in all the meta-analyses was tested using a standard chi-squared test using a significance level of alpha less than or equal to 0.10. We also examined heterogeneity among studies with an I² statistic, which describes the variability in effect estimates that is due to heterogeneity rather than random chance.³³ A value greater than 50 percent may be considered to have substantial variability. If heterogeneity was found, we attempted to determine potential reasons by conducting metaregression using study level characteristics such as baseline values, study duration, and dose ratio (dose ratio of drug 1 divided by dose ratio of drug 2). The dose ratio for each drug was calculated as the dose given in the study divided by the maximum approved dose of drug. We conducted sensitivity analyses by omitting one study at a time to assess the influence of any single study on the pooled estimate.

Because statistically significant findings are more likely to be published than studies without statistically significant results (publication bias), we examined whether there was evidence that smaller, negative studies appeared to be missing from the literature. We therefore conducted formal tests for publication bias using Begg’s³⁴ and Eggers tests³⁵ including evaluation of the asymmetry of funnel plots for each comparison of interest for the outcomes where meta-analyses were conducted for Key Question 1. All meta-analyses were conducted using STATA (Intercooled, version 9.2, StataCorp, College Station, TX).

Unadjusted odds ratios were calculated in instances when the total number of deaths was reported for each arm, the total number of participants was reported for each arm, and when measures of association were either not calculated at all or when a comparator which was not of interest was used as the reference group. These unadjusted odds ratios and confidence intervals were calculated using SAS 9.1.3 using the PROC FREQ command.

Data Entry and Quality Control

After a second reviewer reviewed the data that had been entered into DistillerSR, adjudicated data were resubmitted into Web-based data collection forms by the second reviewer. Second reviewers were generally more experienced members of the research team. In addition, two additional investigators audited a random sample of the reviews to identify problems with data abstraction. If problems were recognized in a reviewer’s data abstraction, the problems were discussed at a meeting with the reviewers. In addition, research assistants used a system of random data checks to assure data abstraction accuracy.

Rating the Body of Evidence

At the completion of our review, at least three investigators graded the quantity, quality, and consistency of the best available evidence addressing Key Questions 1, 2, and 3 by adapting an evidence grading scheme recommended by the Guide for Conducting Comparative Effectiveness Reviews.²⁷ We applied evidence grades to the bodies of evidence about each intervention comparison for each outcome. We assessed the strength of the study designs with RCTs considered best, followed by non-RCTs, and observational studies. We also assessed the quality and consistency of the best available evidence, including assessment of limitations to individual study quality (using individual quality scores), consistency, directness, precision, and the magnitude of the effect.

We classified evidence bodies pertaining to Key Questions 1, 2 and 3, into four basic categories: (1) “high” grade (indicating high confidence that the evidence reflects the true effect and further research is very unlikely to change our confidence in the estimate of the effect); (2) “moderate” grade (indicating moderate confidence that the evidence reflects the true effect and further research may change our confidence in the estimate of the effect and may change the estimate); (3) “low” grade (indicating low confidence that the evidence reflects the true effect and further research is likely to change our confidence in the estimate of the effect and is likely to change the estimate); and (4) “insufficient” grade (evidence is unavailable).

Peer Review and Public Commentary

A draft of the evidence report was reviewed by the peer reviewers, AHRQ representatives, and the EPC Program’s Scientific Resource Center. The draft report also was posted to a Web site for public comment. In response to the comments of the peer reviewers and the public, we revised the evidence report and submitted a summary of the comments and their disposition to AHRQ.