NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Bravata DM, McDonald KM, Gienger AL, et al. Comparative Effectiveness of Percutaneous Coronary Interventions and Coronary Artery Bypass Grafting for Coronary Artery Disease [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2007 Oct. (Comparative Effectiveness Reviews, No. 9.)

Cover of Comparative Effectiveness of Percutaneous Coronary Interventions and Coronary Artery Bypass Grafting for Coronary Artery Disease

Comparative Effectiveness of Percutaneous Coronary Interventions and Coronary Artery Bypass Grafting for Coronary Artery Disease [Internet].

Show details

2Methods

Topic Development

The topic for this report was nominated in a public process. With input from technical experts, the Scientific Resource Center for the AHRQ Effective Health Care Program drafted the initial key questions and, after approval from AHRQ, posted them to a public Web site. The public was invited to comment on these questions. After reviewing the public commentary, the Scientific Resource Center drafted final key questions and submitted them to AHRQ for approval.

Search Strategy

Our search strategy used the National Library of Medicine's Medical Subject Headings (MeSH) keyword nomenclature developed for MEDLINE® and adapted for use in other databases. Appendix A provides the details of our search strategies. We did not limit the searches to the English language.

To identify randomized controlled trials comparing PCI and CABG, we used terms such as angioplasty, coronary, and coronary artery bypass surgery. We also manually searched the reference lists of included articles, conference abstracts, and the bibliographies of expert advisors.

To complement the RCT data, we searched for observational data for two purposes—to evaluate the generalizability of the RCT results and to address key questions left unanswered by the RCTs. To evaluate the generalizability of the RCT results, we first identified relevant comparative registries of patients receiving PCI or CABG through discussion with expert advisors. We then sought articles describing the demographics and outcomes of interest for registry patients. We identified additional comparative registries through our literature search for RCTs and from additional, limited MEDLINE® and internet searches. The Scientific Resource Center also conducted a MEDLINE® search for additional studies from the registries already identified and for additional registries.

To identify systematic reviews, we searched MEDLINE®, the Cochrane Database of Systematic Reviews, and the Web sites of the National Institute for Clinical Excellence, Guidelines.gov, and the NHA Health Technology Assessment Programme. We used results from previously conducted meta-analyses and systematic reviews when appropriate.

To further address key question 2g, additional articles on the volume-outcomes association were identified from previous systematic reviews, 21 23 a technical report on the development of the AHRQ Quality Indicators, 24 and subsequent unpublished updates obtained from the authors on CABG and PCI volume indicators.

The Effective Health Care program is dedicated to identifying as many studies as possible that are relevant to the questions for each of its comparative effectiveness reviews. In order to do so, pharmaceutical and device industry stakeholders are systematically requested to provide information regarding their products (e.g. details of studies conducted). Industry is often most familiar with the existence of scientific data concerning their products - in particular, they know of scientific data, protocols, and methodology that may not have made its way into the published literature. The Scientific Resource Center performed a search of the US Food and Drug Administration website for the original manufacturers of the devices relevant to the key questions for this report. A request for information was subsequently sent to the manufacturers, which included the following: a current product label, published and unpublished randomized controlled trials and observational studies relevant to the clinical outcomes.

Study Selection

We developed criteria for inclusion and exclusion based on the patient populations, interventions, outcome measures, and types of evidence specified in the key questions. We retrieved full-text articles of potentially relevant abstracts and conducted a second review for inclusion by reapplying the inclusion criteria. Results published only in abstract form were not included in our analyses.

Interventions of Interest. As outlined under key question 1, we included a variety of PCI technologies including balloon angioplasty, with or without stents. Similarly, we included traditional on-pump or off-pump bypass procedures and on-pump or off-pump minimally invasive procedures.

Outcomes of Interest. The short- and long-term objective outcomes of interest included survival, event-free survival, non-fatal myocardial infarctions, congestive heart failure, stroke, nosocomial infections, respiratory failure or other pulmonary complications, renal failure, cardiac arrhythmias, other procedural complications, and costs. Additionally, we were interested in short- and long-term subjective outcomes including quality of life, freedom from angina, cognitive impairment, productivity, and functional capacity. We were also interested in intermediate outcomes including completeness of revascularization, target lesion revascularization, restenosis following PCI, CABG graft closure, the need for secondary revascularization procedures, readmission rates, and post-procedure discharge to rehabilitation facilities.

Study Designs of Interest. We sought RCTs that compared PCI and CABG in patients with angiographically-proven CAD. We included all such comparative RCTs without limitation by subject population, year, or type of surgical or percutaneous intervention. For a RCT to be included, at least one article describing that RCT had to report at least one of the objective outcomes. Because the primary aim of this report was to evaluate the comparative effectiveness of the two procedures, we excluded RCTs that compared two or more PCI technologies that did not also include a CABG arm. Similarly, we excluded RCTs that compared two or more CABG technologies that did not also include a PCI arm. Finally, we excluded trials that compared either PCI with medical therapy or CABG with medical therapy, unless the trial involved a three-way randomization to PCI, CABG, and medical therapy and reported a randomized comparison of PCI with CABG.

For observational studies addressing either the generalizability of the RCT data or to address key questions left unanswered by the RCT data, we included studies from clinical or administrative databases with at least 1000 recipients of each of the revascularization procedures. We also included observational studies from registries that compared at least 1000 PCI recipients with at least 1000 CABG recipients. To be included, articles of observational studies had to provide sufficient information about the patient populations (e.g., demographics, pre-procedure coronary anatomy, and co-morbid conditions) and procedures performed (e.g., balloon angioplasty versus bare metal stent versus drug-eluting stent types) for us to be able to compare these populations with those included in the RCTs. Also, the observational studies had to report on the outcomes and populations of interest as defined in our key questions.

To determine whether RCTs or observational studies met inclusion criteria, two authors independently reviewed the title, abstract, and full text (as necessary). Conflicts between reviewers were resolved through re-review and discussion.

Data Extraction

We extracted the following data from the included trials: study design; setting; population characteristics (e.g., sex, age, ethnicity, co-morbid conditions, coronary anatomy); eligibility and exclusion criteria; detailed information about the PCI and CABG interventions performed (including adjunctive medical therapies provided post-procedure); numbers of patients screened, eligible, enrolled, and lost to follow-up; method of outcome ascertainment; and results for each outcome. Data were abstracted by two authors independently onto pre-tested data forms (Appendix E). Data abstraction conflicts were resolved by re-review and discussion with other authors.

Because we were interested in both short-term and long-term comparative outcomes, we extracted all the available survival data for PCI and CABG in 30-day intervals post-procedure. For those studies that provided overall survival data in the form of Kaplan-Meier survival curves, we extracted the data directly from the curves as follows: we imported each survival curve figure into Microsoft Paint and created separate figures for PCI and CABG data. We then removed all extraneous information from each figure (e.g., extra lines, words, figure legends) and saved the files in .jpeg format. We exported the .jpeg files into DigitizeIt software (http://digitizer.sourceforge.net/) that enabled us to specify the axes (x=time, y=percent surviving). The software algorithm provides the maximal x/y coordinates for each survival curve. We exported these curves into Microsoft Excel and reduced each dataset to 12 points per year (i.e., monthly data). We then visually checked these data by comparing them to the survival data reported in the text of the article.

For the observational studies, we abstracted the same demographic/baseline characteristic variables as for the RCTs. For key question 2g, we also abstracted information about hospital and clinician volume levels analyzed in each study, methods of risk adjustment, and outcomes assessed. Given concerns about potential biases in the non-RCT data, we did not abstract comparative outcomes of PCI and CABG from the registries that were not adjusted for key baseline population characteristics (e.g., unadjusted mortality).

Quality Assessment of Individual Studies

We used predefined criteria to assess the quality of included trials and observational studies based primarily on the CONSORT statement 25, 26 of reporting for RCTs relevant to the two procedures of interest. Specifically, we considered the method of randomization, the use of intention-to-treat analysis, the report of drop out rates, and the extent to which valid outcomes were described. Blinding and related criteria are less relevant measures of quality for RCTs in which one set of patients receives a surgical procedure and another set of patients does not.

To assess the quality of the observational studies of the registries, we evaluated the extent to which they reported adequate baseline characteristics about the included population and the extent to which valid outcomes were described adjusted for the baseline characteristics.

We applied a three-category quality grading system (A, B, C) to both RCT and observational studies (Table 1) as has been utilized by several prior AHRQ comparative effectiveness reviews. 27, 28 An assigned grade to a study of one design is not equivalent to the same grade for a study of a different design. This grading system does not attempt to assess the comparative validity of studies across different design strata. For example, a “B” rated RCT is not judged to have the same methodological quality as a “B” rated observational study. Thus, both study design and quality grade should be considered when interpreting the methodological quality of a study.

Table 1. Criteria for grading included studies.

Table 1

Criteria for grading included studies.

Grading the Body of Evidence for Each Key Question

We assigned an overall grade describing the body of evidence for each key question utilizing the GRADE system as described by Guyatt and colleagues (Table 2). 29 Specifically, the grade is based on the number and quality of individual studies, duration of follow-up, the consistency across studies, magnitude of effects, applicability, the likelihood of publication bias, and (especially for the observational studies) the potential influence of plausible confounders.

Table 2. Criteria for grading the body of evidence for each key question.

Table 2

Criteria for grading the body of evidence for each key question.

The grades provide a shorthand description of the strength of evidence supporting the major questions we addressed. However, they may oversimplify the many complex issues involved in appraising a body of evidence. The individual studies involved in formulating the composite grade differed in their design, reporting, and quality. As a result, the strengths and weaknesses of the individual studies addressing each key question should also be considered, as described in detail in the text, tables, and figures.

Data Synthesis

To evaluate the comparative effectiveness of PCI and CABG at 1-month, 6-months, 12-months, 24-months, 36-months, and 60-months post-procedures, we computed two summary effects for each outcome of interest at each of these time intervals using random effects models: summary risk differences and summary odds ratios. To calculate the effects sizes for each study at each time interval for each outcome of interest, we calculated the proportion of patients with the outcome of interest, p PCI, (e.g., survival) and a variance for this outcome p PCI q PCI /n (where q=1-p and n is the sample size) for the PCI patients. We repeated this for the CABG patients. We calculated the risk difference effect size as p PCI-p CABG. We calculated the odds ratio effect size as (p PCI/q PCI)/(p CABG/q CABG). a We performed these calculations using Comprehensive Meta-Analysis software (version 2).

There remains considerable debate in the literature regarding the best metric to use to calculate treatment effects for data from 2 × 2 tables—a detailed discussion of the advantages of different effect size metrics has been published. 31, 32 Throughout the text, we present the summary risk differences as the primary outcome metric because several of the outcomes of interest (e.g., procedural mortality) were rare events and the risk difference is a more stable outcome metric than odds ratios under this circumstance. 31, 32 Additionally, it is a readily clinically interpretable measure. We note that we found consistent results between these two outcome metrics.

For consistency, we adopted the following conventions for presenting the results: we present the summary risk difference as a percent difference, all outcomes were calculated in their “positive frame” (e.g., survival rather than mortality, freedom from angina rather than angina, etc.), and all of the forest plots are oriented such that the studies on the left of the origin favor CABG and those to the right of the origin favor PCI. We present PCI-CABG risk differences such that positive numbers favor PCI and negative numbers favor CABG. Similarly, we present PCI/CABG odds ratios such that ratios greater than 1.0 favor PCI and ratios less than 1.0 favor CABG.

To evaluate the association between the number of mammary artery grafts and survival, we performed meta-regression using the number of subjects randomized for weighting the predictor variables and the risk difference in survival between PCI and CABG as the dependent variable. We performed these calculations using SPSS 11.01.

To minimize heterogeneity, we synthesized only those studies describing similar interventions in similar populations. We performed formal assessments of heterogeneity for summary effects and present the Chi2 statistic for heterogeneity. Additionally, we calculated the I2 statistic measuring the extent of inconsistency among the studies' results—which is interpreted as the approximate proportion of total variation in study estimates that is due to heterogeneity rather than sampling error 33 and considered I2 statistics in excess of 50% to be heterogeneous. For those analyses in which we found heterogeneity among the included studies, we performed sensitivity analysis to explore the effects of individual studies on reported summary effects by removing each study individually. Additionally, we re-calculated separate summary effect size for the balloon-era and stent-era trials and for the single-vessel and multi-vessel trials.

We sought evidence of publication bias by evaluating the association between the sample size of a study and the likelihood of that study reporting statistically significant outcomes by visual inspection of funnel plots.

Peer Review and Public Commentary

A draft of this Evidence Report was reviewed by experts in coronary artery disease, PCI technologies, and CABG technologies (Appendix D). These experts were either directly invited by the EPC or offered comments through a public review process. Revisions of the draft were made, where appropriate, based on their comments. The draft and final reports were also reviewed by staff from the Scientific Resource Center at Oregon Health and Science University. However, the findings and conclusions are those of the authors, who are responsible for the contents of the report.

Footnotes

a

We did not calculate the Mantel-Haenszel estimator of the odds ratio because it cannot be used with a random effects model. We did not calculate the Peto estimator of the odds ratio because it has been shown to be a potential biased estimator that can produce both over- and underestimates of the underlying parameters. 30, 31

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (1.2M)

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...