NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

McCrory DC, Brown C, Gray RN, et al. Management of Acute Exacerbations of Chronic Obstructive Pulmonary Disease. Rockville (MD): Agency for Healthcare Research and Quality (US); 2001 Mar. (Evidence Reports/Technology Assessments, No. 19.)

  • This publication is provided for historical reference only and the information may be out of date.

This publication is provided for historical reference only and the information may be out of date.

Cover of Management of Acute Exacerbations of Chronic Obstructive Pulmonary Disease

Management of Acute Exacerbations of Chronic Obstructive Pulmonary Disease.

Show details

2Methodology

This section documents the methods and procedures that were used to develop the evidence report. It begins with a description of the research questions and the evidence model that guided our work, and proceeds to a detailed description of the techniques and approaches that were used in the literature review, including descriptions of the literature search and review parameters, Medical Subject Headings (MeSH) terms used, types of study designs included, number and identity of databases searched, years included in the search, and quality rating criteria. Quality control methods also are described. The section also provides descriptions of the peer-review process, the role of the report Partners, and the manner in which consumers participated in the project.

Evidence Report Questions and Evidence Model

Key Questions

Clinical Assessment and Prognosis: How well does clinical assessment (including history, physical examination, and laboratory, radiographic, and other tests) discriminate between patients with acute exacerbations of COPD and other causes of worsening respiratory status? How well does clinical assessment (including history and physical examination) predict health outcomes or level-of-care needs (ICU or hospital admission, return visit to the emergency room, or need for ventilatory support) for patients presenting for treatment of acute exacerbations of COPD?

Selected Medical Treatments: How effective are the medical modalities (antibiotics, bronchodilators, corticosteroids, and mucous-clearing strategies) that are used to treat acute exacerbations of COPD in alleviating symptoms, resolving the cause of the exacerbation, preventing hospital admission, and decreasing LOS?

In our preliminary literature review, we identified several hundred head-to-head comparisons of different antibiotic treatments for acute exacerbation of COPD. Our Advisory Panel of Technical Experts suggested that we limit the studies of antibiotic treatment to placebo-controlled trials. A systematic review of the comparator trials might have been useful to inform selection of particular classes of antibiotics, particularly if it allowed stratification by severity of exacerbation. However, nearly all studies of antibiotic treatment for acute exacerbation of COPD were performed in outpatient populations with moderately severe exacerbations; it is not clear if the findings from these studies are applicable to hospitalized patients with more severe exacerbations, particularly because differences in the respiratory tract microbial flora occur based on severity of disease. Furthermore, this review would provide limited clinical usefulness, as geographic and temporal patterns of antibiotic resistance and the availability of antimicrobial agents may be more important determinants of antibiotic selection at the local level. Finally, we were concerned that this large and complex review could command all the resources that were available to this project, to the exclusion of other questions.

NPPV: Does the use of NPPV in patients with respiratory failure secondary to acute exacerbation of COPD prevent intubation and/or improve other outcomes, including mortality, morbidity, length of hospital stay, and cost(s) of care?

Causal Pathway

The preceding key questions are linked to the interventions and outcomes of greatest interest to us in the attached “causal pathway” or evidence model (Figure 1).

Figure 1. Evidence model for management of acute exacerbations of COPD.

Figure

Figure 1. Evidence model for management of acute exacerbations of COPD.

Literature Search and Review Parameters

This section describes the search terms and strategies and the databases that were used in the literature retrieval; the article screening and selection process; methods that were used for developing the data collection form, abstracting data, and reviewing and analyzing the literature; and the results of the literature review.

Search Strategies and MeSH Terms

Drs. Douglas McCrory and Cynthia Brown reviewed several iterations of the literature search strategies and reached a level of refinement that excluded as many nonrelevant articles as possible without jeopardizing the inclusion of relevant articles. The search strategies were critically reviewed by the other clinical investigators in the local Work Group (Drs. David Matchar, Eugene Oddone, and Neil MacIntyre). As an additional quality check, the search strategies were reviewed by the COPD Guideline Collaboration Expert Panel that was convened by our study partners, the American College of Chest Physicians (ACCP) and the American College of Physicians-American Society for Internal Medicine (ACP—ASIM). The Work Group members and the ACCP/ACP—ASIM Guideline Collaboration Expert Panel provided several useful suggestions that were subsequently incorporated. The final search strategies are provided in Appendix A, which also includes the MeSH terms. Briefly, the search strategies combined a concept for “COPD” with a “methods” concept and an “intervention” concept. A subset of the MEDLINE search with high specificity (high yield) was achieved by adding an “acute exacerbation” concept; this strategy was used in the EMBASE and Cochrane Controlled Trials Register (CCTR) databases (MEDLINE [online database], 1999; EMBASE [online database], 1999; CCTR [database on CD—ROM], 1999). Based on recommendations from the Work Group, we eliminated articles with asthma or antibiotics as the focus (see “Exclusion Criteria”).

Bibliographic database searches from the preliminary literature review during the topic assessment and refinement phase of the project also contributed citations to the database.

Databases Searched and Years Included

The most productive literature databases were MEDLINE, CCTR, and EMBASE. MEDLINE was searched from the years 1966 to June 1999 for the topics of clinical assessment, treatment, and NPPV; EMBASE was searched from 1974 to 1999 for all topics; and the CCTR database (Issue 4, 1998, of The Cochrane Library) was searched for the treatment topic. The Cumulative Index to Nursing and Allied Health (CINAHL) and Health Services, Technology, and Research (HealthSTAR) databases also were searched, but they yielded few articles. For the EMBASE searches, Andrew Eisan, a Duke University Medical Center librarian, translated the MEDLINE search strategy for use in EMBASE. The number of articles identified for each of the three major topic areas was: diagnostic = 893, treatment = 1,609, and NPPV = 546.

Screening/Quality Rating Criteria and Selection Process

Screening Criteria

Drs. McCrory, Matchar, and Brown led the effort to develop the screening criteria. These criteria, and those for quality rating of the articles, were discussed at length in several Work Group meetings. The criteria underwent several revisions before their final acceptance by the investigators. The final screening criteria for inclusion and exclusion of articles are described below.

Inclusion Criteria

Articles were selected for inclusion in the systematic review based on the following:

  • Patient population. Study subjects were adults who were likely to have COPD based on clinical diagnosis, spirometry, or known or suspected history; subjects must have been experiencing an acute exacerbation of respiratory symptoms. Qualifying respiratory symptoms may have included increased dyspnea, increased quantity or purulence of sputum, or ARF.
  • Interventions. Studies of clinical assessment, antibiotics, bronchodilators, corticosteroids, mucous clearing strategies, and NPPV were included. Additional criteria were as follows:
    • Bronchodilators: All dose ranges/timing; all formulations (metered-dose inhaler [MDI], nebulized, oral, and IV.
    • Corticosteroids: All dose ranges; all formulations (oral, inhaled, and IV).
    • Mucolytic/expectorant interventions: Drug treatment/respiratory therapies (e.g., bland aerosols; all dose ranges; all formulations [nebulized, oral, and IV]) and physiotherapy (including chest percussion, pneumatic vests, flutter device, etc.).
  • Study designs: We included reports of original research or systematic reviews, but limited ourselves to certain study designs according to the topic and purpose (Sackett, Haynes, and Tugwell, 1985).
    • Therapeutics: Randomized controlled trials (RCTs) and quasirandomized or nonrandomized prospective controlled trials; retrospective historical or concurrent cohort comparisons.
    • Clinical assessment—Diagnosis: Studies describing the results of a test (with or without a reference standard test) in a series of patients suspected of having the condition or in two groups of patients, one known to have the condition and the other known not to have the condition.
    • Clinical assessment—Prognosis: Case series or cohort studies providing longitudinal data.
  • Outcomes: For efficacy, we included studies with outcomes that were measured at least 4 hours after the start of the intervention. In addition, at least one of the following outcomes was measured and reported: mortality, hospitalization and LOS, relapse after discharge from outpatient care, relapse after discharge from inpatient care, health-related quality of life (HRQL)/QOL, symptom severity or duration, decreased need for intubation, improved breathing mechanics, improved ventilation (in arterial partial pressure of carbon dioxide [PaCO2]), decreased need for supplemental oxygen, decreased ICU admissions, improved or lack of deterioration of mental status, ability for information at initial workup to predict any of the aforementioned, or adverse reactions and side effects of any intervention mentioned.

Exclusion Criteria

Articles were excluded from review if they were based on population types, interventions, study designs, and other criteria described below.

  • Patient population—Excluded were studies of patients with: chronic MV needs, tracheostomies, asthma, bronchiolitis obliterans with organizing pneumonia, bronchiolitis obliterans, bronchiectasis, cystic fibrosis, and immunocompromised status (had known lung cancer, HIV/AIDS, or tuberculosis; or were on chemotherapy or radiation therapy for any cause).

The decision to exclude studies of patients with asthma was recommended by the Advisory Panel. There was broad agreement that the mechanisms underlying COPD and asthma are different and result in important differences in response to certain treatments. We decided to include patients with COPD who have coincident asthma, and patients with acute exacerbations of COPD who show evidence of airway reactivity. The following were excluded:

  • Study designs—Nonsystematic reviews (traditional narrative reviews).
  • Interventions—Studies relating to the use of sputum culture.
  • Other—Studies published in non-English languages.

Quality Rating Criteria

The criteria used to rate the quality of the articles included: one for assessing external validity (applied to all included studies) and two for assessing internal validity (one for treatment articles and another for articles on prognosis).

External Validity

The criteria for assessing external validity were:

  1. Validity of the underlying COPD diagnosis
    1. Baseline stable ventilatory status (e.g., FEV1) of study population described
  2. Validity of diagnosis of acute exacerbation of COPD. Definition of acute exacerbation of COPD included at least two of the following:
    1. Increased sputum purulence
    2. Increased sputum volume
    3. Increased dyspnea
  3. Characterization of severity of acute exacerbation of COPD. Study described the severity of acute exacerbation of COPD at enrollment based on at least two of the following:
    1. Mental status change
    2. Work of breathing (i.e., respiratory rate or use of accessory muscles)
    3. Ventilatory status (i.e., FEV1 or PEFR, oxygen [O2] saturation or arterial partial pressure of oxygen [PaO2], and PaCO2)
  4. Duration of followup (treatment articles only). Outcomes assessed at ≥ 24 hours.

Internal Validity—Treatment Articles

For articles on drug treatments and NPPV, the internal validity of individual trials was assessed using the scale devised by Jadad, Moore, Carroll, et al. (1996), which was operationalized as follows:

  1. Was the study described as randomized?
    1 = yes
    0 = no
  2. Was the method of randomization well described and adequate?
    0 = not described
    1 = described and adequate
    -1 = described, but not adequate
  3. Was the study described as double-blind?
    1 = yes
    0 = no
  4. Was the method of double-blinding well described and adequate?
    0 = not described
    1 = described and adequate
    -1 = described, but not adequate
  5. Was there a description of withdrawals and dropouts sufficient to determine the number of patients in each treatment group entering and completing the trial?
    1 = yes
    0 = no

Because it is not possible to double-blind studies of NPPV, the maximum number of points for these studies was 3, based on randomization, adequacy of concealment of allocation, and description of dropouts.

Internal Validity—Articles on Diagnosis and Prognosis

Poor adherence to methodological standards for diagnostic test research has been documented in pulmonary medicine (Heffner, Feinstein, and Barbieri, 1998). We considered using a scale that aggregates methodological standards into levels of evidence, but because of the small number of studies that we identified, we decided that a descriptive approach was preferable. For studies on prognosis, we used the levels of evidence as defined by the National Health Service Research and Development Center for Evidence-based Medicine (Ball, Sackett, Phillips, et al., 1998).

  • Level 1. Inception cohort study with at least 80 percent followup; a systematic review (with homogeneity) of inception cohort studies; or a clinical prediction guide validated on a test set.
  • Level 2. Retrospective cohort study or followup of untreated control patients in an RCT or clinical prediction guide not validated on a test set.
  • Level 3. Case series or poor quality cohort studies.

Screening of Titles and Abstracts

The clinical investigators reviewed the record of each article identified through the literature search and made a judgment, based on the previously described inclusion and exclusion criteria, about whether to include it for further review using the full text of the article. The first set of search results (n = 216 records) was reviewed by all five clinicians, and the strength of their agreement to include/exclude each article was tested by a kappa statistic (Table 4). Clinician pairs are ordered by agreement.

Table 4. Interrater agreement in citation screening.

Table

Table 4. Interrater agreement in citation screening.

Agreement was significantly better than chance for all clinician pairs; however, agreement was only fair for most pairs. To ensure high sensitivity of the screening process despite lack of good to excellent agreement, we used several strategies. First, the search results were stratified into high-yield and low-yield sets (see search strategies). The high-yield set was relatively small (n = 216 MEDLINE articles) and was screened by the five clinicians, with a net inclusion rate of 48 percent. The low-yield stratum was much larger (n = 1,767 MEDLINE articles). Each citation was screened by two clinicians, with a net inclusion rate of 8.6 percent. The search strategies employed in the CCTR and EMBASE databases were of the more specific high-yield variety. Second, we used the combination of a pulmonary medicine clinician and a methodologist clinician as screeners. We believed that the differing perspectives of the reviewers would improve the sensitivity of the selection process.

We took several steps to preserve the integrity of the selection process. First, the reviewers were made aware of the disagreements; next, they reviewed a sample of the citations about which they disagreed so that they could identify the reasons for disagreement. Second, because there was evidence of poor agreement, we required that at least two reviewers screen each citation; citations that were included by EITHER reviewer were kept. This ensured that all relevant citations were retained, even though different raters may have been using slightly different thresholds.

The numbers of articles that were included after the “title and abstract” screening were: clinical assessment = 162, treatment = 185, and NPPV = 166. The inclusion rates for the three topics were 18.1, 11.5, and 30.4 percent, respectively.

Screening of Full-Text Articles

The full text of each article included in the title and abstract screening phase was obtained from the Duke University Medical Center Library. The articles were reviewed according to the previously described criteria. (See Appendix B for the screening/data abstraction form.) The majority of this screening was done by Drs. McCrory and Brown. The numbers of articles included after full text screening were: clinical assessment = 102, treatment = 84, and NPPV = 62. The full text screening also was designed to be inclusive; that is, potentially useful articles were not eliminated.

Data Abstraction and Development of Evidence Tables

Those articles that passed the full-text screening were grouped according to topic and were carefully read in their entirety. At this stage, final determination of inclusion and exclusion criteria was made. Data were abstracted onto a computerized form that had been specially designed with input from all investigators. As data abstraction on each article was completed, the form paired with the article was reviewed by another investigator (clinician) for accuracy and completeness. (An example of a completed data abstraction form is shown in Appendix.)

Statistical Analysis Methods

For most topics, we did not perform any statistical analysis. We reported analyses performed by the authors of original reports. In summarizing data on groups of studies, we used standard statistical methods for rates and proportions or group means. In a few instances, when several comparable studies provided estimates of effectiveness for a given treatment comparison, we tested for homogeneity and, if reasonably similar, combined the individual study estimates using a fixed effects model. Meta-analysis was performed using FAST*PRO software (FAST*PRO, 1992).

Summary of Search Results

Table 5 summarizes the numbers of articles retrieved by the literature searches and the article inclusion/exclusion process.

Table 5. Literature selection process by topic.

Table

Table 5. Literature selection process by topic.

Table 6 shows the source of the articles in the literature database. Numbers are the total for each database, followed by incremental citations excluding MEDLINE, CCTR, and EMBASE, respectively. The search strategy employed in the CCTR database was not targeted to a particular topic; because of the nature of the database, we perceived this primarily as a source of articles about treatment efficacy. For that reason, the full count of CCTR citations is listed under “treatment articles”; however, this database did identify some citations relevant to diagnostic and NPPV reviews. Databases are listed in order of the number of articles identified for the treatment search.

Table 6. Literature identification by source database.

Table

Table 6. Literature identification by source database.

Role of Report Partner

Two study partners formed the “private” half of the “public-private partnership” fostered by the Agency for Healthcare Research and Quality (AHRQ) through the Evidence-based Practice Centers (EPC) initiative. The initial partner was the Permanente Medical Group, Inc., of Oakland, California. The Permanente Medical Group played a significant role in the study by nominating COPD as a task order topic. The ACP—ASIM and the ACCP were added as partners very early in the project. Like the Permanente Medical Group, ACCP and ACP—ASM were developing COPD guidelines and thought collaboration with the EPC would be mutually beneficial. Each group was represented on the project's Advisory Panel of Technical Experts and Peer Review Panel. Through their participation on these panels, the two groups were involved at key decision stages of the study, including providing consultation on the key literature search questions and reviewing the first drafts of the evidence tables and the evidence report. The Permanente Medical Group, ACCP, and ACP—ASIM also assisted in writing a dissemination plan for the evidence report.

Quality Control

Internal and external quality control mechanisms were incorporated into each phase of the development of the evidence report.

Internal Quality Control Mechanisms

Quality control procedures were integrated into each step of the literature review. The search strategies had several checks on their comprehensiveness: (1) the strategies were checked and finalized by a medical librarian at the Duke University Medical Center Library who specializes in evidence-based literature reviews; (2) the articles cited in reference lists of reviewed articles were compared with those in the ProCite literature database and, when appropriate, articles were copied, reviewed, and added to the ProCite database; and (3) the Advisory Panel of Technical Experts was asked to submit articles and other documents that met the specified inclusion criteria as a check on the effectiveness of the search strategies.

With regard to the content of the evidence tables, several quality control procedures were included: (1) screening of all articles by at least two clinicians; (2) abstracting of information into the evidence report by two clinician-trained abstractors; and (3) overreading of all abstracted information by at least one clinician. The clinicians responsible for screening articles, training the abstractors, and overreading the abstracted information were paired so that each pair included a pulmonologist and an internist. Kappa statistics were calculated to determine the strength of agreement between clinician-reviewers in the article screening stage and to identify areas where agreement was low. In such cases, the reviewers identified discrepancies and agreed on an interpretation of the selection criteria. To maximize the sensitivity of the screening process, we included articles selected by either reviewer, recognizing that the data abstraction process would permit reexamination of inclusion/exclusion status.

External Quality Control Mechanisms

Two external oversight and review panels were established—the Advisory Panel of Technical Experts and the Peer Review Panel.

The Advisory Panel of Technical Experts, which was initiated early in the project's 12-month timeframe, reviewed progress on the evidence report at four key stages of its development: (1) the identification of the key literature search questions; (2) the first drafts of the evidence tables; (3) the need for any supplemental analyses; and (4) the draft of the evidence report. Draft documents were discussed as a group in two conference calls, in individual telephone calls, and in written communications. The nine-member Panel consisted of clinical and methodological experts in relevant specialty areas, including pulmonology, critical care medicine, respiratory therapy, internal medicine, and epidemiology. Panel members also were chosen to assure representation of three other important constituencies: the Department of Veterans Affairs, managed care organizations, and consumers. Consumers were represented by the American Lung Association's North Carolina office.

The primary function of the Peer Review Panel was to review and comment on the complete draft of the evidence report. The 27-member Panel, which included AHRQ staff and the Advisory Panel of Technical Experts, consisted of clinical and methodological experts in relevant specialty areas, including pulmonology, critical care, respiratory therapy, internal medicine, and epidemiology. Other constituencies represented were consumers, the Department of Veterans Affairs, managed care organizations, and the Cochrane Airways Group. The report was modified based on the Panel's comments, with close attention to relevant studies not included in the report, misinterpretation of findings, and other issues deserving revision within the constraints of the methodology, timeframe, and budget. The format of the report was designed by AHRQ.

PubReader format: click here to try

Views

  • PubReader
  • Print View
  • Cite this Page

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...