NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Aronson N, Lefevre F, Piper M, et al. Management of Chronic Asthma. Rockville (MD): Agency for Healthcare Research and Quality (US); 2001 Sep. (Evidence Reports/Technology Assessments, No. 44.)

  • This publication is provided for historical reference only and the information may be out of date.

This publication is provided for historical reference only and the information may be out of date.

Cover of Management of Chronic Asthma

Management of Chronic Asthma.

Show details



This report is the product of a systematic literature review of the evidence on five key questions related to the management of chronic asthma. These are: (1) whether chronic use of ICS improves long-term outcomes for children with mild-to-moderate asthma; and whether chronic use of ICS in children results in long-term adverse effects; (2) whether, for patients with mild-to-moderate asthma, early initiation of long-term control medication (i.e., ICS) prevents asthma progression; (3) whether, in patients with moderate asthma, adding other long-term controller medications (i.e., leukotriene modifiers, long-acting beta-agonists, or theophylline) to low-moderate dosages of ICS improves control or lowers ICS dosage; (4) whether adding antibiotics to standard care improves the outcomes of treatment for acute exacerbation of asthma; and (5) whether addition of a written asthma action plan to medical management alone improves outcomes; and whether a peak flow monitor-based plan is superior to a symptom-based plan.

The review of treatment-related adverse effects was limited to Key Questions 1 and 3. For Key Question 1, only data on long-term adverse effects in children are reported. In Key Question 3, the short-term adverse effects that were reported in those studies included in the evaluation of the addition of other medications to ICS are summarized.

This is a systematic review of published evidence; inclusion was limited to full length reports published in peer-reviewed medical journals. Articles published in the English language or published in a foreign language with English abstract were included in this systematic review. For assessment of efficacy outcomes, this systematic review was limited to controlled trials, as many characteristics of asthma patients (e.g., disease severity, treatment compliance, concurrent treatments) are likely to affect the outcomes of interest and thus, confound the interpretation of the effects of the specific treatment being evaluated. Most of the trials included in this systematic review were randomized, but nonrandomized controlled trials were also included. Uncontrolled studies were excluded from this review, except for the review of adverse effects due to ICS.

The protocol for this review was prospectively designed to define: study objectives; search strategy; patient populations of interest; study selection criteria and methods for determining study eligibility; outcomes of interest; data elements to be abstracted and methods for abstraction; and methods for study quality assessment.

A supplementary meta-analysis accompanies this systematic review. Meta-analyses of the addition of long-acting beta-agonists to either a fixed ICS dose, or to a standard ICS dose in comparison with an increased ICS dose alone were conducted for the following outcomes: FEV1 and PEF lung function outcomes; and beta-2 agonist usage. Other symptom-based outcomes, such as symptom-free days, and utilization outcomes, such as ER visits and hospital admissions, were abstracted and considered for meta-analysis. However, due to variability in reporting or lack of sufficient data, useful meta-analyses of these outcomes were not possible.

The development of the evidence report and supplementary analysis was subject to extensive expert review. A Technical Advisory Group (TAG) provided ongoing guidance on all phases of this project. In addition, a preliminary analysis of the evidence base for this report was reviewed by the Blue Cross and Blue Shield Association Technology Evaluation Center (TEC) Medical Advisory Panel (MAP). This interdisciplinary panel comprises experts in technology assessment methods and clinical research, and also includes managed care physicians from Blue Cross and Blue Shield and Kaiser Permanente health plans. The draft report was also reviewed by a panel of external reviewers that included experts and stakeholders. (Appendix A lists the members of the TAG, external expert reviewers, and the Blue Cross and Blue Shield Association TECMAP.)

The TAG included eight members. Stanley Szefler, M.D.; William Busse, M.D.; Noreen Clark, Ph.D.; William Kelly, Pharm.D.; and Romain Pauwels, M.D., Ph.D. are all nationally recognized experts in asthma treatment and/or clinical research, and were appointed by the NHLBI. Barbara P. Yawn, M.D., M.S., M.S.P.H. and Lee Albert Green, M.D., M.P.H. are clinicians with experience both in asthma and in evidence-based medicine/guideline development, and were appointed by the American Academy of Family Physicians (AAFP). Louis M. Mendelson, M.D., an expert in pediatric asthma, was appointed by the American Academy of Pediatrics (AAP).

In order to construct a balanced panel of peer reviewers, a broad-based mailing search was conducted to identify qualified reviewers. A total of 73 letters were sent to all asthma-related societies and consumer groups, device manufacturers, pharmaceutical companies, and other major professional groups with an interest in asthma. From the responses, a group of 15 peer reviewers was compiled, representing independent asthma experts and methodologists, together with appointees from major professional societies, consumer organizations and private industry. Four reviewers were invited by the TEC under the auspices of this task order for their expertise in pediatrics, asthma, and systematic review methodology. Eight of the external reviewers were appointed by professional organizations: the American Medical Association; American Lung Association; American College of Chest Physicians; American College of Emergency Physicians; American Society of Health-System Pharmacists; National Medical Association; American College of Asthma, Allergy and Immunology; and the AAP.

One external reviewer represents the National Institute of Allergy and Infectious Disease of the National Institutes of Health. Two external reviewers represent the pharmaceutical industry. One reviewer was from the technical staff of Aventis Pharmaceuticals Products Inc., (formerly Rhone-Poulenc Rorer), which markets Azmacort® (triamcinolone acetonide inhalation aerosol), which is "indicated in the maintenance treatment of asthma as prophylactic therapy." Another was from the technical staff of 3M Pharmaceuticals, which markets Maxair® (pirbuterol acetate inhalation aerosol), which is "indicated for the prevention and reversal of bronchospasm in patients 12 years of age and older with reversible bronchospasm including asthma" and QVAR® (beclomethasone dipropionate HFA inhalation aerosol), which is "indicated in the maintenance treatment of asthma as prophylactic therapy."

Search Strategy for the Identification of Articles

A comprehensive literature search was performed that attempted to identify all publications of relevant controlled trials (see "Selection Criteria, Types of Studies"). Both MEDLINE and EMBASE databases were searched. These online sources were searched for all articles published since 1980 that included at least one of the following textwords (tw) or Medical Subject Headings (MeSH®) terms in their titles, their abstracts, or their keyword lists:

  • Leukotriene antagonists (including all MeSH terms under this heading) OR zileuton (tw) OR montelukast (tw) OR zafirlukast (tw) OR cromolyn (tw) OR nedocromil (tw) OR theophylline (including all MeSH terms under this heading) OR albuterol(MeSH) OR albuterol(tw) OR salmeterol (tw) OR flunisolide (tw) OR fluticasone (tw) OR beclamethasone (tw) OR budesonide (tw) OR dexamethasone(tw) OR triamcinolone (tw) OR steroids (including all MeSH terms under this heading)
  • Adrenergic beta-agonists (including all MeSH terms under this heading) OR albuterol(tw) OR bitolterol(tw) OR isoetharine(tw) OR isoproterenol(tw) OR metaproterenol(tw) OR orciprenaline(MeSH) OR pirbuterol(tw) OR terbutaline(tw) OR ipratropium(tw) OR adrenal cortex hormones (including all MeSH terms under this heading)
  • (Peak expiratory flow rate(MeSH) OR (peak(tw) AND flow(tw)))
  • (Meter(tw) OR meters(tw) OR monitor(tw) OR monitors(tw) OR monitoring(tw))
  • (Action(tw) AND (plan(tw) OR plans(tw))) OR self care(MeSH) OR patient care planning (MeSH) OR patient participation(MeSH)
  • Beclomethasone(tw) OR budesonide(tw) OR dexamethasone(tw) OR flunisolide(tw) OR fluticasone(tw) OR triamcinolone(tw)
  • Leukotriene antagonists (including all MeSH terms under this heading) OR zileuton(tw) OR montelukast(tw) OR zafirlukast(tw)
  • Cromolyn(tw) OR nedocromil (tw)
  • Theophylline (including all MeSH terms under this heading)
  • Adrenergic beta-agonists (including all MeSH terms under this heading) OR orciprenaline (MeSH) OR albuterol(tw) OR bitolterol(tw) OR isoetharine(tw) OR isoproterenol(tw) OR metaproterenol(tw) OR pirbuterol(tw) OR terbutaline(tw) OR salmeterol(tw)
  • Antibiotics (including all MeSH terms under this heading)

The search results were then limited to include only those articles that were indexed under the MeSH® term asthma (including all MeSH terms under this heading) OR asthma (tw), that addressed studies on human subjects, and that were indexed under any of the following study design terms:

  • Clinical trials (including all MeSH terms under this heading) OR intervention studies (MeSH) OR double-blind method (MeSH) OR single-blind method (MeSH) OR placebos (MeSH) OR random allocation (MeSH)
  • Document type=controlled clinical trial OR document type=randomized controlled trial
  • Control? (truncated tw) OR placebo? (truncated tw) OR random? (truncated tw) OR blind? (truncated tw)
  • Cohort studies (MeSH)

Additional details on MEDLINE and EMBASE search strategies can be found in Appendix B. The MEDLINE and EMBASE databases were last searched in August 2000. Total retrieval through this date is 4,235 English and 343 non-English references.

To supplement the above strategy, the abstracts presented at the 2000 meeting of the American Thoracic Society also were searched. In addition, potentially relevant studies published before 1980 that were referenced in the post-1980 literature, or identified as key references by the TAG were retrieved and evaluated. Recently published articles were identified by TEC staff or by TAG members.

A total of 647 full-length journal articles in English were retrieved after the abstract review (see "Methods of the Review" in this chapter). Each study was initially assessed for potential to address any of the topics of interest, and reviewed against all potentially relevant study selection criteria. A further 21 articles in languages other than English but with an English language abstract, were also reviewed for possible inclusion. A total of 87 articles met the study selection criteria for inclusion in this systematic review.

Study Selection Criteria

Criteria that were specific to each key question were developed for selecting studies for inclusion in this review. Following is a summary of the criteria used for defining the types of participants, types of interventions and types of studies. In general, this systematic review was limited to comparative intervention trials that used a concurrent control group. However, as described in the following sections, observational studies were included for two questions: (1) immediate versus delayed corticosteroid use; and (2) adverse effects.

Types of Participants

All patients included in this systematic review had persistent asthma requiring treatment. Where key questions addressed a population of a specified level of severity, judgements of severity level for study populations were based on the 1997 NHLBI guidelines (National Heart, Lung, and Blood Institute, 1997).

However, few if any studies used selection criteria that exactly matched specific NHLBI severity classifications. Many studies included lung function eligibility parameters that spanned two or more levels of severity. Few studies included symptom frequency, a major determinant of severity by the NHLBI system, as an eligibility criteria. As a result, the NHLBI criteria were grouped and/or modified in order to classify study populations into the following defined categories:

Mild-to-Moderate Asthma

Limits defined as:

  • FEV1 >60 percent of predicted, PEF variability >20 percent; OR
  • symptoms >2x/week to daily; OR
  • nocturnal symptoms more than 2x/month; OR
  • population could not be classified into any of the above categories, but study appeared to address a population primarily consisting of mild to moderate asthmatics; OR
  • population was a mixed population where the majority appeared to be mild to moderate asthmatics.

Moderate Asthma

Limits defined as:

  • FEV1 or PEF 60-80 percent predicted and PEF variability >30 percent; OR
  • daytime symptoms >1x day; OR
  • nocturnal symptoms >1x/week OR
  • exacerbations >2x per week, affecting activity; OR
  • daily use of inhaled short-acting beta-2 agonists; OR
  • population could not be classified into any of the above categories but study appeared to address a population primarily consisting of moderate asthmatics; OR
  • population was a mixed population where the majority appeared to be moderate asthmatics.

More Severe Asthma

Results and Conclusions Parts 1, 2, and 3 of this evidence report address management issues for severity levels of either mild-to-moderate or moderate asthma. During initial evidence review topic formulation, these were identified as the populations for which information was most lacking. Studies that otherwise met inclusion criteria but evaluated populations of asthmatics that were primarily composed of severe asthmatics were, therefore, excluded from this review.


Where studies included a mixed population, results of the subgroup of interest to the key question were included when the study stratified at least 10 similarly treated asthma patients and reported baseline demographics for the stratified subgroup.


Where key questions addressed a pediatric population, the following inclusion criteria were applied:

  • studies that enrolled only patients <18 years of age; or
  • studies that stratified outcomes for patients <18 and reported baseline demographics for the stratified subgroup.

In addition, for retrospective studies of long-term adverse effects of ICS, studies were included that:

  • enrolled children and/or young adults up to the age of 40 years, and indicated that a substantial proportion of the study population had been treated as children with ICS for asthma.

Patients with Exacerbations of Asthma

Patients given standard care for exacerbation of asthma included the following populations: patients without signs and symptoms of a bacterial infection; patients with signs and symptoms of a bacterial infection; patients with signs and symptoms of sinusitis.

Types of Interventions

Except for investigation of long-term adverse events of ICS therapy, studies that compared the outcomes of managing asthma with the treatment of interest to an identified standard were required.

Chronic Inhaled Corticosteroids for Children with Mild-to-Moderate Asthma

  1. Studies that made any of the following comparisons were included:
    • ICS vs. placebo
    • ICS vs. no treatment control
    • ICS vs. an alternative medication for mild asthma (as-needed or long-acting beta-2 agonists, theophylline, mast-cell stabilizers [e.g., cromolyn, nedocromil], or combinations of these medications)
    • addition of ICS to usual care for mild asthma (as-needed or long-acting beta-2 agonists, theophylline, mast-cell stabilizers [e.g., cromolyn, nedocromil], or combinations of these medications).
  2. studies making appropriate comparisons were limited to those for which the treatment duration was at least 12 weeks; and
  3. were limited to those for which at least 90 percent of included patients had not been treated with other long-term control medications (leukotriene antagonists, long-acting beta-2 agonists, ICS) for at least 4 weeks prior to start of ICS.

Adverse Effects of Chronic Inhaled Corticosteroids in Children

Studies were included that:

  1. reported on ICS treatment;
  2. for which the treatment duration was at least 1 year.

Early Compared to Delayed Initiation of Long-Term Controller Medication

Studies were included in which:

  1. Some or all patients started long-term control medication during the study (ICS, leukotriene antagonists, cromolyn/nedocromil, or theophylline):
    • treatment group treated immediately following the diagnosis of asthma compared to a control group that received the same treatment after a period of delay; OR
    • population stratified by duration of asthma prior to initiation of long-term control medication and outcomes compared across the different strata;
  2. Treatment duration was at least 1 year;
  3. At the start of the study, no more than 10 percent of the population (a) were currently being treated with, or (b) had been continuously (>1 month) treated in the past with the long-term control medication being studied.

Addition of Other Therapies to Inhaled Corticosteroids

Studies were included in which:

  1. Study comparisons included:
    • ICS alone to ICS plus leukotriene antagonists, or long-acting beta-2 agonists, or theophylline; OR
    • two different long-term control medications in patients on ICS; OR
    • the addition of an alternative medication to an increased dose of ICS for patients already on corticosteroids;
  2. Treatment duration was at least 4 weeks;
  3. At least 90 percent of patients in the study were on ICS or the subgroup of patients on ICS were analyzed separately and this subgroup otherwise met the eligibility criteria for this question;
  4. Not more than 10 percent of the patients in the population or in the subgroup were on oral corticosteroids.

Addition of Antibiotic Therapy to Standard Care for Exacerbations of Asthma

Studies were included in which standard care plus antibiotics was compared to standard care alone in the treatment of acute asthma exacerbations.

Standard care was defined as asthma medications; symptomatic relief medications such as decongestants and cough suppressants; and supportive care such as fluids and monitoring.

Written Asthma Action Plans

Studies were included if:


The intervention delivered in the study was a written action plan (based either on peak flow monitoring or symptoms) as defined by the following three components:

  • the patients were given a written algorithm;
  • the algorithm identified specific changes in symptoms or other clinical indicators that should trigger adjustments in medications; and
  • the algorithm provided specific instructions on how to adjust medications in response to such triggers.

Many publications lacked sufficient detail on the written asthma plan, so a brief survey was sent to the researchers of each article that was reviewed. If a study lacked sufficient detail for reviewers to determine whether a written action plan had been used, the survey response was used to make the determination.


The study compared:

  • medical management alone vs. medical management plus a written action plan; or
  • The use of a peak-flow meter based action plan plus medical management vs. a symptom based-action plan plus medical management.


Treatment duration was at least 12 weeks.


The intervention and control groups received the same treatment, except that:

  • the intervention group also received a written action plan; or
  • if both groups received a written action plan, in one group medication adjustments were triggered by symptoms and in the comparison group the trigger was peak flow readings; or
  • different schedules of peak flow monitoring were compared; or
  • the use of peak flow monitoring and/or written asthma plan for routine chronic management was compared to use for acute exacerbations.

Because delivery of a written action plan requires instruction to patients, 1 hour or less of patient education, instruction or training was considered to be integral to the action plan.

Studies were excluded if the comparison of interest was confounded by additional treatment components in the intervention group that were not provided to the control group.

  • Commonly occurring examples of such confounding of the effects of a written asthma plan were: optimization of medications in the intervention group only; or education programs of more than 1 hour in the intervention group only.

Types of Studies

Selection Criteria for All Studies Included in this Systematic Review

  1. Full-length report in peer-reviewed medical journals.
  2. Published in the English language; or published in a foreign language with English abstract.
  3. Study reported outcomes relevant to this systematic review.
  4. Where there were multiple reports of a single study, only the report judged to be most recent and complete, based on number of included patients and length of follow-up, was included.

If additional relevant outcomes were included in the duplicate reports, these data were abstracted and added to the data from the primary report with citation to the supplementary articles

Selection Criteria for Studies of Effectiveness of Long-Term Controller Medications and Antibiotics

  1. Study design was a comparative or crossover clinical efficacy trial with a concurrent control group.
  2. For studies of antibiotic therapy only, crossover design was excluded, since antibiotic therapy targets an acute exacerbation that cannot be reliably duplicated in a crossover design.
  3. For studies of early compared to delayed initiation of long-term controller medication, prospective or retrospective cohort studies were also included, when patients were stratified by duration of asthma prior to long-term control medication use and outcomes compared across the different strata.
  4. Reports on a group of at least 10 evaluable, similarly treated asthma patients per study arm.

Selection Criteria for Studies of Long-Term Adverse Effects of Inhaled Corticosteroid Use in Children

  1. Study design was a comparative clinical trial, cohort study, case control study, or cross-section study.
  2. Reported on a group of at least 25 evaluable, similarly treated asthma patients per study arm.
  3. For growth outcomes:
    • Studies of short-term growth were restricted to randomized clinical trials.
    • Studies of long-term growth were restricted to studies that assessed final attained adult height and controlled for confounding variables.
  4. For subcapsular cataract, clinical series studies were also included.
  5. For HPA axis function, studies also were included that used a pre-post single-arm design, where baseline HPA axis function was measured before initiation of ICS.

Selection Criteria for Studies of Written Asthma Action Plans

  1. The study was a randomized controlled trial in which patients were randomly allocated to the intervention and control groups.
  2. The study reported on a group of at least 25 evaluable, similarly treated asthma patients per study arm.

Studies were excluded if the study design did not include random allocation of subjects to study group. For example, a study that randomized physicians to offer or not offer a written action plan to patients was excluded.

Types of Outcome Measures

Trials were included if they reported at least one of the following outcomes, each of which were compared and analyzed separately:

  1. Lung function measures:
    • FEV1
    • PEF
    • Bronchial hyperresponsiveness
  2. Patient (or family) reported symptom-based measures
    • frequency of symptoms (symptom-free days, percent of days with symptoms, percent of nights with symptoms)
    • symptom scores
    • frequency of acute exacerbations
    • frequency of nocturnal awakenings
    • overall or asthma-specific quality of life
  3. Utilization parameters
    • hospitalizations
    • intensive care unit admissions
    • ER and urgent care visits
    • missed work and school days
  4. Medication use outcomes
    • oral corticosteroid use
    • short-acting beta-2 agonist use
  5. Treatment-related morbidity in children and adolescents
    • vertical growth
    • effect on bone mineralization and osteoporosis
    • suppression of HPA axis
      -cortisol levels
      -ACTH stimulation testing (i.e., cosyntropin stimulation)
    • ocular toxicity
  6. Treatment-related morbidity outcomes for studies that evaluated the addition of other medications to ICS:
    • headache
    • central nervous system (CNS) morbidity (e.g., seizures) and tremors
    • cardiac dysfunction
    • gastrointestinal (GI) dysfunction: dyspepsia, nausea, vomiting, diarrhea
    • upper respiratory infections and sinusitis
    • throat irritation, hoarseness, unpleasant taste
    • sleep disorders
    • hepatic toxicity

Adverse Events

Data on adverse events were abstracted only from included studies that reported long-term adverse events for ICS use in children (Results and Conclusions, Part 1) and from included studies of the addition of treatment medication to continuing ICS therapy (Results and Conclusions, Part 3).

For each study arm, the numbers of enrolled patients experiencing specific adverse events were abstracted exactly as reported by study authors. No attempt was made to stratify according to severity, since few studies presented information on severity. If studies reported the total number of patients experiencing any adverse event, or experiencing asthma treatment-related adverse events, these data were also abstracted. Finally, the number of patients who dropped out of the study due to adverse events were abstracted separately from patients who dropped out due to disease progression or acute exacerbation. Where reported, results for these three parameters were compared between treatment arms within each study by chi-square or Fisher's exact test.

Individual adverse events were categorized in groups as described in the preceding section. Numbers of patients reported as experiencing individual adverse events were summed within each category. Because a single patient could be represented more than once in this summation, these results were only compared qualitatively for obvious differences between study arms.

Abstraction and analysis of data on adverse events present particular difficulties. The difficulties encountered in this project are representative of the general problem of the limitations of clinical trials as a source of data on adverse events. One well-recognized problem is that some adverse events may be so infrequent that clinical trials are not large enough to capture events that may be of concern when the treatment is used in the general population of patients. A second problem is inconsistency in how adverse events are reported and measured. Efforts to improve standards in reporting of randomized trials have emphasized the need for more thorough and systematic reporting of the spectrum of adverse effects for an intervention (McPeek, Gilbert, and Mosteller, 1980).

Methods of the Review

Determining Study Eligibility

Study selection was a two-stage process. All abstracts were initially reviewed by one member of the study team. Any excluded abstracts were reviewed by a second member of the study team. If the second reviewer agreed that the abstract should be excluded, then the citation was excluded. If either reviewer indicated that the abstract should be included, the article was retrieved for full review against the formal study selection criteria.

The full-length journal articles for all included abstracts were reviewed independently by two researchers using the full-length journal article selection criteria for all topics of interest. Included articles were assigned to a topic(s) of interest; excluded articles were assigned a coded reason for exclusion. Where the two reviewers were in disagreement, a third review was performed by one of the authors of this report. If no consensus was reached following the third review, the article was discussed by the entire asthma study team and a consensus decision was reached. If substantial disagreement remained after review by the entire study team, the article was brought to the TAG, and consensus reached after consultation with TAG members. The resulting bibliography of included studies was circulated to the TAG for review for possible omissions.

Reports Published in Languages Other than English

The literature search identified 343 titles and/or abstracts of reports that had an English abstract but were published in languages other than English. Abstracts were reviewed according to the abstract selection criteria. From these, 21 full-length journal articles were retrieved for review. Publication languages included Japanese, Spanish, Danish, Polish, German, French, and Chinese. A translator was identified for each language, and the article was reviewed against the inclusion criteria by the translator with the assistance of one of the study reviewers. Of the 21 articles, two met study selection criteria and were included in this systematic review.

Data Abstraction

Two reviewers independently abstracted data from each eligible study, recording it with electronic database software (Microsoft® Access 97). The data elements that were abstracted are listed in the data abstraction forms. Data elements were grouped into the following broad categories: trial identifiers; study design and methods (including enrollment and withdrawal numbers); patient characteristics; lung function outcomes; symptom outcomes, medication outcomes, utilization outcomes, and adverse events. If an article did not report exact numerical values for one or more of the data elements, the reviewers estimated them from figures if they were available in the published reports.

Detailed printed directions for consistent data abstraction were provided to all reviewers. Initially, all reviewers abstracted a test set of three articles and reported and discussed results in detail with supervising staff. Reviewers were then divided into pairs and assigned papers for specific topics. After each pair of reviewers completed data abstraction, their databases were compared electronically. Because electronic comparison of the two databases revealed both substantive and nonsubstantive differences, it was not possible to quantify only the substantive differences. Nonsubstantive differences included differences in spelling, capitalization, wording, spacing, minor differences in estimation from graphs, and other discrepancies that were easily resolved. Substantive differences included errors in abstraction and differences in interpretation that were discussed and, in most cases, resolved by consensus of the two reviewers. In rare cases, discrepancies were resolved by a third reviewer. Frequent staff meetings allowed for discussion of common problems and further directions for consistent abstraction.

Quality Assessment for Sensitivity Analysis

The objective of quality assessment for this systematic review was to identify a group of higher quality trials, for purposes of sensitivity analysis. The meta-analysis included a quantitative sensitivity analysis, and throughout this systematic review, qualitative sensitivity analyses have been included in study conclusion summaries. The sensitivity analyses compared the results reported and conclusions reached from all included studies to results and conclusions drawn by examining the outcomes of only higher-quality studies.

Sensitivity analysis based on study quality is useful because trials of lower quality generally overestimate the effectiveness of an intervention compared to higher quality trials. Approximately two decades ago, Chalmers and coworkers showed that randomized trials report smaller treatment effects than nonrandomized studies (Chalmers, Smith, Blackburn et al., 1981). Subsequently, many methodologists have attempted to identify the characteristics that define the quality of randomized trials, and to test whether such characteristics have an effect on study results (Schulz, Chalmers, Hayes et al., 1995). Recent analyses suggest that well-designed observational studies (using either a cohort or case-control design) may produce estimates of effectiveness that are comparable to randomized controlled trials (Concato, Shah, and Horwitz, 2000; Benson and Hartz, 2000). Nonetheless, experimental design using randomized controls remains the gold standard for studies of efficacy (Pocock and Elbourne, 2000).

Although many quality scales have been used to assess the quality of randomized controlled trials, there is a dearth of empirical evidence to validate such scales. Indeed, Juni and colleagues recently illustrated the hazards of using summary quality scores to select or pool studies for meta-analysis (Juni, Witschi, Bloch et al., 1999). They identified 25 different quality scales, which they tested for a meta-analysis of 17 trials comparing low molecular weight and standard heparin. No significant association between summary quality scores and treatment effects was found; and the results of different quality scales yielded different conclusions concerning which treatment was superior.

Although the use of quality summary scores is problematic, there are three domains of study quality that have been tested in empirical studies. These are: concealment of treatment allocation during randomization; double-blinding; and handling of withdrawals and exclusions. While there is evidence suggesting that these quality domains are associated with more valid estimates of treatment effects, not all domains have been reported as significant in all studies (Mulrow and Oxman, 1997; Schulz, Chalmers, Hayes et al., 1995; Juni, Witschi, Bloch et al., 1999; Moher, Pham, Jones et al., 1998). In an editorial accompanying the Juni study, Berlin and Rennie (1999) suggested that, to be clinically relevant, quality assessment of trials should focus on key aspects of research design relative to the outcomes of interest. Thus, where an outcome requires subjective judgement, for example, assessment of asthma symptoms, double-blinding may be of paramount importance. However, double-blinding may matter less for outcomes where there is little discretion regarding assessment or interpretation.

Moreover, assessment of study quality generally depends on information reported in journal articles, and the absence of such information may reflect incomplete reporting rather than flawed study design. This point is especially germane to studies published prior to the CONSORT (Consolidated Standards of Reporting Trials) statement, which was published in the Journal of the American Medical Association in 1996, in order to disseminate a standard for completeness of reporting in journal articles (Begg, Cho, Eastwood et al., 1996). As was the case in the prior evidence reports for the Agency for Healthcare Research and Quality (AHRQ) performed by this Evidence-based Practice Center, information on concealment of allocation was reported infrequently (Aronson, Seidenfeld, Samson et al., 1999; Aronson, Seidenfeld, Piper, et al., in press). This was found to be true of recent trials, and not confined to those papers published prior to the CONSORT statement.

To supplement the general study quality characteristics that have been validated in the literature, six asthma-specific quality indicators were also developed for purposes of sensitivity analysis. These were primarily study design features to control for confounders of treatment effect relevant to the clinical setting of asthma. These included: establishing reversibility of airway obstruction, controlling for other medication use, reporting compliance, and addressing seasonality. In addition, a priori reporting of power calculations and accounting for exclusions and withdrawals were judged to be study quality characteristics pertinent to this body of evidence. A limitation of the asthma-specific quality indicators is that they have not been validated. These indicators are based on the judgement of the authors of this evidence report in consultation with the TAG.

Criteria to Define Higher Quality Trials for the Sensitivity Analysis

The definition for higher quality studies is applicable only to randomized controlled trials and excluded nonrandomized controlled trials and single-arm studies. It includes general quality indicators that have been shown to be associated with a bias in magnitude of effect, and asthma specific study features that control for potential confounders of outcomes.

To be defined as a higher quality study for purposes of sensitivity analysis, a trial needed to meet three general quality indicators:

  1. The study was a double-blinded randomized controlled trial.
  2. At least one of the following thresholds for minimizing exclusions from analysis was met:
    1. Less than 10 percent of subjects within each study arm were excluded from the analysis AND the percentage of subjects excluded from analysis in each arm was less than a 2:1 ratio; OR
    2. less than 5 percent of subjects were excluded in each study arm; OR
    3. results were reported as an intention-to-treat analysis, i.e., all patients randomized to treatment were included in the endpoint analysis for the outcomes of interest.
  3. Allocation of patients to treatment arms was concealed.

In the meta-analyses, additional sensitivity analyses for the effects of study quality were performed using modified criteria. The meta-analyses required at least three studies for pooling. As a result of the dearth of trials reporting concealment of treatment allocation, the initial attempt at sensitivity analysis for study quality failed to yield three studies that could be combined. As an alternative, two sensitivity analyses were conducted with modified criteria for defining higher quality studies. First, the general quality criteria were relaxed by dropping the requirement for concealment of allocation while simultaneously restricting to studies that reported a minimum of four of the six asthma-specific quality indicators. Second, the criteria were further relaxed by dropping the requirement for asthma-specific indicators. Thus, the most relaxed definition of higher quality studies required only that two general quality criteria be met: (1) double-blinding; and (2) meeting the predefined threshold for minimizing exclusions from analysis.

Application of the General Quality Criteria

A study was classified as double-blinded if stated as such in the publication without further description of the method of blinding and if the study used a placebo. If a placebo was used, but there was no mention of double blinding, the study was classified as single-blinded. If a placebo was not used, or if there was no mention that a placebo was used, or if it was stated that the study was unblinded, the study was classified as unblinded.

"Excluded from the analysis" refers to all patients who were randomized to treatment in the study but were not included in the analysis of results. Subjects excluded from the analysis were those not included in the results for any reason, including: withdrawn after randomization, lost to followup, or with missing data. In the evidence tables, the number of excluded subjects for each study equals the number of enrolled (randomized) patients minus the number of evaluable patients.

Concealment of allocation addresses whether the initial allocation of patients to different treatment arms was concealed from the subjects and investigators and reported in the publication. According to the Cochrane Reviewer's Handbook (Version 3.0.2), "Using an appropriate method for preventing foreknowledge of treatment assignment is crucially important in trial design. When assessing a potential participant's eligibility for a trial, those who are recruiting participants and the participants themselves should remain unaware of the next assignment in the sequence until after the decision about eligibility has been made. Then, after assignment has been revealed, they should not be able to alter the assignment or the decision about eligibility."

Allocation concealment is distinct from the method of randomization and can be achieved in a number of ways including use of a central treatment assignment site, use of pharmacy-prepared and coded drugs, or use of preprepared opaque envelopes containing the treatment assignment if one of the onsite investigators is involved in assignment to treatment arms. Studies have shown that trials with clearly inadequate concealment allocation or with unclear allocation concealment due to lack of reporting may yield exaggerated estimates of treatment effect compared to trials with clearly adequate allocation concealment (Schulz, Chalmers Hayes et al., 1995).

Rationale for the Asthma-Specific Quality Indicators

To supplement the general study quality characteristics that have been validated in the literature, six asthma specific quality indicators were developed, which were based on the rationale described.


Power calculations

The reporting of power calculations performed a priori indicates that the researchers prospectively determined both the primary outcome(s) of interest and the magnitude of effect considered clinically meaningful for those outcomes. The clinical setting of asthma offers a variety of potential outcomes, and various ways of reporting these outcomes. The inclusion of formal power calculations performed a priori reduces the potential for selective reporting of outcomes.


Accounted for excluded patients

Adequate accounting for excluded patients allows a more complete determination of whether dropouts differ by treatment arm. If patients drop out from each arm for substantially different reasons, then the likelihood of withdrawal bias is increased. For example, it is possible for a much larger number of dropouts to occur in the control group compared to the treatment group due to lack of efficacy, especially when the control group receives a placebo or no treatment.


Established reversibility of airway obstruction

Establishing reversibility of airway obstruction is the standard approach to differentiating asthma from chronic obstructive pulmonary disease (COPD). Studies of adult patients that do not establish reversibility as an eligibility criteria for study entry may include a substantial proportion of patients with predominant COPD. The inclusion of such patients would confound study results, as they may respond to asthma treatment differently.


Controlled for other medication use

Patients who enter asthma clinical trials may be receiving a variety of medications in addition to the study medication. The impact of these other medications may confound study results by a direct effect on the signs and symptoms of asthma, or through interaction with the study medication.


Reported compliance

The rate of compliance in a clinical trial may affect the magnitude of effect observed. Also, the effectiveness of medications in clinical practice depends on the patient's compliance with treatment. Reporting the rates of compliance indicates whether the observed treatment effect may be biased by noncompliance, and how likely patients will comply with the treatment in the clinical setting.


Addressed seasonality

The clinical expression of asthma will often vary by season. If patients are enrolled over a period of time that spans several seasons, there is the possibility that seasonality will affect eligibility for the study, baseline lung function and symptom parameters, and outcome measurements. Thus, results observed may be confounded by seasonal effects, rather than measuring actual treatment effects.

Study Sponsorship

As a separate issue from assessment of study quality, the included trials were classified by source of research support. Using the acknowledgements of support or provision of study drug in published papers from each study and/or institutional affiliation of authors, the trials were categorized as having been funded by one of the following:

  • research grants from government or other nonprofit agencies only;
  • research grants from pharmaceutical manufacturers only;
  • supplies from pharmaceutical manufacturers only;
  • pharmaceutical manufacturers and nonprofit agency support; or
  • no sponsorship reported.


Meta-analyses of the following outcomes were conducted:

  1. Lung function outcomes: FEV1, PEF
  2. Puffs per day of short-acting beta-2 agonist

A minimum of three studies for each meta-analysis was required. Not all studies included in this systematic review reported all outcomes of interest, nor did all studies report each outcome in similar, combinable ways. Thus, not all included studies were used for each meta-analysis. However, there is no indication that outcomes not reported, or methods of reporting differed systematically among studies; thus, meta-analysis results should not be biased by selective reporting of outcomes among the included studies.

Expression of Outcomes

Lung Function Outcomes

FEV1 and PEF can either be reported as absolute measures (liters and liters per minute, respectively) or as a percentage of the predicted value for age, sex, height, and race based on published standards. FEV1 can be measured before or after administration of a bronchodilator. Such variability in reporting makes it difficult to directly combine reported results in a meta-analysis. A general method for combining studies with continuous outcome measures based on different scales is the method of effect sizes. Several authors have described this method (Cohen, 1977, Rosenthal and Rubin, 1979; Glass, 1980; Hedges, 1981; Rosenthal, 1984; Hedges and Olkin, 1985). The effect size of an experiment, d, is defined as:

d=(MT - Mc) / S, (1)

where Mt and Mc are the sample means of the treated and control arms respectively and S is the estimated standard deviation (SD) of the population. S could be the SD in the control arm, or it could be a pooled estimate.

Assuming a normal distribution for the individual observations with equal variances in each arm of the experiment, the pooled estimate of S is given by:

Image f3717_EQU002.jpg

where St2 is the sample variance of the treated arm, and Sc2 is the sample variance of the control arm. Effect sizes are in units of SD and do not express effect in the outcomes scale used in the study for clinical measurement. However, effect size can be converted to clinical units that indicate treatment effect by multiplying effect size by the pooled SD, S.

The variance of d is:

Image f3717_EQU003.jpg

where n1 and n2 are the sizes of the samples from the two subpopulations (Hedges and Olkin, 1985).

For lung function outcomes in each reporting study, the effect size was calculated for the response variable of difference from baseline in each study arm. In some studies, the information reported was insufficient for the direct calculation of effect size. For example, the SD of the difference value was not always available. However, where an appropriate test of significance was reported (e.g., analysis of variance), effect size was estimated using the published p-value and the difference from baseline (Rosenthal, 1994). When the p-value was reported as less than an upper limit, that upper limit was used to generate a conservative estimate of effect size (e.g., 0.001 when reported as <0.001). Upper limits as large as 0.05 were not used. When reported as within a range, the midpoint of the range was used (e.g., 0.025 for 0.05<p<0.01).

In some cases, the only available information was mean and SD or standard error of baseline and final values, with no p-value specified. If SDs or standard errors were not reported but were shown as error bars on graphs and could be reasonably estimated, these were included. For these studies, direct calculation of effect size would be inaccurate due to an overestimate of the variances, since the pre- and post-treatment values are related, but the correlation coefficient is unknown. Studies that reported sufficient data to calculate the effect size by pre- and post-treatment values as well as by other methods were used to estimate a correction factor. For studies that reported only pre- and post-treatment means and SDs or standard errors, the correction factor was applied to the SDs or standard errors before calculating the effect size (for details, see "Meta-Analysis Technical Supplement" at the end of this chapter).

In combining the calculated effect sizes, results were first calculated for subsets of studies reporting lung function outcomes in like units, i.e., liters (L) and percent predicted for FEV1, and liters per minute and percent predicted for PEF. Then effect sizes for all relevant studies were combined. (See Evidence and Meta-Analysis Tables 3-1 through 3-21.)

Pooled SD values were calculated from several studies for both liters and percent predicted for FEV1, and for liters per minute and percent predicted for PEF. These values were used to convert effect sizes to treatment effect values (difference in change from baseline between study arms) for both lung function outcomes (for details, see "Meta-Analysis Technical Supplement" at the end of this chapter).

Puffs Per Day of Short-Acting Beta Agonist

Most studies reporting short-acting beta-2 agonist use did so using units of puffs per day. These results were combined directly as differences in change from baseline between study arms, without converting to effect size.

Test for Homogeneity

For each meta-analysis, a test for homogeneity was carried out according to DerSimonian and Laird (1986).

Meta-Analysis Methods

Most meta-analyses are performed on a group of studies with a common endpoint. The assumption is often made that these studies all estimate the same parameter, such as an odds ratio, and the analysis is referred to as a fixed-effects analysis. The opposite of a fixed-effects model is a random effects model. The random-effects model produces estimates that are more conservative than fixed-effects models. The idea of a random-effects model is that the parameter sampled does not remain constant from study to study. Instead, it varies randomly, and is, in fact, a random variable sampled from some distribution. The problem then is to estimate the center of the distribution of the parameter of interest, and the variance of the distribution. This methodology is especially appropriate for studies of asthma therapy because of the differences in medication dose, disease severity, length of followup, and study quality in each study.

Random-effects models differ from fixed-effects models in that a measure, v, of the variation between studies is included in computation of the total uncertainty used to compute weights for each estimate. One conventional measure of v is:

Image f3717_EQU004.jpg

where X2 is the usual chi-squared measure of heterogeneity for the m studies and where wj = 1 / vj, and vj is the variance of the estimated odds ratio from study j. If the value of v is computed to be negative, it is usually set to zero. The random effects weighted mean odds ratio is:

Image f3717_EQU005.jpg

where Image f3717_THETA.jpg j is the estimated odds ratio from study j, and wj* = 1/[vj + v]. The variance of the weighted mean odds ratio in the random effects model is

Image f3717_EQU006.jpg

Since v is usually larger than zero, each wj* is usually larger than the corresponding fixed effects weight wj, and so the variance of the random effects weighted mean is usually larger than the variance of the fixed effects weighted mean. There are several methods for obtaining estimates of v, including some described by DerSimonian and Laird (1986) and Hedges and Olkin (1985). The method described by Hedges and Olkin (1985) is an empirical Bayes estimator, and is the one used in this analysis. This particular estimator works well for as few as two studies, and if the studies are homogeneous, the estimates approach those of the fixed-effects model. The calculations were carried out using the FAST*PRO software as described by Eddy and Hasselblad (1992).

Meta-Analysis Technical Supplement

Estimation of a Correction Factor for Studies that Reported Only Pre- and Post-Treatment Means and Standard Deviations or Standard Errors

In some cases, outcomes were reported only as mean and SD or standard error (SE) of baseline and final values, with no p-value specified. For these studies, direct calculation of effect size would be inaccurate due to an overestimate of the variances, since the pre- and post-treatment values are related, but the correlation coefficient is unknown. A correction factor for FEV1 pre- and post-treatment SDs or SEs was estimated as follows:

The true SE was estimated from Boyd (1995), in two ways. From the confidence interval of the treatment effect, SE = 0.080. From the F-value, SE = 0.090. Thus, on average, SE=0.085. The value calculated from pre- and post-treatment means and variances is 0.146. Thus SE or SD from pre- and post-treatment means and variances must be multiplied by 0.58 to get the correct value.

From the van der Molen, Postma, Turner et al., (1997), study, the true SE estimated from the confidence interval of the treatment effect, is 0.050. The value calculated from pre- and post-treatment means and variances is 0.113. Thus, SE or SD from pre- and post-treatment means must be multiplied by 0.44 to get the correct value.

The correction factor was averaged to approximately 0.5 and applied to the SDs or SEs of those studies for which effect size was calculated using pre- and post-treatment data (FitzGerald, Chapman, Della Cioppa et al., 1999; Boulet, Cartier, and Milot, 1998; Grutters, Brinkman, and Aslander, 1999; Li, Ward, Thien et al., 1999; McIvor, Pizzichini, Turner et al., 1998; Pauwels, Lofdahl, Postma et al., 1997; Bouros, Bachlitzanakis, Kottakis et al., 1999; Kips, O'Connor, Inman et al., 2000).

Similar calculations were carried out for PEF, using data from studies by Boyd (1995), and Bouros, Bachlitzanakis, Kottakis et al. (1999). Estimates of the correction factor were 0.37 and 0.26, respectively. A conservative SD/SE correction factor of 0.5 was applied to calculations of effect size from two small studies by Li, Ward, Thien et al. (1999) and McIvor, Pizzichini, Turner et al. (1998).

Calculation of Average Standard Deviation to Convert Effect Size to Clinically Meaningful Units

Effect sizes are unitless and not clinically meaningful. However, they can be converted to clinical units by multiplying by a pooled SD of the change from baseline parameter reported in the desired units. Pooled SD values were calculated from hseveral studies that reported applicable data for both liters and percent predicted for FEV1, and for liters per minute and percent predicted for PEF as follows:


  • PubReader
  • Print View
  • Cite this Page

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...