NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet]. York (UK): Centre for Reviews and Dissemination (UK); 1995-.

Cover of Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews

Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet].

Show details

Methodological quality of studies and patient age as major sources of variation in efficacy estimates of influenza vaccination in healthy adults: a meta-analysis

, , and .

Review published: .

CRD summary

This review investigated potential sources of variation in efficacy estimates of influenza vaccine in healthy adults. The authors concluded that influenza vaccination significantly reduced influenza in healthy adults, but that estimates of efficacy varied according to the age of the participants and study quality. This was a well-conducted review and the authors' conclusions are likely to be robust.

Authors' objectives

To investigate potential sources of heterogeneity in efficacy estimates of influenza vaccination in healthy adults.

Searching

MEDLINE (1969 to 2002), EMBASE and the Cochrane Controlled Trials Register (no dates reported) were searched for eligible studies published in the English language before December 2002; the search terms were reported for the MEDLINE search. The reference lists of retrieved articles were also checked for additional relevant studies.

Study selection

Study designs of evaluations included in the review

Randomised controlled trials (RCTs) and quasi-randomised controlled trials were eligible for inclusion. Studies were defined as quasi-randomised when individuals were assigned to treatment groups using quasi-random methods, such as alternation or date of birth. Studies that stated that they were randomised were categorised accordingly, even if the method of randomisation was not adequately described.

Specific interventions included in the review

Studies that compared influenza vaccines with placebo or control vaccines were eligible for inclusion. Thirty-eight of the included studies evaluated the efficacy of the parenteral inactivated vaccine, nine assessed live aerosol vaccines, and six assessed aerosol inactivated vaccines. Thirty-six studies were placebo-controlled, 16 used the B vaccine as the control, and one trial used other vaccines as the control. The included trials were performed between 1965 and 1999.

Participants included in the review

Studies of humans, in which at least 70% of the participants were aged between 15 and 65 years and did not have medical conditions that would place them at high risk for complications of influenza, were eligible for inclusion. The majority of the included studies were performed in the USA, and the mean age of the participants ranged from 16 to 45 years.

Outcomes assessed in the review

Studies that reported enough data to estimate vaccine efficacy for the prevention of clinically and/or laboratory confirmed cases of naturally occurring influenza were eligible for inclusion. The majority of the included studies evaluated the efficacy of the vaccine for the prevention of clinically confirmed influenza.

How were decisions on the relevance of primary studies made?

Two reviewers independently assessed the studies against predefined inclusion criteria. The authors did not state how any disagreements were resolved.

Assessment of study quality

The studies were assessed for validity using two validated quality assessment scales: the Chalmers scale and the Jadad scale. The studies were also assessed for the following individual criteria: adequacy of reporting of randomisation, allocation concealment and double-blinding. Trials scoring more than 2 on the Jadad scale, or the median score or greater on the Chalmers scale, were classified as high quality. Two reviewers, blinded to authors, institution and journal of publication, independently assessed validity. The authors did not state how any disagreements were resolved.

Data extraction

The data extraction was carried out with each article blinded to authors, institution and journal of publication. The authors did not state how many reviewers performed the data extraction.

Data relating to confirmed cases of influenza in both vaccine and control groups were extracted in a contingency table format, and relative risks (RRs) with 95% confidence intervals (CIs) were calculated. When more than one definition was used in the same study for clinically confirmed cases of influenza, data using the definition most closely related to that of the Centers for Disease Control and Prevention were extracted. When more than one definition was used in the same study for laboratory confirmed cases of influenza, data related to the more sensitive definition (seroconversion) were used. Studies with more than two arms were split into sub-trials, and the placebo group was split equally between the different sub-trials generated.

Methods of synthesis

How were the studies combined?

A weighted average of the RRs was calculated using both random-effects (DerSimonian and Laird) and fixed-effect (Mantel-Haenszel) models. Separate meta-analyses were conducted for clinically confirmed and laboratory confirmed data.

Publication bias was assessed by a funnel plot, using an adjusted rank correlation test and a regression asymmetry test to assess statistical significance.

How were differences between studies investigated?

Heterogeneity was assessed using the chi-squared test. Meta-regression analyses were performed to examine the association between trial quality and other variables with the estimated effects of influenza vaccination. The association between date of publication and study quality was also examined.

Results of the review

Twenty-six studies were included in the review, of which 12 were divided into sub-trials as they described the results of different trials and/or contained more than two treatment arms. Therefore, 53 trials (or sub-trials) were included, with a total of 46,581 participants. Forty-seven trials were classified as randomised controlled trials (RCTs) and 6 trials were classified as quasi-RCTs.

The mean quality assessment score was 0.39 out of a maximum possible score of one (range: 0.13 to 0.77) when using the Chalmers scale, and 2.52 out of a maximum possible score of 5 when using the Jadad scale. More recent studies scored higher for quality.

There was a statistically significant reduction in clinically confirmed cases of influenza with vaccination (49 trials; RR 0.78, 95% CI: 0.72, 0.84, P<0.0005; random-effects model). When using the fixed-effect model, the pooled RR was 0.83 (95% CI: 0.80, 0.86). There was evidence of statistically significant heterogeneity between studies (chi-squared 142.39, d.f.=48, P<0.0005).

There was a statistically significant reduction in laboratory confirmed cases of influenza with vaccination (25 trials; RR 0.37, 95% CI: 0.29, 0.47, P=0.001; random-effects model). When using the fixed-effect model, the pooled RR was 0.38 (95% CI: 0.33, 0.43). There was evidence of statistically significant heterogeneity between studies (chi-squared 52.19, d.f.=24, P=0.001).

Higher quality studies had a smaller effect size than lower quality studies. However, there was still a statistically significant reduction in both clinically confirmed (28 trials; RR 0.86, 95% CI: 0.80, 0.91) and laboratory confirmed (14 trials; RR 0.41, 95% CI: 0.31, 0.55) cases of influenza with vaccination in the higher quality trials (results presented according to Chalmers scale; the results were similar using the Jadad scale).

An aerosol inactivated vaccine was more effective at preventing clinically confirmed cases of influenza (6 trials; RR 0.45, 95% CI: 0.28, 0.73) than a parenteral inactivated vaccine (35 trials; RR 0.77, 95% CI: 0.70, 0.85) or a live aerosol vaccine (8 trials; RR 0.85, 95% CI: 0.77, 0.92).

Vaccination was more effective in patients aged younger than 33 years than in patients aged 33 and older. The RRs for clinically confirmed and laboratory confirmed cases were 0.54 (15 trials; 95% CI: 0.44, 0.67) and 0.22 (5 trials; 95% CI: 0.13, 0.37), respectively, for those younger than 33 years and 0.89 (23 trials; 95% CI: 0.85, 0.94) and 0.43 (16 trials; 95% CI: 0.33, 0.57) for those aged 33 and older.

Other covariates tested (placebo control versus B vaccine control, matching of vaccine and circulating strains, recommended vaccine strains used) had smaller effects on the results.

Statistically significant publication bias was present amongst trials that assessed clinically confirmed cases of influenza (P=0.002 adjusted rank correlation test; P<0.0005 regression asymmetry test), but not amongst trials that assessed laboratory confirmed cases of influenza (P=0.072 adjusted rank correlation test; P=0.083 regression asymmetry test).

Authors' conclusions

The review showed a statistically significant benefit of influenza vaccination in preventing clinically and laboratory confirmed cases of influenza, although there was statistically significant heterogeneity amongst the individual studies. The age of the participants and study quality, both inversely correlated with vaccine efficacy, were the only covariates found to contribute significantly to this heterogeneity. The authors warned that these findings should be interpreted with caution and considered as hypotheses to be evaluated in future high-quality studies, given the importance of a reliable estimate of influenza vaccination efficacy from a health policy point of view.

CRD commentary

The review question was clear in terms of the study designs, participants, interventions and outcomes of interest. The authors searched relevant electronic databases and the search terms were reported. However, there were no attempts to identify unpublished studies and only studies published in English were included, thus increasing the potential for language and publication bias. The authors found evidence of statistically significant publication bias for studies assessing clinically confirmed cases of influenza. Study quality was assessed using two validated quality assessment scales. Two reviewers independently assessed studies for inclusion and performed the validity assessment, thus minimising errors and reviewer bias.

Adequate details of the included studies were presented. Appropriate measures of effect were calculated, and the authors assessed statistical heterogeneity and investigated it when it was found. This was a well-conducted systematic review and the authors' conclusions are likely to be reliable.

Implications of the review for practice and research

Practice: The authors did not state any implications for practice.

Research: The authors stated that further high-quality clinical trials, designed to facilitate future pooled analyses, are required to provide definitive answers for policy-makers.

Funding

Italian Ministry of Health; Emilia Romagna Regional Health Agency.

Bibliographic details

Villari P, Manzoli L, Boccia A. Methodological quality of studies and patient age as major sources of variation in efficacy estimates of influenza vaccination in healthy adults: a meta-analysis. Vaccine 2004; 22(25-26): 3475-3486. [PubMed: 15308375]

Indexing Status

Subject indexing assigned by NLM

MeSH

Aging /immunology; Humans; Influenza Vaccines /immunology /therapeutic use; Influenza, Human /immunology /prevention & control; Publication Bias; Quality Assurance, Health Care; Randomized Controlled Trials as Topic /standards; Regression Analysis

AccessionNumber

12004009953

Database entry date

31/01/2006

Record Status

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.

Copyright ┬ę 2014 University of York.
Bookshelf ID: NBK71195

Views

  • PubReader
  • Print View
  • Cite this Page

Similar articles in PubMed

See reviews...See all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...