CRD summary

This review investigated potential sources of variation in efficacy estimates of influenza vaccine in healthy adults. The authors concluded that influenza vaccination significantly reduced influenza in healthy adults, but that estimates of efficacy varied according to the age of the participants and study quality. This was a well-conducted review and the authors' conclusions are likely to be robust.

Authors' objectives

To investigate potential sources of heterogeneity in efficacy estimates of influenza vaccination in healthy adults.

Searching

MEDLINE (1969 to 2002), EMBASE and the Cochrane Controlled Trials Register (no dates reported) were searched for eligible studies published in the English language before December 2002; the search terms were reported for the MEDLINE search. The reference lists of retrieved articles were also checked for additional relevant studies.

Study selection

Assessment of study quality

The studies were assessed for validity using two validated quality assessment scales: the Chalmers scale and the Jadad scale. The studies were also assessed for the following individual criteria: adequacy of reporting of randomisation, allocation concealment and double-blinding. Trials scoring more than 2 on the Jadad scale, or the median score or greater on the Chalmers scale, were classified as high quality. Two reviewers, blinded to authors, institution and journal of publication, independently assessed validity. The authors did not state how any disagreements were resolved.

Data extraction

The data extraction was carried out with each article blinded to authors, institution and journal of publication. The authors did not state how many reviewers performed the data extraction.

Data relating to confirmed cases of influenza in both vaccine and control groups were extracted in a contingency table format, and relative risks (RRs) with 95% confidence intervals (CIs) were calculated. When more than one definition was used in the same study for clinically confirmed cases of influenza, data using the definition most closely related to that of the Centers for Disease Control and Prevention were extracted. When more than one definition was used in the same study for laboratory confirmed cases of influenza, data related to the more sensitive definition (seroconversion) were used. Studies with more than two arms were split into sub-trials, and the placebo group was split equally between the different sub-trials generated.

Methods of synthesis

Results of the review

Twenty-six studies were included in the review, of which 12 were divided into sub-trials as they described the results of different trials and/or contained more than two treatment arms. Therefore, 53 trials (or sub-trials) were included, with a total of 46,581 participants. Forty-seven trials were classified as randomised controlled trials (RCTs) and 6 trials were classified as quasi-RCTs.

The mean quality assessment score was 0.39 out of a maximum possible score of one (range: 0.13 to 0.77) when using the Chalmers scale, and 2.52 out of a maximum possible score of 5 when using the Jadad scale. More recent studies scored higher for quality.

There was a statistically significant reduction in clinically confirmed cases of influenza with vaccination (49 trials; RR 0.78, 95% CI: 0.72, 0.84, P<0.0005; random-effects model). When using the fixed-effect model, the pooled RR was 0.83 (95% CI: 0.80, 0.86). There was evidence of statistically significant heterogeneity between studies (chi-squared 142.39, d.f.=48, P<0.0005).

There was a statistically significant reduction in laboratory confirmed cases of influenza with vaccination (25 trials; RR 0.37, 95% CI: 0.29, 0.47, P=0.001; random-effects model). When using the fixed-effect model, the pooled RR was 0.38 (95% CI: 0.33, 0.43). There was evidence of statistically significant heterogeneity between studies (chi-squared 52.19, d.f.=24, P=0.001).

Higher quality studies had a smaller effect size than lower quality studies. However, there was still a statistically significant reduction in both clinically confirmed (28 trials; RR 0.86, 95% CI: 0.80, 0.91) and laboratory confirmed (14 trials; RR 0.41, 95% CI: 0.31, 0.55) cases of influenza with vaccination in the higher quality trials (results presented according to Chalmers scale; the results were similar using the Jadad scale).

An aerosol inactivated vaccine was more effective at preventing clinically confirmed cases of influenza (6 trials; RR 0.45, 95% CI: 0.28, 0.73) than a parenteral inactivated vaccine (35 trials; RR 0.77, 95% CI: 0.70, 0.85) or a live aerosol vaccine (8 trials; RR 0.85, 95% CI: 0.77, 0.92).

Vaccination was more effective in patients aged younger than 33 years than in patients aged 33 and older. The RRs for clinically confirmed and laboratory confirmed cases were 0.54 (15 trials; 95% CI: 0.44, 0.67) and 0.22 (5 trials; 95% CI: 0.13, 0.37), respectively, for those younger than 33 years and 0.89 (23 trials; 95% CI: 0.85, 0.94) and 0.43 (16 trials; 95% CI: 0.33, 0.57) for those aged 33 and older.

Other covariates tested (placebo control versus B vaccine control, matching of vaccine and circulating strains, recommended vaccine strains used) had smaller effects on the results.

Statistically significant publication bias was present amongst trials that assessed clinically confirmed cases of influenza (P=0.002 adjusted rank correlation test; P<0.0005 regression asymmetry test), but not amongst trials that assessed laboratory confirmed cases of influenza (P=0.072 adjusted rank correlation test; P=0.083 regression asymmetry test).

Authors' conclusions

The review showed a statistically significant benefit of influenza vaccination in preventing clinically and laboratory confirmed cases of influenza, although there was statistically significant heterogeneity amongst the individual studies. The age of the participants and study quality, both inversely correlated with vaccine efficacy, were the only covariates found to contribute significantly to this heterogeneity. The authors warned that these findings should be interpreted with caution and considered as hypotheses to be evaluated in future high-quality studies, given the importance of a reliable estimate of influenza vaccination efficacy from a health policy point of view.

CRD commentary

The review question was clear in terms of the study designs, participants, interventions and outcomes of interest. The authors searched relevant electronic databases and the search terms were reported. However, there were no attempts to identify unpublished studies and only studies published in English were included, thus increasing the potential for language and publication bias. The authors found evidence of statistically significant publication bias for studies assessing clinically confirmed cases of influenza. Study quality was assessed using two validated quality assessment scales. Two reviewers independently assessed studies for inclusion and performed the validity assessment, thus minimising errors and reviewer bias.

Adequate details of the included studies were presented. Appropriate measures of effect were calculated, and the authors assessed statistical heterogeneity and investigated it when it was found. This was a well-conducted systematic review and the authors' conclusions are likely to be reliable.

Implications of the review for practice and research

Practice: The authors did not state any implications for practice.

Research: The authors stated that further high-quality clinical trials, designed to facilitate future pooled analyses, are required to provide definitive answers for policy-makers.

Funding

Italian Ministry of Health; Emilia Romagna Regional Health Agency.

Bibliographic details

Villari P, Manzoli L, Boccia A. Methodological quality of studies and patient age as major sources of variation in efficacy estimates of influenza vaccination in healthy adults: a meta-analysis. Vaccine 2004; 22(25-26): 3475-3486. [PubMed: 15308375]

Indexing Status

Subject indexing assigned by NLM

MeSH

Aging /immunology; Humans; Influenza Vaccines /immunology /therapeutic use; Influenza, Human /immunology /prevention & control; Publication Bias; Quality Assurance, Health Care; Randomized Controlled Trials as Topic /standards; Regression Analysis

AccessionNumber

12004009953

Database entry date

31/01/2006

Record Status

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.