Public health
Influenza vaccination: policy versus evidence
Associated Data
Short abstract
Each year enormous effort goes into producing influenza vaccines for that specific year and delivering them to appropriate sections of the population. Is this effort justified?
Viral infections of the respiratory tract impose a high burden on society. In the last half of the 20th century, efforts to prevent or minimise their impact centred on the use of influenza vaccines. Each year enormous effort goes into producing that year's vaccine and delivering it to appropriate sections of the population. Here, I will discuss policies on the use of inactivated vaccines for seasonal influenza; the evidence for their efficacy, effectiveness, and safety (“effects”); and possible reasons for the gap between policy and evidence.
Policies
Every vaccination campaign has stated aims against which its effects must be measured. The US Advisory Committee on Immunisation Practices produces a regularly updated rationale for vaccination against influenza.1 The current version identifies 11 categories of patients at high risk of complications from influenza (box).
The rationale rests on the heavy burden that influenza imposes on the population and the benefits of vaccination. For example, reductions in cases, admissions to hospital, mortality of elderly people in families with children, contacts with healthcare professionals, antibiotic prescriptions, and absenteeism for children and household contacts are the main arguments for extending vaccination to healthy children aged 6-23 months in the United States.2 Canada introduced a similar policy in 2004.3 Less comprehensive policies recommending vaccination for all people aged 60 or 65 and over are in place in 40 of 51 developed or rapidly developing countries.4 On the basis of single studies, the World Health Organization estimates that “vaccination of the elderly reduces the risk of serious complications or of death by 70-85%.”5 Given the global nature of these recommendations, what type of evidence should we expect to support them and what does available evidence tell us?4
Which evidence?
When considering the best evidence for vaccination we must take into account the unique epidemiological features of influenza viruses and the rationale for immunisation. The incidence and circulation of seasonal influenza and other respiratory viruses vary greatly each year, each season, and even in each setting. A systematic review of the incidence of influenza in people up to 19 years' old reported a seasonal variability of 0-46%; during a five year period the average incidence was 4.6% in this age group. During a period of 25 years the incidence was 9.5% in children under 5.6 Because of this variability and lack of carryover protection from one year's vaccine to the next,7 especially if the virus changes its antigenic configuration, single studies reporting data from one or two seasons are difficult to interpret. Single studies are also not reliable sources for generalising and forecasting the effects of vaccines, especially when numbers are small. They introduce further instability into already problematic forecasting. Additional limitations to our forecasting ability are imposed by our use (and misuse) of studies assessing the effects of influenza vaccines. Although the effect assessed depends on the aims of the particular campaign, most concentrate on serious effects (such as pneumonia or death) and person to person transmission (table 1). Field efficacy studies are only relevant when viral circulation is high, but no one can forecast with precision the impact on next year's influenza.influenza.
Table 1
Effects of inactivated influenza vaccines and preferred designs of primary studies to assess them
| Effect | Definition | Preferred study design | Relevance for public health |
|---|---|---|---|
| Efficacy | Capacity of the vaccine to induce antibody responses (immunogenicity) to influenza viruses | Placebo controlled RCT | Important for the yearly registration of new vaccines containing the forthcoming “season's” viral antigens. Immunogenicity is the only way of testing the likely efficacy of the candidate vaccine in the absence of viral circulation |
| Field efficacy | Capacity of the vaccine to prevent influenza A or B virus and its complications | Placebo controlled RCT | High, if viral circulation is high (as in an epidemic or a pandemic). Studies assessing field efficacy are usually well resourced with reliable and quick virological feedback and cases of influenza are recognised as such. Estimates of efficacy cannot be generalised to seasons with low circulation of the influenza virus even if other respiratory viruses have a higher circulation |
| Effectiveness | Capacity of the vaccines to prevent influenza-like illness and its consequences | Placebo controlled RCT | High in conditions of good match between vaccine and viral antigen and high viral circulation. Higher if effects on major outcomes are reported |
| Harms | A harmful event potentially associated with exposure to influenza vaccines | Placebo controlled RCT or non-randomised, comparative study | Depends on incidence, latency, and type of harm |
RCT, randomised controlled trial.
Studies of the effects on influenza-like illness and its complications most closely replicate real life conditions because no one knows what agent (if any) causes this disease. Influenza-like illness is an acute respiratory disease caused by many different viruses (including influenza A and B), which presents with symptoms and signs that cannot be distinguished from those of influenza. Influenza-like illness does not have documented laboratory isolation of the causative agent and is the syndrome that most commonly presents to doctors (“the flu”).
In general the most powerful and reliable studies are those that “average” out several years and perform subanalyses by setting, population, viral circulation, and viral-vaccine antigenic match—variables that affect interpretation of the effects of a vaccine. Systematic reviews are the best way to perform such analyses, and provide powerful evidence weighted by the methodological quality of the studies involved. Large datasets containing several decades of observations help us to assess the performance of vaccines more accurately.
The evidence
I searched for relevant systematic reviews when updating and expanding the Clinical Evidence chapter on influenza (see bmj.com)—evidence was plentiful. The examples in table 2 show the strength of the evidence and the contradictions in relation to the stated aims of the vaccination campaign. Whenever possible, I chose evidence gathered in the optimal circumstances (for inactivated vaccines)—high viral circulation and a good match between the viral antigen and the vaccine.
Table 2
Examples of evidence from systematic reviews comparing inactivated influenza vaccines with placebo or no intervention
| Population | Study design included in review | Outcome | No of participants | Vaccine field efficacy or effectiveness* |
|---|---|---|---|---|
| Children aged up to 23 months8
| RCT†
| Influenza
| 786
| 0.55 (0.18 to 1.69)
|
| Influenza-like illness
| —
| No data
| ||
| Children 6 years or more8
| RCT†
| Influenza
| 710
| 69%; 0.31 (0.22 to 0.45)
|
| Influenza-like illness
| 18 912
| 28%; 0.72 (0.66 to 0.78)
| ||
| Children up to 16 years8
| RCT†
| Transmission
| 123
| 1.68 (0.56 to 4.99)
|
| School absence
| 254
| 0.46 (0.17 to 1.22)
| ||
| Lower respiratory tract infection
| 136
| 0.30 (0.01 to 6.17)
| ||
| Admission to hospital
| 765
| 1.41 (0.62 to 3.24)
| ||
| Death
| —
| No data
| ||
| Healthy adults9
| RCT†
| Influenza
| 2411
| 67%; 0.33 (0.22 to 0.49)
|
| Influenza-like illness
| 5579
| 22%; 0.78 (0.67 to 0.91)
| ||
| Admission to hospital
| 5261
| Relative risk fixed effects model 0.65 (0.34 to 1.22)
| ||
| Working days lost
| 5572
| Weighted mean difference random effects model −0.12 (−0.24 to 0.00)
| ||
| Healthcare workers (to protect elderly patients in their care)10
| Cluster RCT and cohort
| Influenza
| 752
| 0.87 (0.46 to 1.63)
|
| Lower respiratory tract infection
| 1059
| 0.70 (0.41 to 1.20)
| ||
| Death from pneumonia
| 1059
| 39%; 0.61 (0.38 to 0.98)
| ||
| All-cause mortality
| 2496
| 40%; 0.60 (0.50 to 0.73)
| ||
| Elderly people in the community; high circulation of influenza virus and good vaccine-antigen matching11
| Cohort
| Influenza-like illness
| —
| No data
|
| Influenza
| 427
| 95%; 0.05 (0.01 to 0.37)
| ||
| Pneumonia
| —
| No data
| ||
| Death from influenza and pneumonia
| 163 391
| 0.87 (0.70 to 1.09)
| ||
| All-cause mortality (not adjusted for confounding)
| 300 332
| 41%; 0.59 (0.50 to 0.70)
| ||
| All-cause mortality (adjusted for confounding)
| 742 575
| 47%; odds ratio random effects model 0.53 (0.46 to 0.61)
| ||
| Elderly people in institutions; high circulation of influenza virus and good vaccine-antigen matching11
| Cohort
| Influenza-like illness
| 5963
| 23%; 0.77 (0.64 to 0.94)
|
| Influenza
| 658
| 1.04 (0.43 to 2.51)
| ||
| Pneumonia
| 4482
| 46%; 0.54 (0.42 to 0.70)
| ||
| Death from influenza and pneumonia
| 6127
| 42%; 0.58 (0.41 to 0.83)
| ||
| All-cause mortality
| 305
| 60%; 0.40 (0.21 to 0.77)
| ||
| Patients with asthma12 | RCT† | Influenza related exacerbation (early) | 696 | Risk difference fixed effects model 0.01 (−0.02 to 0.04) |
| Patients with chronic obstructive pulmonary disease13 | RCT† | Exacerbations (total number) | 180 | Weighted mean difference −0.37 (−0.64 to −0.11), P=0.006 |
RCT, randomised controlled trial.
Three problems are immediately apparent. The first is heavy reliance on non-randomised studies (chiefly cohort studies), especially in the elderly. This makes assessment of methodological quality an important part of data interpretation. For example, of 40 datasets assessing the effects of influenza vaccines in elderly people in institutions, only 26 reported data on viral types in circulation and only 21 gave information on vaccine content. Insufficient data were available in 11 of 17 retrospective studies of elderly people in institutions to allow reviewers to assess the authors' claim of “high” or “epidemic” viral circulation.11,14 A metaanalysis of inactivated vaccines in elderly people showed a gradient from no effect against influenza or influenza-like illness to a large effect (up to 60%) in preventing all-cause mortality. These findings are both counterintuitive and implausible, as other causes of death are far more prevalent in elderly people even in the winter months.15,16 It is impossible for a vaccine that does not prevent influenza to prevent its complications, including admission to hospital.
A more likely explanation for such a finding is selection bias, where one half of the study population (hemi-cohort) systematically differs from the other in one or more key characteristics.14-16 In this case, the vaccinated hemi-cohort may have been more mobile, healthy, and wealthy than the control hemi-cohort, thus explaining the differences in all-cause mortality.11,14 The same effect is seen in stronger study designs (such as cluster randomised trials) that are badly executed, which introduces bias.10 Its presence seems to be a marker of confounders that persist even after adjusting for known ones, and it makes accurate interpretation of the data difficult. Caution in interpretation should thus be the rule, not the exception. This problem (in the opposite direction—with frailer people more likely to be vaccinated) has been identified before but not heeded.17 The only way that all known and unknown confounders can be adequately controlled for is by randomisation.
The influence of poor study quality is also seen in the outcome of a review of evidence supporting the vaccination of all children to minimise transmission to family contacts.18 Five randomised studies and five non-randomised studies were reviewed, but although data were suggestive of protection, its extent was impossible to measure because of the weak methods used in the primary studies.18
The second problem is either the absence of evidence or the absence of convincing evidence on most of the effects at the centre of campaign objectives (table 2). In children under 2 years inactivated vaccines had the same field efficacy as placebo,8 and in healthy people under 65 vaccination did not affect hospital stay, time off work, or death from influenza and its complications.9 Reviews found no evidence of an effect in patients with asthma or cystic fibrosis, but inactivated vaccines reduced the incidence of exacerbations after three to four weeks by 39% in those with chronic obstructive pulmonary disease.12,13,19 All reviewers reported small data sets (such as 180 people with chronic obstructive pulmonary disease13), which may explain the lack of demonstrable effect.
The third problem is the small and heterogeneous dataset on the safety of inactivated vaccines, which is surprising given their longstanding and widespread use. A Cochrane Database Systematic Review found only one old trial with data from 35 participants aged 12-28 months.8 In the general population of elderly people, despite a dataset of several million observations, safety was only reported in five randomised controlled trials (2963 observations in total) on local and systemic adverse events seen within a week of giving parenteral inactivated vaccine.11 Although there appears to be no evidence that annual revaccination is harmful, such a lack of knowledge is surprising.
Gap between policy and evidence
The large gap between policy and what the data tell us (when rigorously assembled and evaluated) is surprising. The reasons for this situation are not clear and may be complex. The starting point is the potential confusion between influenza and influenza-like illness, when any case of illness resembling influenza is seen as real influenza, especially during peak periods of activity. Some surveillance systems report cases of influenza-like illness as influenza without further explanation. This confusion leads to a gross overestimation of the impact of influenza, unrealistic expectations of the performance of vaccines, and spurious certainty of our ability to predict viral circulation and impact. The consequences are seen in the impractical advice given by public bodies on thresholds of the incidence of influenza-like illness at which influenza specific interventions (antivirals) should be used.20
The confusion between influenza and influenza-like illness is compounded by the lack of accurate and fast surveillance systems that can tell what viruses are circulating in a setting or community within a short time frame, and after the “season” is finished give an accurate picture of what went on to enable better forecasting of future trends.21 Accurate surveillance must be based on a properly worked out sampling system for cases of influenza-like illness that meet set criteria, with accurate and quick feedback of a presumptive microbiological diagnosis. Without this, we cannot generalise from random sampling.
Another reason may be “availability creep.” In their efforts to deal with, or be seen to deal with, policy makers favour intervention with what is available—registered influenza vaccines. A similar philosophy is the “we have to make decisions and cannot wait to have perfect data” approach. This attitude may have an altruistic basis but has two important consequences. Firstly, it uses up resources that could be invested in a proper evaluation of influenza vaccines or on other health interventions of proven effectiveness. Secondly, the inception of a vaccination campaign seems to preclude the assessment of a vaccine through placebo controlled randomised trials on ethical grounds. Far from being unethical, however, such trials are desperately needed and we should invest in them without delay. A further consequence is reliance on non-randomised studies once the campaign is under way. It is debatable whether these can contribute to our understanding of the effectiveness of vaccines. Ultimately non-randomised designs cannot answer questions on the effects of influenza vaccines.
The optimistic and confident tone of some predictions of viral circulation and of the impact of inactivated vaccines, which are at odds with the evidence, is striking. The reasons are probably complex and may involve “a messy blend of truth conflicts and conflicts of interest making it difficult to separate factual disputes from value disputes”22 or a manifestation of optimism bias (an unwarranted belief in the efficacy of interventions).23
Whatever the reasons, it is a sobering thought that Archie Cochrane's 1972 statement that we should use what has been tested and found to reach its objectives is as revolutionary now as it was then.
Notes
Details of the search strategy are on bmj.com
Contributor: TJ designed and wrote the paper and is the sole contributor and guarantor.
Competing interests: TJ owned shares in Glaxo SmithKline and received consultancy fees from Sanofi-Synthelabo (2002) and Roche (1997-9).
Facebook
Twitter
Google+