Reconstructing the epidemic

The PR and MR models are both sufficiently flexible to be able to reproduce the two epidemic waves of the 2009 pandemic. The estimated incidence curves are reproduced in Figure 3. The estimated epidemic in the North is consistent across both models. London and the West Midlands are characterised by bigger first waves of infection (and subsequently smaller second waves) under the PR model, the opposite being true for the South. This is apparent from the peaks in Figure 3 and the given population-level attack rates in Table 3 (age-specific attack rates can be found in Appendix 3). Peak timings in both waves of infection are the same under both modelling approaches, and coincide with the start of school holidays, with the exception of the second wave in the West Midlands. Here, a sufficient supply of susceptible individuals remains in the population after the holiday to allow transmission to increase once more (albeit briefly). This may well, however, be a phenomenon of different school term dates in this region to those that predominate elsewhere in the country.

FIGURE 3

Estimated weekly number of new A/H1N1pdm infections for (a) London under the PR model; (b) London under the MR model; (c) the West Midlands under the PR model; (d) the West Midlands under the MR model; (e) the North under the PR model; (f) the North under (more...)

TABLE 3

Posterior median (and 95% credible interval) for cumulative incidence of infection, number of cases (both given in thousands) and attack rates, by region and by pandemic wave (May–August or September–December)

Estimated epidemic characteristics

Table 4 presents estimates of some key transmission parameters under both models. There is a pleasing consistency across the modelling approaches in the parameter estimates. For example, estimates for the (initial) reproductive number $R_0^{\text{init}}$, derived from the exponential growth rates, are centred on 1.8, with the region-specific estimates of the PR model being tightly distributed around this value. This is in broad agreement with other estimates for R₀ obtained from a review of 2009 pandemic transmission parameters,⁵⁰ and a slight increase on what had been estimated for the single region version of the model.¹⁵ In a similar (single-region) modelling study, much higher estimates for the R₀ associated with the A/H1N1pdm virus have been derived, although this was over the course of a later third wave of pandemic infection occurring in the winter season 2010–11.⁵¹ Similarly, the estimates for the other transmission parameters are robust to the model specification [note the overlapping nature of the credible intervals (CrIs) in Table 4]. In particular, parameter m₁, which gives the down-weighting applied to all contacts involving adults, is estimated consistently to be in the range 0.57–0.62. Estimates for m₃ indicate that the summer school holiday period reduced the rate of effective infectious contacts among the 5–14 years age group to below 3% of the school term-time figure. However, when averaged over all age groups, this represents a drop in $R_0^{\text{init}}$ of between 43% (in London) and 50% (in the South). To compare, a Canadian study recorded a 28% drop in transmissibility during a similar school holiday period.⁵² The reduction in the effective contact rates in the other school holidays, as measured by parameters m₄ and m₅ were neither as well estimated (note the width of the CrI attached to the estimates for parameter m₄) nor did they indicate a similar reduction in the contact rates, the shorter duration of these holidays evidently causing a milder disruption to routine contact patterns. Estimates for the proportion symptomatic, ϕ, do appear to be rather low, although consistent across approaches and with an estimate of 11% based on a closely observed outbreak.⁵³

TABLE 4

Posterior median (and 95% credible interval) for key parameters by modelling approach. Estimates of the reproductive number (R₀) from the PR model are 1.79 (1.74, 1.83), 1.80 (1.76, 1.85), 1.82 (1.78, 1.87), 1.77 (1.73, 1.80) for London, West Midlands, (more...)

Comparison between meta-region and parallel-region modelling

In Table 5, the posterior mean deviance is used to discriminate between different formulations of the MR model (to be discussed further in Goodness of fit), comparing each formulation relative to the comparable PR model. In fitting these models, MCMC provides a sample of parameter values {θ⁽¹⁾, . . . , θ⁽ⁿ⁾}, from which the posterior mean deviance, $D_m=-(2/n){\displaystyle {\sum }_{j=1}^n\text{log}\left(L_m({\mathit{y}}_{1:K};{\mathit{\theta }}^{(j)})\right)}$ can be derived, where L_m(·;·) indicates the likelihood under a specific model, m. Note that lower values of D_m are preferred. The discrepancy between the PR model and the best-performing MR model is 57.89. Owing to the regional variation permitted by the PR model in the estimation of $R_0^{\text{init}}$ and I₀, the PR model has six more parameters than the MR model. This improvement in deviance for such a small number of parameters suggests that the PR represents a significantly better fit to the data. This compounds the practical benefit of the PR model being markedly faster to implement; it is more suited to parallel computation and the calculation of $R_0^{*}$ in Equation 20 of Appendix 1, requires the calculation of eigenvalues of (7 × 7) matrices rather than the (28 × 28) or (44 × 44) matrices required by the MR model.

TABLE 5

Posterior mean (and standard deviation) deviances for some candidate parameterisations of the MR model, expressed as a discrepancy from the deviance of the PR model. The smaller values of the posterior mean deviance represent models providing a better (more...)

Finding an optimal parameterisation

Inferences drawn from either the PR or the MR modelling approach are found to be sensitive to the precise form of the regression for the background rates of GP consultation. Because of this, it was important to implement submodels of Equations 9 and 10 in order to most appropriately characterise the changes in consultation behaviour over the pandemic period. Again, the posterior mean deviance was used to identify a preferred model. The real-time PR model was repeatedly implemented with the higher-order interactions systematically removed from Equations 9 and 10 in the hope of finding simplified regression models without incurring any significant loss of fit to the data. Additionally, some age groups and regions were paired together to cover gaps where data were too sparse to warrant the additional age/region effects. Under the PR model, the seemingly optimal choice for the regression model, and the one that has been used in the generation of all the results presented in this section, is:

with the rates in the North found to be equal to those in the South, B_r,a(t_k) = B_s,a(t_k). This is unsurprising given the sparsity of virological data in the North to accurately estimate the non-pandemic consultation rates. Also, the rates in the two youngest age groups have been set to be equal (note the sum over the a index omits a = 1), B_r,1(t_k) = B_r,2(t_k), which, again, is not an unreasonable finding given that only the virological swabbing and not the QSurveillance GP data sets provide data with sufficient granularity to distinguish between the first two age groups (< 1 year and 1–4 years).

When the same model refinement process was undertaken using the MR model, the same regression equations were again preferred.

Having established the form of the regression equation for the background consultation rates, the next stage of model building in the MR approach was to consider the alternative model formulations of Chapter 3, The meta-region model, governing how the model handles density dependence, random commuting and the choice of the initial seeding. Examination of the posterior mean deviances presented in Table 5 shows that density dependence is best accounted for by scaling entries of the contact matrices by the population of the region, not the population of the relevant stratum, that is, by replacing N_r,a and N_v,a in Appendix 1, Equation 27 with N_r and N_v, the sum of the regional populations over age groups. Furthermore, it was found that within-region transmission that is density dependent (corresponding to the case α = 1 in Equation 27 of Appendix 1) gave better model fit than either frequency-dependent transmission (α = 0) or a mixture of the two (α = 0.5).

Meta-region model performance is highly sensitive to the choice of initial seeding of infectivity, with the hybrid seed performing most strongly. When the number of strata is expanded to partition between non-commuting and commuting adults, there was no consistent improvement in model performance (nor any particular worsening). However, the extra complexity and computation required to evaluate the model with the expanded number of strata indicates that the reduced stratification would be preferred in a real-time context. This suggests that the effects of inter-region transmission are either highly transient, sufficiently so that its effects are swallowed up by the choice of seeding, or the movement of individuals between regions is poorly characterised by the commuting data. The formulation of the contact matrices assumes that infected individuals move as freely as uninfected individuals, and this may well be unrealistic. However, accounting for this would further reduce any difference in model performance between the two approaches, and thus would not lead to any material adjustment of the conclusions.

All results presented that quote the MR model will refer to the best-performing variant with α = 1, density dependence governed by the regional population size, using the hybrid seed and assuming commuting at random.

Goodness of fit

Appendices 3 and 4 give goodness-of-fit plots for three of the data types (GP consultation data, virological positivity data and serological data) under the PR and MR models respectively. There is no apparent lack of fit under either model, with most data points lying within the 95% predictive intervals. In a couple of instances the seropositivity predicted by the model is too high (Greater London, ≥ 65) and in others too low (Greater London 5–14). It would seem that this is due to poor estimation of the initial proportion of susceptible individuals (this is an a priori estimation, it was not carried out as part of the real-time model effort). Even in these cases, the PR model gives the better fit to these outlying data points, with fewer points missing their predictive intervals. Elsewhere, the performance of the PR model is evidently superior, in accordance with the findings already presented.

Simulated data

It was decided to copy many of the features of the 2009 pandemic. The simulated outbreak starts with an initial burst of infections in the spring, so that the epidemic is in full exponential growth by the time of an over-summer school holiday. The school holiday acts as a break on transmission, partitioning the outbreak into two distinct waves of infection. Although we only consider this one underlying epidemic, we consider two different data scenarios. In the first scenario, it is assumed that there is direct information on confirmed cases, such as might occur in the surveillance of severe disease (e.g. hospitalisation or ICU data from USISS). In the second scenario, ILI consultations, contaminated by non-pandemic infections replace the confirmed case data. Both data streams are assumed to exist alongside serological data. In the second scenario, it is necessary to have companion virological swabbing data to identify the degree of contamination in the ILI data.

To ensure that the task of epidemic tracking is realistic, in the same vein as the NPFS introduction in 2009, it is assumed that there is a ‘shock’ in the data provided by the surveillance schemes. Such a shock would be provided by a public health intervention designed to alleviate overcrowding in primary health-care services or to reduce the demand on hospital beds. The net result of the intervention is that a much reduced proportion of cases report their symptoms to the respective surveillance schemes, with the timing of the intervention, as in 2009, following shortly on from the over-summer closure of schools for the holiday period.

To illustrate the size of the system shock that is being considered, the synthetic data to be used in the second scenario are presented in Figure 4.

FIGURE 4

Synthetic data generated to test the SMC algorithms. Top row: (a) observed number of GP consultations; and (b) swab positivity data with numbers representing the size of the weekly denominator. Bottom row: (c) serological data; and (d) the pattern of (more...)

Scenario 1: a naive algorithm

Here, we want to compare the relative performance of the SMC algorithm against what we consider to be a gold standard, the MCMC algorithm that was used to derive epidemic inference in earlier analysis of the 2009 data.¹⁵

To attempt this, MCMC analyses are carried out after 50, 70, 83, 120, 164 and 245 days of data have been observed. The SMC algorithm was then applied starting from the MCMC-derived posterior from 50 days, to see if, after the addition of 20 consecutive days, or batches, of data, the SMC and MCMC derived posteriors are statistically similar. This process was repeated to see if SMC could also bridge the gap between the MCMC analyses at days 70 and 83, days 83 and 120, days 120 and 164, and days 164 and 245.

Initially, a fast, naive SMC algorithm was tried in application to the first data scenario, where we have contamination-free hospitalisation data. Here, the MCMC step embedded within the SMC algorithm would only last for one iteration, and rejuvenation of the particle set would only take place after the assimilation of whole batches of data (none of the fractional addition of data discussed in the When to rejuvenate? When are the particles degenerate? discussion in Chapter 3, Sequential Monte Carlo). Figure 5 shows some of the results. In the scatter plots, the light green points show the starting MCMC-obtained distribution. Against this, left-hand scatterplots are to be compared with the right-hand scatterplots. On the right-hand side, all scattered points (except the light-green points) are of the same colour, because they are all of equal weight and of equal importance. In the left-hand column, the darker the point, the greater weight it carries.

FIGURE 5

Comparison of naive SMC (left-hand-side panels) and MCMC (right-hand-side panels) at (a) t_k = 70 days, (b) t_k = 120 days, (c) t_k = 245 days, via scatterplots for the parameters ψ and υ.

The immediate point to notice is that the SMC-obtained posteriors in the top and bottom panels would appear to be comparable to the MCMC-obtained posterior distributions, but the posterior distribution obtained by the naive SMC algorithm at time t_k = 120 displays significant degeneracy. The algorithm has not tracked the movement of the posterior density over the interval from 83 days to 120 days. It is this sample impoverishment that makes the naive SMC inefficient at such a time.

Referring to the plots of the simulated data in Figure 4, the superimposed vertical green arrows identify points in time where there are particularly informative observations. Immediately after time t_k = 83, there is a shock to the surveillance system as a public health intervention diverts people away from GP surgeries, drastically cutting the proportion of infections that are reported in the data. At time t_k = 110, there is a particularly large batch of serological data that are particularly informative. These two occurrences make it particularly hard for the naive algorithm to track the epidemic. Particularly after the t_k = 83 days shock, a number of new parameters become active (i.e. begin to have influence over the likelihood). As parameters first begin to move away from their prior density, it can cause severe depletion of the particle set. In comparison, the 50–70 day and the 164–245 day intervals are relatively uneventful and much easier for the algorithm to track.

It is this phenomenon that motivates a focus on the day 83 to day 120 interval moving forward, and also motivated the algorithmic adaptations discussed at the end of Chapter 3, section Sequential Monte Carlo. We also consider the second data scenario, where syndromic GP counts, inclusive of non-pandemic noise and virological swabbing data are available.

Scenario 2: heavy-duty sequential Monte Carlo

If the MCMC-derived posteriors are to be treated as a (albeit computationally costly) gold-standard, a measure of similarity is needed between the SMC- and the MCMC-derived distributions, and for this we use Küllback–Leibler (KL) divergence.⁵⁴ This is a statistic that gives a measure of how different an estimate for a probability distribution is to its ‘true’ target. Here, we are presuming that it is the MCMC that represents the truth.

Table 6 gives the KL statistics achieved when the full SMC algorithm was used over the day 83 to day 120 interval, breaking the interval down even further so we can look at KL discrepancies in the immediate aftermath of the shock at t_k = 83 days. The table also shows three different levels of ICC threshold used as a stopping criterion for the MCMC phase. At each time, the ‘gold standard’ MCMC analysis was repeated numerous times and then referred back to a reference analysis. This was done to build up a distribution of KL statistics, so that the SMC analysis could be given a KL ‘target’, the upper 95 percentile of the KL statistics calculated on the sample of MCMC analyses. If the SMC analysis had a KL statistic that was lower than the target value, then it could be said to be indistinguishable from the MCMC analyses. At this point, we know that the SMC algorithm is responding adequately.

TABLE 6

Performance of the adapted SMC algorithm over the interval 83–120 days by ICC threshold

From Table 6 it can be seen that, from about day 87 onwards, the SMC algorithm (for the ICC thresholds 0.1 and 0.2) begins to regularly hit its KL target. The problem, therefore, lies in the days immediately preceding this point in time, where, not only are the KL statistics large, but the MCMC-component of the SMC algorithm requires vast numbers of iterations and becoming prohibitively time-consuming.

Addressing the speed issue first, it was found that one particular parameter was causing the slow convergence. As discussed elsewhere,⁴⁷ it was found that, if proposals for the overdispersion for the negative binomial data η were made separately to the rest of the parameter vector θ (which is updated together in one block), convergence could be achieved much more rapidly. Figure 6 shows the improvement in the number of iterations required per day from over 400 per particle under the original SMC algorithm to 70 under the tailored version on day 89, with this improved performance evident for a number of days in the aftermath of day 83. Under both schemes, rejuvenations are required at the same times, but do not require the same computational effort. Both versions of the algorithm perform similarly from around day 90–91 onwards.

FIGURE 6

Computational demands of the SMC algorithm. (a) Number of MH steps required by the continuous-time SMC algorithm per rejuvenation against the timing of the rejuvenation for both the continuous-time algorithms (black and green correspond to with and without (more...)

This simple tailoring of the algorithm, only really necessary while a parameter is only weakly informed by the data and has an uninformative prior attached to it, evidently speeds up the algorithm, but it is necessary to show that this causes no degradation in terms of the inference that can be gathered. Table 7 repeats the exercise of Table 6, showing performance that is almost identical over the period 84–87 days (inclusive), the period over which there is a substantial speed-up in the implementation of the algorithm. Thereafter, for the thresholds 0.2 and 0.1 the performance is comparable to MCMC, with the exception of what appears to be an anomalous reading for the 0.1 threshold at 90 days.

TABLE 7

Performance of the tailored SMC algorithm over the interval 83–120 days by ICC threshold

However, the failure of the algorithm to hit the MCMC thresholds over the interval 85–87 days (and 88, 89 days also, not shown) is a concern, and motivates an examination of scatterplots akin to those of Figure 5. In Figure 7 we look at scatterplots of the MCMC- and SMC-derived posterior distributions for two regression parameter components of the non-pandemic ILI consultations, β_B. The MCMC plots are comparable to the SMC-derived plots immediately to their left. What they show is that, most strikingly on the 84–85 day and the 85–86 day steps, the MCMC algorithm is not capable of the same coverage of the space that the SMC algorithm achieves. This is a result of poor convergence of the MCMC algorithm. It gets stuck within a smaller range of values. It may be that the MCMC algorithm needs many more iterations to properly sample the full range of values, but it is already at a considerable computational disadvantage (typical runs are of chains of length 750,000 iterations).

FIGURE 7

The evolution over time of the marginal joint posterior for two components of the parameter vector β^B, comparing SMC-obtained (left-hand-side panels) and MCMC-obtained (right-hand-side panels) posterior distributions. Light-green points indicate (more...)

There is, therefore, a strong suggestion that the MCMC analysis, far from being a gold standard, is actually inferior to the SMC analysis. Where Table 7 seemingly shows the supposed inability of the SMC to provide a posterior sample that could be considered representative of the MCMC sample, this may actually be a result of the weakness of the MCMC algorithm, and SMC is preferred.

Returning to Figure 6, on the arrival of the data from day 84, each particle can be seen to require 220 iterations to accurately transition to a suitable posterior sample. When considered across 10⁴ particles, this represents a number of evaluations of the full likelihood that is less than four times that which is required by the 750,000 iterations of the MCMC algorithm typically used to derive a sample. As multiple chains are typically required to correctly diagnose convergence and to provide a sample, it can be seen that the SMC would only require only very modest benefits from parallelisation to be quicker to compute. When placed on a computing cluster, the SMC is considerably quicker to implement. The SMC algorithm developed here was implemented on a cluster that, depending on availability, permitted simultaneous calculation on 100 + processors. The particles are distributed evenly across the available processors, so that calculations on many particles are ongoing in parallel. Only at the resampling step and in the calculations of the ESS and the ICC is information shared across the processors. At day 83, we are considering the batch of data for which the greatest computational effort is required for the SMC analysis to derive inference. Elsewhere, the computational benefits of SMC are more clearly observed (e.g. by the end of the 83- to 120-day interval, less than 10 iterations are required for rejuvenations). For batches of data that do not lead to a rejuvenation of the particle sample, the SMC updates require a negligible amount of time to compute and are evidently very much quicker than the MCMC analyses, which still have to run very long chains to produce a reasonable posterior sample.