NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Adams J, Bateman B, Becker F, et al. Effectiveness and acceptability of parental financial incentives and quasi-mandatory schemes for increasing uptake of vaccinations in preschool children: systematic review, qualitative study and discrete choice experiment. Southampton (UK): NIHR Journals Library; 2015 Nov. (Health Technology Assessment, No. 19.94.)

Cover of Effectiveness and acceptability of parental financial incentives and quasi-mandatory schemes for increasing uptake of vaccinations in preschool children: systematic review, qualitative study and discrete choice experiment

Effectiveness and acceptability of parental financial incentives and quasi-mandatory schemes for increasing uptake of vaccinations in preschool children: systematic review, qualitative study and discrete choice experiment.

Show details

Chapter 5Discrete choice experiment

A DCE was undertaken to estimate the value parents place on key attributes and associated attribute levels of preschool vaccination programmes. DCEs describe a service or intervention in terms of a number of characteristics, or ‘attributes’ (e.g. where vaccinations are delivered, who delivers the vaccination and what information is provided). The extent to which an individual values a service or an intervention would be expected to vary as a function of the ‘levels’ of the attributes (e.g. one attribute could be the location in which vaccinations are administered, with levels being a local surgery or a community clinic).

Discrete choice experiments allow us to explore the relative importance of each attribute of the vaccination service that may influence a parent’s decision to vaccinate their child and allow uptake of services configured in different ways to be predicted.98 DCEs are a well-established methodology in health economics to elicit preferences on health-care products, programmes and in the valuation of preference for health states99101 and offer an additional approach to investigating service acceptability.

Discrete choice experiments involve three inter-related components: (i) an experimental design used to implement a choice survey and generate choice data; (ii) a quantitative statistical analysis to estimate preferences from choice data; and (iii) the use of the resulting model to either derive welfare measures or construct other policy analyses.98

Financial attributes/levels can also be incorporated into a DCE. Therefore, it is possible to determine a population’s willingness to pay (WTP) for, or accept, an intervention. The marginal willingness to accept (WTA) is defined here as the minimum monetary value that would be required to compensate for a change in the level of a certain attribute,102 that is, when individuals face a reduction in utility derived from moving from one scenario to another.

A DCE was designed for the following: (i) to establish the preferences of parents and carers of preschool-aged children for vaccination programmes with differing attributes and levels; (ii) to provide policy-relevant information on parental preferences on the configuration of vaccination programmes in England; (iii) to establish the likely minimum level of effective parental incentives; and (iv) to predict uptake levels of the different configurations of vaccination programmes.

Methods

We adhered to published guidance for undertaking a DCE study.98,103 In accordance with good practice, the DCE adhered to the four stages:

  • stage 1 – identification of attributes and levels
  • stage 2 – experimental design
  • stage 3 – data collection
  • stage 4 – data analysis and interpretation.

These are described in turn.

Stage 1: identification of attributes and levels

The results from published systematic reviews,57,16 the systematic review (see Chapter 3), the qualitative study (see Chapter 4) and a focused search of the general literature were used to develop a comprehensive list of potential attributes for inclusion in the DCE. This assessed the literature around immunisation services and programmes available for preschool children in England. Twelve studies identified that preferences for specific health professionals, parental education and perceived effectiveness of vaccinations are important factors influencing parental decision-making. An expert panel (comprising the project team and steering group members) deliberated these attributes and associated levels in a series of discussions. This process generated a provisional list of attributes and associated levels that may influence the uptake of preschool vaccination programmes. In all cases, attributes and levels had to be plausible in both clinical and policy terms.98 Information from all sources available to us informed the identification of maximum and minimum attribute levels, where relevant.

In addition to service configuration attributes, parental incentive attributes were included so that WTA could be indirectly estimated from the DCE.6 Incentive attributes were framed as financial rewards, which would be offered to parents for immunising their child as either a universal (all parents) or a targeted reward (targeted at parents highly unlikely to immunise their children). These attributes were included to arrive at an estimate of the minimum incentive (if any) that the population would be willing to accept in exchange for having a child immunised, and enabled the likely value of effective incentives to be established.

This process resulted in a list of nine attributes and associated levels (Table 8).

TABLE 8

TABLE 8

The DCE attributes and levels developed during stage 1

Establishing acceptability of the provisional list of attributes and levels

The Parent Advisory Group [see Chapter 1, Parent Advisory Group (public involvement)], comprising eight parents and guardians of preschool children, was consulted on the comprehensiveness and acceptability of the provisional list of attributes and levels. This took the form of an interactive workshop, which was facilitated by JA, RM and LT. Participants were presented with a hand-out (see Appendix 4) of the list of attributes and levels in Table 8 and their views on each were elicited. In addition, for attribute 3 (method of conveying information on net benefit from vaccination) participants were presented with examples of natural frequencies, clustered bar graphs and pictographs.

Participants reported mixed preferences for type of health-care professional administering vaccinations. With regard to location of vaccinations services, there was a dominant preference for community settings (e.g. Sure Start Centres). Participants also suggested additional locations for the provision of vaccinations, such as a vaccination bus, to be situated outside schools, and local pharmacies.

Few participants expressed a preference for vaccination services within a GP surgery; this was because of protracted waiting times and a lack of choice of appointment times, with several intimating that they had received a letter stating the date and time when they were required to attend their GP surgery. Extended waiting periods were considered particularly problematic, owing to a lack of child-friendly spaces and activities in GP surgeries. When participants were asked about the maximum time they would wait for an appointment, there was a general consensus that up to 60 minutes would be acceptable. Participants also reported difficulties in attending afternoon appointments, owing to the necessity of collecting other children from school at around 3 p.m.

The attribute of ‘method of conveying information on net benefit from vaccination’ elicited strong preferences for numerical presentation alone or the use of clustered bar graphs. Contrary to expectations, pictographs attracted largely negative comments and were considered to be unclear. Participants were in general agreement that receipt of information on benefits and risks was desirable to inform their decisions about immunising their children. However, those present in the workshop reported that they had received very little information on the benefits of vaccination. Indeed, the majority of participants would have preferred to have received balanced information on benefits and risks of vaccination presented to them verbally by a health-care professional (e.g. a health visitor) prior to appointments (e.g. 7 days before a scheduled appointment).

With regard to parental incentives, there was initially a strong opposition to any type of incentives being offered for immunising children. However, several participants conceded that £10–20 per visit (assuming seven visits) would be acceptable, with the caveat that this ‘reward’ should be universal (as opposed to targeted at those parents unlikely to immunise their children).

The workshop also identified very strong preferences for as few visits as possible, and it was unlikely that including this as a variable attribute in the DCE would yield any meaningful data.

Changes made to the provisional list of attributes and levels

These findings guided additional expert panel discussions to refine the provisional list of attributes and levels. Number of appointments was designated as a fixed factor and amended in accordance with recent NHS guidance (published in July 2014) that proposes five visits for the routine components of the child programme plus three visits (aged 2, 3 and 4 years for the influenza vaccination), with up to three injections per visit.104 Consequently, this fixed attribute reads as follows: ‘Number of appointments to complete the full vaccination programme = eight (with up to three injections per appointment).’

The type of health-care professional administering vaccinations was intrinsically linked to location. Therefore, this attribute was amended to capture preferences on a range of health-care professionals administering vaccinations at specific locations.

The members of the expert panel were in agreement that balanced information needs to be presented on the probabilities of disease and its consequences alongside absolute risk reduction in disease from vaccination. Taking into account feedback on the range of parental information needs on the benefits and risks of vaccination, the attribute ‘Method of conveying information on net benefit from vaccination’ was replaced with the following two attributes:

  1. Mode of information provision about vaccinations (benefits, risks and consequences) prior to appointment. This reflects the provision of ‘balanced’ information in terms of benefits, risks and consequences of a parent’s decisions to vaccinate or not vaccinate his or her child prior to the first appointment. The level was via the post, and, in order to demarcate the different methods of electronic information provision, a further two levels were included (e-mail and multimedia via the internet).
  2. Method of conveying information on consequences of disease, including absolute risk reduction in probability of disease from vaccination. There is good evidence that pictographs can effectively support the communication of balanced probabilistic information to people irrespective of their health literacy level.105 However, the examples of pictographs presented to participants in the workshop were not well received and were not considered clear or valuable. Furthermore, the graphical methods used in the workshop were illustrative examples, and a separate research project is warranted to identify the optimal mode, form and information content of graphical risk presentations in this case context. Therefore, the decision was made to omit illustrative examples of graphical display. To enable elicitation of preferences on generic methods of communicating probabilistic information, the following levels were presented textually: numerical (percentages and frequencies ‘out of 100 children’); graphical methods such as bar graphs; and both numerical and graphical methods.

Given the value that workshop participants placed on verbal information provided by trusted health-care professionals, the levels of the attributes related to benefit and risk information, and method of probabilities on consequences and risk reduction from vaccination, were all suffixed with ‘plus verbal information at time of appointment’.

The phrase ‘normal hours’ was replaced with ‘working hours during school term times (9 a.m. to 5 p.m.)’ in the levels of the attribute ‘Availability of appointments’.

References to quasi-mandatory schemes were raised during the expert panel discussion. However, these were not deemed to be prudent for inclusion in the DCE, as exploring mandates and incentives within the same DCE would be counterintuitive and impose too many design constraints; that is, inclusion of a mandate in a choice scenario would prohibit the inclusion of levels related to parental incentives (form, type and value). However, an item to assess preferences for no reward (current practice), targeted versus universal rewards, and mandatory schemes was given in the questionnaire included with the DCE.

The terms ‘reward’ and ‘incentive’ were used interchangeably in the descriptions of DCE attributes and the associated levels. Expert panel discussions emphasised the need for consistency (and interpretation, as there are qualitative differences between the two terms); the term ‘reward’ was deemed to be more accessible to parents and any references to incentive were replaced with ‘reward’ in the DCE attributes and levels and other survey items. Levels for the parental reward attribute were also amended based on discussions with the Parent Advisory Group.

For the attribute ’waiting time at each appointment’, the three levels were amended in accordance with the maximal value identified from the workshop (60 minutes), with a capped value (up to 120 minutes) to facilitate interpretation of the parameter estimates.

The revised list of eight attributes and levels for inclusion in the DCE are shown in Table 9, with number of visits to complete the full vaccination programme designated as a ninth, fixed attribute.

TABLE 9

TABLE 9

The DCE revised list of attributes and levels used in paper piloting

Stage 2: experimental design

The DCE experimental design followed guidelines for best practice.106 All possible combinations of attributes and levels described in Table 9 would generate a prohibitively large number of choice scenarios, namely 12,150 (the total number of possible combinations can be calculated as 35 × 52 × 2, i.e. five attributes with three levels, two attributes with five levels and one attribute with two levels). Therefore, we used a D-efficient design to identify the most efficient combination of choice sets while still being able to estimate the main effects and all higher-order interactions.98,107

The D-efficient design was generated using Ngene design software (version 1.1.1, ChoiceMetrics Pty Ltd, Sydney, NSW, Australia) with input from an expert in the design and analysis of DCEs and following guidelines for best practice.106 The best design generated by Ngene was chosen with the aim of minimising the standard errors.108

Not all attributes and level combinations were plausible (e.g. if type of reward was ‘no reward’ then value of reward must be ‘£0’). Design constraints were utilised to ensure plausibility and reduce hypothetical bias (where the hypothetical nature of the questions result in biased responses). Excluding implausible combinations of attribute levels affected level balance and statistical efficiency (a good choice design is level balanced where all levels of an attribute occur with equal frequency across the total number of choice sets included in all versions of a questionnaire). Levels were unbalanced with regard to type of reward, with ‘no reward’ appearing less frequently; and incentive amount, with ‘£0’ appearing less frequently. For respondents with strong preferences for those under-represented attribute levels of ‘no reward’ and ‘£0’ incentive value, the opt-out option was assumed to present a valid alternative choice. Therefore, following guidelines for best practice, level balance (and hence some statistical efficiency) was sacrificed in favour of a practical and plausible design with the aim of increasing in overall response efficiency.

In order to minimise respondents’ cognitive burden, each participant was presented with 18 choice questions. The design allowed for four blocks of 18 choice scenarios to maximise variance in the data. Each respondent was assigned to one block randomly.

Questionnaire design

A paper-based questionnaire survey (see Appendix 5 for the final version) was designed, with reference to published guidance on good survey design.109

The questionnaire survey instrument included the following sections:

  • Study information: what the work is aiming to achieve, what participation would involve, the estimated time to complete the survey and the purpose of the research, including obtaining consent to participate.
  • Screening questions, based on inclusion criteria.
  • Respondents’ (and, where applicable, their partners’) demographics, socioeconomic status (household income, highest level of education, employment status, ethnicity) and self-assessed health status.
  • Details of the respondents’ children (number, age, gender, presence of disability).
  • Introduction and explanation of the choice task with an illustrative example of pairwise choice scenario (choice set), followed by presentation of choice sets based on the Ngene design in the format shown in Table 10.
  • Questions to assess how difficult the respondents found the DCE task to complete.
  • Questions on the influence of financial incentives on decisions to immunise, including for those individuals who stated that they would require an incentive to vaccinate, the minimum value of that incentive (their WTA), and, for those who would not require an incentive, the maximum level at which they believed an incentive should be set (including £0). Questions also included whether cash or vouchers would be preferable and the reasons underpinning the minimum acceptable incentive value they stated (if applicable).
  • Questions about preferences for organisation of vaccination services (universal, targeted, mandatory or current practice).
  • Information on vaccinations, including mode information, was received, extent information needs were fulfilled and alternative sources of information were consulted.
  • Intentions for immunising participants’ youngest child, followed by a series of attitudinal questions designed to assess attitudes towards the safety, importance/value and efficacy of vaccinations.
  • A ranking exercise in which respondents were asked to rank order the eight attributes in the choice sets.
  • An open-ended question asking for any further information on the topic of vaccination in preschool age children.
TABLE 10

TABLE 10

Illustrative example of a pairwise choice set used in DCE paper piloting

Paper piloting the discrete choice experiment and questionnaire

Paper piloting was undertaken with five parents or guardians of preschool-aged children (one male and four female) using a ‘think-aloud’ approach110 to test respondents’ understanding of the wording of questionnaire survey items, the DCE choice task, the definitions of attributes and levels; and identification of salient ‘missing’ attributes and levels. The sample size for this aspect was determined by data saturation – that is, we continued conducting additional interviews until no new issues were raised.

The first two participants took considerable time to read text in both columns of the choice sets (i.e. they were experiencing excessive cognitive burden that prohibited an expeditious and valid comparison of attribute levels between the pairwise scenarios). Therefore, the format of presenting the pairwise choice sets shown in Table 11 was used to reduce cognitive burden in subsequent pilot participants.

TABLE 11

TABLE 11

Illustrative example of the revised format pairwise choice set used in DCE paper piloting

Based on responses during the paper piloting, participants expressed a desire to know why information on demographics and socioeconomic status was being sought. In response, we included the following information: ‘the following questions ask about the characteristics of the parents responding to the survey (so we can demonstrate that we have collected information from a representative cross section of parents living in England) and to explore how characteristics of parents could help us to design better vaccination services’. An option of ‘prefer not to say’ was also added to items in this section.

Participants also noted that knowledge of the full vaccination schedule was a prerequisite for responding to the questionnaire items. Consequently, in the introduction section a link to details of the vaccination schedule currently recommended by the NHS was included.

Other comments that resulted in amendments were that participants wanted to know whether or not pharmacists would have received specific training on vaccinations in order to consider this attribute as a valid option. In the section on WTA, one participant queried the locations at which shopping vouchers could be redeemed (this item was subsequently amended to state that shopping vouchers would be accepted in most high-street shops and supermarkets). The paper pilot identified no other salient issues (other than minor typographical errors and clarification of potentially ambiguous definitions in the demographic and socioeconomic status items).

The findings of the paper-pilot, along with additional expert panel discussions, informed the development of a revised list of attributes and levels (Table 12) along with a final paper-based version of the questionnaire survey for electronic piloting.

TABLE 12

TABLE 12

The DCE final attributes and levels

Electronic piloting of the discrete choice experiment questionnaire

A market research company (ResearchNow.com) converted the amended paper-based questionnaire into an online survey. The survey was then subjected to an initial ‘soft launch’ with 40 respondents to establish usability of the survey, as well as to assess response fatigue (time taken to complete) and understanding of the DCE choice sets and WTA questions. This identified an issue with the WTA question, whereby several respondents entered a minimum incentive value of £0 after stating that they would require an incentive in order to vaccinate their child. Only positive values were considered to be valid responses to this question. This section of the survey was redesigned to minimise this type of response. The original wording of the WTA question was:

Would you be willing to accept a financial reward for immunising your child? (Y/N)

Please state the minimum amount you would be willing to accept.

This was amended to:

Would you require a financial reward to immunise your child? (Y/N)

If Yes:

What is the minimum amount you would require?

If No:

If offered a financial reward on completion of vaccinations would you take it? (Y/N)

What do you think should be the maximum financial reward offered to parents for immunising their child?

A second ‘soft launch’ was undertaken with the revised WTA questions (n = 37). In addition to the tests conducted as part of the first soft launch, further data analysis was undertaken at this stage. This included running preliminary regression analyses (n = 77) and analysing data relating to participants’ understanding of the DCE questions. The majority (82%) of respondents stated that they fully understood the questions, while 17% partially understood and around 1% reported that they did not understand the DCE questions. This high level of understanding of the DCE was deemed satisfactory and no further changes were made to the DCE design or survey questionnaire.

Stage 3: data collection

Ethical approval for all aspects of this study (piloting and main study data collection) was granted from Newcastle University Ethics Committee (reference 00748).

Recruitment and data collection was subcontracted to the market research company (ResearchNow.com), who adhere to the highest standards of market research ethics, as described in the Market Research Society’s Code of Conduct.111 ResearchNow use a number of methods to recruit individuals onto their panels, including e-mail, online marketing and website targeting. ‘By invitation’ methods are also used, where individuals with known characteristics are directly targeted. Small (£1–2) incentives in the form of shopping vouchers were paid to participants, as per their normal procedures. Data were returned to the research team in an anonymised format. The research team did not receive contact details or any personal identifier information from survey participants.

The first soft-launch data collection started in November 2014. Final survey data collection commenced in December 2014, with the final data set available in early January 2015.

Inclusion criteria

Two samples were included in the study: relative to the likelihood of not having their children fully vaccinated, parents were identified as either ‘at high risk’ or ‘not at high risk’ of not having their children fully vaccinated. It is particularly important to explore the preferences of parents ‘at high risk’, as this subgroup would probably be the primary target of any parental incentive scheme. It was also important to explore the preferences of parents who were ‘not at high risk’ in order to determine the wider impact of population-wide changes in the configuration of vaccination services.

Individuals who met the following criteria were eligible to complete the survey:

  • a parent or guardian of one or more children aged under 5 years
  • currently residing in England
  • a member of an online research panel held by the subcontracting market research company ResearchNow.com.

Respondents who additionally met the following criteria were included in the ‘at high risk’ sample:57,10,11

  • living in one of the 20% most deprived areas of England – as identified by Index of Multiple Deprivation 201085 score of lower super output area of residence, calculated from postcode of residence
  • the parent or guardian of a child aged under 5 years old who has a physical or mental disability
  • a single parent or guardian
  • aged under 20 years
  • a parent or guardian of more than three children (of any age).

Respondents who did not meet any of the above criteria were assigned to the ‘not at high risk’ sample.

Sample size

Lancsar and Louviere98 highlight the complexities and problems of performing sample size calculations for DCEs and the need for further research in this area. Previous studies using DCEs for exploring experience factors in health-care settings have included samples ranging from fewer than 50112 to almost 4000,113 and robust choice models have been estimated from sample sizes of between 50 and 100 respondents.114 Optimal sample size requirements for DCEs depend on knowledge of the true choice probabilities, which are not known prior to undertaking the research115 and, therefore, DCE sample size estimates are generally based on rules-of-thumb and budget constraints. Given the number of attributes included in the DCE, it was estimated that a minimum sample size of 400 [i.e. 200 ‘at high risk’ (50 per block) plus 200 ‘not at high risk’ parents (50 per block)] would provide sufficient statistical power based on a rule of thumb of a minimum of 10 observations per parameter estimate plus an additional 50.

Stage 4: data analysis and interpretation

Descriptive statistical techniques were used to describe the sociodemographic profile and characteristics of parents and their children of the full sample, and the subgroups ‘at high risk’ and ‘not at high risk’ of incompletely vaccinating their children. Appropriate tests of differences (chi-squared tests and t-tests) were used to establish any statistically significant differences between the subsamples as a function of sociodemographic and child variables.

The DCE approach allows an analysis of individual stated preferences in response to hypothetical choices and enables the quantification of the relative importance of each attribute/level during the decision-making process.103,107 When presented with hypothetical options (i.e. choice scenarios) that describe alternative specifications of a vaccination service, respondents are assumed to choose the scenario they prefer. The higher a respondent’s preference for certain attribute levels, the more likely they are to choose that scenario over any alternative.

Appendix 6 provides a technical description of the data analytic strategy applied to the choice scenario data. In short, the initial analysis employed a conditional logit model116 which is based on three assumptions: (i) independence of irrelevant alternatives (IIAs) (i.e. the ratio of probabilities for any two alternatives is assumed to be independent of the attribute levels in a third alternative); (ii) error terms are independent and identically distributed across observations; and (iii) no preference heterogeneity (i.e. homogeneous preferences across respondents).

As the number of observations for the opt-out option was too small (around 5% of chosen options) to perform the appropriate Hausman test, we assumed a violation of the IIA assumption, which would result in biased parameter estimates. Therefore, mixed-effects logistic regression models were considered more appropriate to analyse the choice set data. Mixed logit models counteract any violations in the assumptions of conditional logit models and permit the investigation of unobserved preference heterogeneity, that is, varying model estimates across individuals.

Mixed logit models were, therefore, used to establish whether or not the eight attributes presented in the choice scenarios were statistically significant predictors of parents’ preferences. Positive coefficients in the models represent a positive preference (utility) associated with a particular level of an attribute, whereas negative coefficients represent a negative preference (disutility) associated with a particular level of an attribute compared with the reference level. p-Values < 0.05 indicate whether positive or negative preferences are statistically significantly different from zero. Mixed logit models were undertaken using choice set data from the full sample and the two subgroups.

For all models the intercepts [alternative specific constants (ASCs)] and time attribute are assumed to be random and normally distributed [means and standard deviations (SDs) reported]; all other parameters in the model remain fixed (mean estimates reported only). For all mixed logit models, positive values for the intercepts (ASCs for options A and B) would indicate a general parental preference for vaccinating their children. Attributes capturing ‘time’ and ‘value’ were included in the analysis as linear variables. Effects coding was used for all categorical attributes (using +1, 0 and –1 to represent different attribute levels) to facilitate estimation of main effects across all categorical attributes and levels.117 The level ‘no reward’ in the attribute ‘type of reward’ was omitted in these analyses owing to multicollinearity; the no-reward situation was already coded by the according category in the type of reward attribute and additionally included in the choice sets descriptions only for plausibility.

Marginal WTA values for all statistically significant attributes in the mixed logit models were also calculated for the full sample and subgroups in the form of a minimum monetary value that would be required as ‘compensation’ for any change in the level of an attribute associated with gains or losses in utility.102 This enables an estimate of the trade-offs between attribute levels and the magnitude of the coefficient in the mixed logit models (i.e. the WTA values can be compared to determine the relative strength of preferences). The estimated WTA values, therefore, represent the amount of financial incentive (in £) that parents would need to receive in order to compensate for accepting a level of an attribute of a vaccination service that is associated with a negative preference (disutility).

We did not check for internal inconsistency, as a number of studies have now shown their DCEs are internally consistent and valid. The qualitative work and pre-testing of different versions of the questionnaire during the development stage further helped to reduce the risk of internal inconsistencies.118121

Responses to the following questions presented after the respondents had completed the choice task (DCE) were analysed using appropriate descriptive statistics for the full sample and two subgroups: parents’ preferences for organisation of vaccination services (universal, targeted, mandatory or current practice); influence of financial incentives on decisions to immunise, including for those individuals who stated that they would require an incentive to vaccinate, the minimum value of that incentive (their WTA), and, for those who would not require an incentive, the maximum value they thought an incentive should be (their WTP); preferences for cash or voucher rewards and the reasons underpinning the minimum acceptable incentive value (if applicable); information received on vaccinations, including the way that information was received, the extent their information needs were fulfilled and alternative sources of information consulted; and the rank order assigned to the eight attributes presented in the DCE choice task.

Items used to assess parents’ intentions for immunising their youngest child, and attitudes on safety, importance/value and efficacy of vaccinations, were analysed using overall means and SDs for each subscale in accordance with the factor structure reported in Kennedy et al.122 Differences between subgroups were analysed using independent t-tests and chi-squared statistics.

Observed and predictive uptake rates for vaccination services were calculated for the full sample. Observed uptake rates were based on responses to the 18 choice questions. Predictive uptake rates were based on findings of the mixed logit models and used to produce estimated probabilities that control for all levels included in the 72 choice sets in the DCE across all respondents.

Uptake rates were also calculated for vaccination services based on the most and least preferred choice scenarios (as defined by the DCE results) and were compared with predicted uptake rates based on the scenario that represents current practice and an ‘opt-out option’. These statistics are expressed as percentages and capture the variation in uptake rates associated with different configurations of vaccination services based on preferences across the sample.

Results

Sample characteristics

Table 13 presents a summary of the sociodemographic profile and characteristics of the study respondents in the overall sample (n = 521) and subgroups of respondents classified to be ‘at high risk’ (n = 259, 49.7%) of not fully vaccinating their children and ‘not at high risk’ (n = 262, 50.3%).

TABLE 13

TABLE 13

Characteristics of sample

Respondents had a mean age of 34 years. The majority of respondents were white (86%); female (71%); employed or on maternity leave (73%); and in a relationship (76%). The modal annual income was £35,000 to < £50,000 (23%). Almost half had a degree or post-graduate education (48%). The majority of the sample reported ‘good’ health (53%) over the past 12 months and had an average of two children (4% of whom were reported to have a physical or mental disability).

Many of the differences between subgroups reflect the criteria for allocation to groups. There were significantly more female and single or separated/divorced/widowed respondents, with, on average, more children and children with disabilities in the ‘at high risk’ group. A significantly smaller proportion of respondents in the ‘at high risk’ group reported good health over the past 12 months. The ‘at high risk’ group also had significantly greater and smaller proportions of respondents in the lowest (< £15,000) and highest (> £70,000) annual household income ranges, respectively. Significantly larger proportions of respondents in the ‘not at high risk’ group had a degree or post-graduate education and were employed or on maternity leave, with significantly greater numbers of unemployed respondents in the ‘at high risk’ group. Patterns of ethnicity were approximately equivalent across subgroups.

Regression model results (discrete choice experiment data)

Model 1 results (full sample)

The results of the mixed logit models for the full sample are presented in Table 14. The ASCs for options A and B are both positive and statistically significant, indicating that there is a general preference for vaccinations, compared with not vaccinating (option C). Indeed, only 5% of all respondents chose the opt-out option, ‘I would not vaccinate my child’.

TABLE 14

TABLE 14

Summary of mixed-effects logit regression exploring preferences for DCE attribute levels (total sample, n = 521)

The SDs for all random coefficients, namely ASCs and time variables, are statistically significant, indicating the existence of preference heterogeneity among respondents for the characteristics of vaccination services in the DCE choice sets.

Given the use of effects coding, the reported results for each of the attribute levels indicate the distance from average utility (specified as the negative sum of coefficients for all but the reference level) derived from a specific attribute. According to equation (2) (see Appendix 6), positive coefficients indicate a positive preference (utility) and negative coefficients indicate a negative preference (disutility) associated with a specific attribute level compared with the reference category. Non-significant attribute levels indicate respondents’ indifference between the specific level and the reference level of each attribute.

There were statistically significant negative preferences (disutility) for pharmacists delivering vaccination in a local pharmacy (WTA = £142.40) and community nurses delivering vaccinations in a community bus stationed at schools (WTA = £67.96); for use of charts or pictures to convey information on risks and benefits (WTA = £55.45); use of targeted rewards (WTA = £194.40); and for longer waiting times at each appointment (WTA = £8 per minute of additional wait after having waited for 30 minutes).

On the other hand, there was a statistically significant positive preference (utility) for cash rewards. Higher-value rewards were associated with a statistically significant increase in utility.

Models 2 and 3 results (subgroup analyses)

Results of the mixed logit models for the subgroups of respondents classified as ‘at high risk’ and ‘not at high risk’ are presented in Table 15. There was a general preference for vaccinations in both subgroups as well as the existence of preference heterogeneity for the characteristics of vaccination services in the DCE choice sets. This result is consistent with the full sample (model 1), in which only 5% of all respondents use the opt-out option ‘I would not vaccinate my child’. In the subgroup analyses, 6% of the ‘at high risk’ group and 4% of the ‘not at high risk’ group use the opt-out option.

TABLE 15

TABLE 15

Summary of mixed-effects logit regression exploring preferences for DCE attribute levels, by subgroup

There were statistically significant negative preferences (disutility) for pharmacist-delivered vaccinations in both the ‘at high risk’ and ‘not at high risk’ groups; although this disutility was substantially stronger in the former than the latter group (WTA = £356.84 and £88.44, respectively). In the ‘not at high risk’ group there was a statistically significant disutility for vaccinations provided by community nurses in a school-based vaccination bus (WTA = £104.17) and utility for vaccinations provided by health visitors in the community.

Parents in the ‘at high risk’ group also had statistically significant disutility for receipt of information on the benefits and risks of vaccinations in the form of charts and pictures (WTA = £186.12). Furthermore, only the ‘not at high risk’ group had a statistically significant positive preference (utility) for additional flexibility around availability of appointments.

Both subgroups expressed significant disutility for targeted rewards (WTA = £461.65 and £126.28 in the ‘at high risk’ and ‘not at high risk’ group, respectively), whereas only respondents at high risk of not fully vaccinating their children had a statistically significant utility for cash rewards. Increased value of rewards was statistically significant only in the ‘not at high risk’ group.

Finally, increased waiting times were associated with significant disutility in both subgroups (WTA values for each additional minute waiting after 30 minutes was £22.22 and £4.55 in the ‘at high risk’ and ‘not at high risk’ group, respectively).

Additional questions on attitudes towards preschool vaccination service organisation

Individual attitudes towards the organisation of vaccination services and attitudes towards financial rewards for preschool vaccinations were captured using the questionnaire which followed the DCE. The questionnaire also included open-ended WTA and WTP questions using the contingent valuation method. Results of these additional questions are presented in Table 16.

TABLE 16

TABLE 16

Attitudes concerning organisation of preschool vaccination services

One-quarter of respondents stated that they would require a financial incentive to vaccinate their child, with a higher proportion of respondents in the ‘at high risk’ group than ‘not at high risk’ groups indicating this (31% vs. 19%). Of those who stated that they would require an incentive, the average ‘minimum’ WTA was £110 (greater in the ‘at high risk’ than in the ‘not at high risk’ group) and the most frequently cited reason was compensation for their time, followed by time off work.

Consistent with the results of the logit models, there was a generic preference for cash as opposed to voucher rewards as well as for universal as opposed to targeted rewards.

Although a large majority of respondents stated that they would not require a financial incentive to vaccinate their child, approximately 80% would accept one if offered. The maximum value of any incentive (for an unspecified vaccination programme) on average was £69.41 and was substantially larger in the ‘at high risk’ subgroup (£81.34) than in the ‘not at high risk’ group (£57.41). Again, the most common reason influencing these values was compensation for time to attend appointments.

Worthy of note is that approximately one-fifth of respondents expressed a preference for current organisation of vaccination services and a further one-fifth for mandatory schemes. Preference for the latter was greater in the ‘not at high risk’ subgroup (31% vs. 24%), but this difference was not statistically significant.

Intentions and attitudes towards the safety, importance, value and efficacy of preschool vaccinations

On average, the majority (76%) of respondents in the full sample stated an intention for their youngest child to receive all of the recommended vaccinations. Respondents in the ‘at high risk’ subgroup were significantly less likely to express an intention for their children to receive all the recommended vaccinations (69%) than those in the ‘not at high risk’ subgroup (83%).

Attitudes towards importance, value and efficacy of preschool vaccinations were also generally positive, with mean values of ≈4 for all items (Table 17). However, attitudes were significantly more positive in the ‘not at high risk’ subgroup, compared with the ‘at high risk’ group for 9 out of the 12 attitudes items (related to value/importance and safety of vaccinations). Significant differences in proportions of responses ‘strongly disagree/not at all likely’ were found between subgroups, with greater numbers of respondents in the ‘at high risk group’ stating ‘strongly disagree/not at all likely’ for items on perceived susceptibility of their child, other children and family members to serious disease if children are not immunised.

TABLE 17

TABLE 17

Perceived importance, value and efficacy of preschool vaccinations

Information about vaccinations

The majority of respondents (55%) stated that they had not received any information regarding vaccinations in the past 3 months, with similar proportions in both subgroups (Table 18). Of those who had received information, the majority reported that they had received this by post (67%). Respondents were also asked ‘to what extent would you agree the information you received for your youngest child in the last 3 months addressed all your information needs about vaccination?’. Overall, 63% (n = 148) of the full sample responded strongly agreed or agreed with the statement.

TABLE 18

TABLE 18

Requirements for, and sources of, information about vaccination

Additional information regarding vaccinations was sought by 43% of respondents, including 49% of parents ‘at high risk’ of incompletely vaccinating their children and 37% of those ‘not at high risk’. Information sources consulted by the full sample included the internet (57%), GP (43%) and friends or family (41%). There were no statistically significant differences between subgroups for any of the responses to the questions about vaccination information.

Ranking exercise

The rank order (highest to lowest) of importance for each of the eight attributes in the DCE based on mean scores for the overall sample was:

  1. location type of health-care professional administering vaccinations (mean score 2.81, SD 1.82)
  2. how information is received prior to vaccination (mean score 3.10, SD 1.85)
  3. availability of appointments (mean score 3.10, SD 1.53)
  4. how information on risks and benefits is communicated (mean score 3.26, SD 1.73)
  5. waiting times (mean score 4.59, SD 2.00)
  6. value of rewards (mean score 6.05, SD 1.69)
  7. type of reward – cash or voucher (mean score 6.25, SD 1.45)
  8. who receives the reward, for example universal or targeted (mean score 6.84, SD 1.32).

Predictive uptake rates

Appendix 7 presents the percentages of respondents choosing specific options from the 72 choice sets presented in the DCE as well as the predicted probability of uptake estimated from the mixed logit models.

Predicted uptake rates for a vaccination incentive scheme (i.e. options A or B) ranged from 13% (choice situation 27, option B; observed, 22%) to 85% (choice situation 27, option A; observed, 70%). Conversely, predicted uptake for the no vaccination scheme (i.e. opt-out option C) ranged from 2% (choice situation 4; observed, 3%) to 7% (choice situation 52; observed, 11%).

Predicted uptake rates, based on a hypothetical choice set forcing respondents to choose between the most preferred scenario [health visitor in the community at local clinic or children’s centre; vaccination information: electronic (e-mail); risk information: numbers and charts/pictures; during working hours and out of hours; cash reward; value (£280); universal reward; waiting time (up to 30 minutes); ASC] and a current practice alternative [practice nurse; vaccination information: written (post); risk information: numbers; during working hours; type of reward: no reward; value (£0); parents receiving reward: no reward; waiting time (up to 30 minutes); ASC], were 66% for the most preferred scenario in the total sample (60% in the ‘at high risk’ sample, 55% in the ‘not at high risk’ sample), 32% for the current practice scenario in the total sample (38%; 44%) and 2% for the opt-out option (2%; 2%).

Similarly, for a choice set including the least preferred scenario [pharmacist; vaccination information: electronic (internet); risk information: charts/pictures; during working hours; shopping voucher; value (£0); targeted reward; waiting time (up to 120 minutes); ASC] current practice scenario and an opt-out option, the predicted uptake rates were estimated as 15% for the least preferred scenario in the total sample (36% in the at high risk sample, 33% in the not at high risk sample), 80% for the current practice scenario in the total sample (61%; 63%) and 5% for the opt-out option (3%; 3%).

Discussion

Summary of results

This is the first DCE to investigate the differential configuration of vaccination services and their potential impact on parents’ decisions to vaccinate their children in accordance with the full programme recommended by the NHS, including the efficacy of incentive-based schemes on service acceptability and uptake rates.

Analysis of choice set data from the DCE (hypothetical vaccination services described in the choice scenarios) revealed that ≈95% of parents had strong preferences for vaccinating their children and we found evidence of statistically significant preference heterogeneity for different characteristics of vaccination services.

In terms of type of health-care professional administering vaccinations and their location, there was a strong negative preference for pharmacists. Statistically significant negative preferences (disutility) were identified for vaccination services with three characteristics. In descending rank order of WTA these were (i) pharmacists with specialist training (compared with practice nurses within GP practices), with stronger disutility for this characteristic in the ‘at high risk’ group; (ii) targeted rewards to parents who are ‘at high risk’ of not adhering to the full vaccination programme (as opposed to universal rewards for all parents), which was stronger in the group ‘at high risk’ of incompletely vaccinating their children; and (iii) longer waiting times for each appointment with greater disutility, as indicated by WTA values in the ‘at high risk’ group per additional minute after 30 minutes had expired.

We also found evidence of differences in preferences for characteristics of vaccination services as a function of a parent’s risk status. In those ‘at high risk’ of incompletely vaccinating their children, there were significant positive preferences (utility) for cash rewards for completing the full vaccination schedule (as opposed to no reward), and a significant disutility associated with presenting information on the risks and benefits of vaccinations using graphical methods (compared with only numerical presentation).

In those ‘not at high risk’ of incompletely vaccinating their children, there were significant positive preferences (utility) for provision of vaccinations by health visitors in the community and strong negative preferences for community nurses in a school-based vaccination bus. This group also preferred more flexible appointment times (out of hours). They also assigned greater utility to increasing reward values.

There were no statistically significant positive or negative preferences for mode of information provision about vaccinations (benefits and risks) prior to appointment (written vs. electronic).

Consistent with the results of the logit models, there was a generic preference for cash as opposed to voucher rewards as well as universal as opposed to targeted rewards in the attitudinal questionnaire. We also found evidence of a ‘gift horse attitude’ being adopted towards the issue of incentives. The majority of parents reported that they did not require a reward (the ≈20% who did reported an average ‘minimum’ WTA value of £110), but the majority would accept a financial reward for completion of the vaccination programme if it was offered to them (maximum value ranged from £57.41 in the ‘not at high risk’ group to £81.34 in the ‘at high risk’ group). Compensation for time to attend appointments was the factor that most influenced the rationale underpinning the minimum and maximum values suggested by parents.

There was minority support (≈20%) for current practice and for mandatory schemes, should they be rolled out, with support for the latter greater in those ‘not at high risk’ of incomplete vaccination (31%).

Responses to items about the intentions of parents for their youngest child to receive all of the recommended vaccinations were high, but not maximal (i.e. 76% overall), and there was a statistically significant lower rate of expressed intentions in the subgroup ‘at high risk’ of incomplete vaccination (69% vs. 83% in the ‘not at high risk’ group). In the full sample, attitudes towards the safety, importance/value and efficacy of immunisations was positive across both subgroups, although statistically significantly greater numbers of respondents in the ‘at high risk’ group stated ‘strongly disagree/not all likely’ for items on the perceived susceptibility of their child, other children and family members to serious disease if children are not vaccinated.

Approximately 50% of parents in both subgroups had not received any information about vaccinations in the past 3 months. Although the majority (63%, n = 148) were satisfied that the information they had received fulfilled all of their information needs, substantial numbers of respondents (43%) sought additional information, most frequently via the internet.

Interpretation of results in relation to the literature

Differences in expressed preferences for organisational and incentive characteristics of vaccination services between the subgroups of parents can largely be explained by the criteria used to classify them. As would be expected, respondents classified as ‘at high risk’ of incompletely vaccinating their children had statistically significantly more children, more children with a disability, lower household income levels, higher rates of unemployment, lower educational attainment and were poorer self-assessed than parents classified as ‘not at high risk’. Respondents ‘at high risk’ of incompletely vaccinating their children were also more likely to be female and single or separated, divorced or widowed. These sociodemographic characteristics have previously been reported to strongly influence parental intentions, attitudes and decision-making about childhood vaccination.5,7

Strong opposition to pharmacists delivering vaccinations may be a result of concerns around safety and perceived adequacy of training. This is supported by comments to this effect from the Parent Advisory Group workshop in the development phase of this study. Disutility in parents ‘not at high risk’ of incompletely vaccinating their children for vaccinations provided by community nurses in vaccination buses located at schools may be attributable to these parents placing a greater value on flexibility. This is reflected in this subgroup having significant utility for health visitor-administered vaccinations in the community, and the availability of out-of-hours appointments.

It is not surprising that cash rewards were preferred by parents in the ‘at high risk’ group, given the lower-income status of these parents. However, increasing the value of a reward was associated with significant utility only in the subgroup of parents who were ‘not at high risk’ of incomplete vaccination. Therefore, cash and higher-value non-financial incentives may be more effective depending on the parents’ inherent risk status.

Disutility for targeted rewards was found in the DCE logit regression models, the survey responses and in the discussions with the Parent Advisory Group. Reasons provided by Parent Advisory Group members were that some parents may be susceptible to ‘game playing’ in the form of strategically delaying vaccinations if they thought that a reward might be given. Furthermore, targeted rewards were considered by some parents in the Parent Advisory Group to represent a reward for ‘bad behaviour’ (i.e. not vaccinating children). The finding that parents ‘at high risk’ of incompletely vaccinating would be less willing to wait at vaccination appointments than those ‘not at high risk’ may be related to the former group experiencing greater difficulties with arranging child care or cover at work as well as other practical considerations while they are waiting for appointments (e.g. having more children to manage).

There is robust evidence that graphical displays such as bar graphs and pictographs can effectively support the communication of balanced probabilistic information to people irrespective of their health literacy level.105 However, parents ‘at high risk’ assigned disutility to this method of information provision. This was reflected in the Parent Advisory Group workshop, where examples of graphical display (pictographs and bar graphs) were not well received. This finding for disutility of graphical methods in the DCE may be an artefact created by presenting these options using textual descriptions as opposed to mocked-up examples.

The high proportion of parents reporting non-receipt of information within the past 3 months may be influenced by recall bias. It will also be influenced by the specific age of their children and whether or not the children were due vaccinations in the previous 3 months. However, this finding provides some further evidence that quality information provision is currently suboptimal, and this was supported by comments made in the Parent Advisory Group.

The use of the internet is common to investigate issues relating to health. However, not all sites are reputable and there is a risk of parents being exposed to inaccessible, unbalanced and unreliable information that may impact on their decisions about vaccinating their children.

Suboptimal intentions to vaccinate their youngest child were more pronounced in those identified as ‘at high risk’ of incompletely vaccinating their children. This may in part be related to disutility associated with long waiting times for appointments, suboptimal information provision and lower perceived susceptibility of their child, other children and family members to serious disease if children are not vaccinated.

Strengths and weaknesses

A major strength of our approach to eliciting preferences for different characteristics of vaccination services is the use of a DCE embedded within an online survey. We adhered to best practice guidelines for design and development of DCEs (with a clear audit trail and engagement of an expert Parent Advisory Group) and survey questionnaires to maximise internal validity and external validity.

The strength of the DCE approach is that it permits an examination of multiple factors influencing decision-making and that all choices involve trade-offs between levels of multiple factors, which cannot be readily elucidated, rank ordered or quantified using other methods. The increased internal and external validity of the DCE approach is evidenced by findings of the ranking exercise that does not account for preferences as a function of different levels of attributes or their complex trade-offs with levels of other attributes. Moreover, we incorporated WTA terms into the DCE analyses to facilitate the interpretation of results in terms of the strength of positive or negative preferences.

Discrete choice experiments have been criticised as difficult for participants to understand. To minimise the risk of this we conducted substantial development work. The overwhelming majority of respondents indicated a good understanding of the DCE.

We acknowledge that the sample recruited by the research company is effectively a non-random (convenience) sample that may not be representative of the target population. Furthermore, by using the recruitment method we did, it is not possible to determine response rates to the survey. However, we asked ResearchNow to recruit a national sample of parents that fulfilled our inclusion criteria, which were stratified into groups according to pre-defined criteria in order to maximise representativeness. Previous DCE research suggests that alternative methods of recruitment and data collection in this context, such as random postal or telephone surveys, achieve very low response rates, which also threatens representativeness.123 By using a stratified sampling matrix we were at least able to ensure a degree of representativeness in the sample.

We combined health-care professional delivering immunisations and the place of delivery of immunisations into one attribute. Although these may be seen as separate constructs, they are inextricably linked, and separating them out would have imposed a prohibitive number of design constraints, which would have had negative implications for the model parameters and conclusions that could be drawn from the results.

As with the qualitative study, we were primarily asking participants to reflect on interventions that few, if any, of them would have had any personal experience of. This may have limited their ability to give fully informed opinions.

Furthermore, and as with the qualitative study, we restricted the DCE to consideration of incentives for parents of children who had fully completed the preschool vaccination schedule described in Table 1. This was both to reflect existing practice, as described in the systematic review, and because partially rewarding partial immunisation would have introduced significant complexity into the design of the DCE.

Implications of findings for research

Respondents desired probabilistic information about the benefits and risks of vaccination and expressed strong preferences for numerical presentation of these data. Natural frequencies using consistent denominators (out of 100 children aged under 5 years) are likely to impact positively on parents’ understanding and perceptions of the benefits of vaccination. It is also important to ensure that parents receive this information prior to appointments. There is also a pressing need to evaluate the impact of probabilistic information on perceptions of benefit from vaccination and uptake rates. More research is needed to engage parents in an iterative codesign process to design optimally acceptable and usable information that conveys robust and balanced data on the consequences of disease and the benefits and risks of vaccinations.

According to our findings, incentive-based schemes are likely to be optimal in future vaccination service design. However, the cost of rolling out incentive-based schemes may be prohibitive and may prove difficult to justify. Mandatory schemes may, therefore, be the next best alternative for increasing uptake rates for vaccination services, although in this study only a minority of parents preferred these. The factors that may increase acceptance of mandatory schemes warrant further research and additional DCEs could be conducted to explore parental preferences for how a mandate for vaccination might be imposed.

Implications of findings for policy and practice

Differences in predicted uptake rates for the most and least preferred scenarios in the DCE show that the specification of a financial incentive scheme is essential when aiming to effectively increase vaccination rates among the population in general or across specific subgroups. According to our findings, any policy that specifically targets specific groups using incentives would meet significant resistance, as there were strong negative preferences for targeted incentives.

Furthermore, taking into account similarities and differences in preferences expressed by parents ‘at high risk’ and ‘not at high risk’ of completing the full programme could be valuable for any policy directive at improving uptake rates. This will serve to maintain uptake rates in parents who are likely to complete the full programme and potentially increase uptake rates in the high-risk group.

Findings from the DCE suggest that universal rewards would be acceptable and could be made available as a cash payment or voucher (with the latter at an increased monetary value). A public education campaign may also be warranted, given suboptimal intentions to vaccinate, low perceived susceptibility to serious disease if children are not vaccinated and issues related to information provision. However, such campaigns are expensive and frequently do not impact on the actual behaviour of the public. An alternative would be to review and update the information currently provided to include robust, probabilistic information on the consequences of disease and the net benefit from vaccination.

Conclusion

This is the first use of a DCE to elicit parental preferences for the optimal organisation of vaccination services and the relative importance of different service configurations. We found that universal ‘high-value’ rewards, in the form of cash payments for parents, are likely to increase uptake rates within populations ‘at high risk’ of incomplete vaccination without any negative impact on current high uptake rates of those ‘not at high risk’ of incomplete vaccination. The cost of incentives could be offset to some extent by offering additional flexibility in terms of alternative community settings for vaccinations and out-of-hours appointments. The provision of robust and balanced information on consequences of disease and the benefits and risks of vaccination in numerical format prior to appointments is also likely to increase uptake rates. Further research to optimise the provision and quality of such information is warranted. Mandatory schemes may be more acceptable alternatives to incentives, and further research should investigate parental preferences for the organisation of such schemes.

Copyright © Queen’s Printer and Controller of HMSO 2015. This work was produced by Adams et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Included under terms of UK Non-commercial Government License.

Bookshelf ID: NBK326957

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (21M)

Other titles in this collection

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...