Introduction

In a large CRCT (STAMPP) combining an adapted version of the SHAHRP with a brief parental information component, we found that the intervention reduced self-reported HED in the past 30 days at T3 follow-up compared with EAN, but that it did not reduce self-reported harms associated with own drinking. Similarly, intervention pupils reported significantly fewer heavy drinking episodes in total at T2 and T3 than control pupils. There were no clear cost savings in terms of service utilisation associated with the intervention. However, from a health economic perspective, STAMPP was considered to weakly dominate EAN, as it was associated with a cost saving and was more effective.

At first glance, it is difficult to reconcile these apparently inconsistent findings. However, one explanation may lie in the manner in which all three outcomes were assessed and/or analysed. HED was assessed using one question written on a detailed and colourful page including pictures of drinks and corresponding unit content, allowing participants to accurately determine drinking in the past month. With regard to harms, it is possible that the specific harms examined were not age-appropriate. Although they are similar to those used in previous Australian and Northern Irish studies, the participants in the present study were approximately 1 year younger than the participants in both of those studies. On reflection, harms examined such as getting sick, getting in trouble with the police or attending a hospital may have been less relevant than the unasked harms of, for example, losing possessions, losing friendships or damaging their reputation that probably accompany early adolescent drinking behaviour. In addition, not taking frequency of harms into account in the primary analyses may have resulted in a loss of information. This suggests that our primary outcome measure of alcohol-related harm, as specified in the DAP, may not have been sensitive enough to detect age-appropriate harms, and thus was not able to identify any other alcohol-related harm (in addition to harm related to alcohol toxicity) that may have occurred and/or been prevented during the trial.

There were no clear or consistent effects identified in planned secondary or subgroup analyses [age, gender, SES, alcohol use at baseline, location (Scotland vs. NI)]. Stronger claims about the preventative impact of the intervention could be made if a clear pattern of consistent effects across a range of related outcome measures had been identified. It is possible that longer-term follow-up might reveal such effects, especially with regard to self-reported harms, which were low in both control and intervention pupils (the median number of harms was 0 in both arms; 68% of existing drinkers in control schools reported experiencing one or more harms at final follow-up compared with around 62% of existing drinkers in intervention schools). One indication of the potential developmental prevention impact of the intervention is that, at T2, HED showed a small reduction associated with the intervention, an impact that was stronger at T3 follow-up, albeit with a different follow-up measure. It is plausible that the final outcome measure for HED used at T3 was a more sensitive measure than the other measure used at earlier follow-ups (more than five drinks in an episode) and we cannot discount this possibility. The use of mediation analysis to assess lagged effects of changes in hypothesised targeted programme constructs may help to better understand these developmental effects.

In the light of the lack of statistically significant subgroup interaction effects and suggestions of multicollinearity between HED at baseline and the main effects in the model, we undertook additional exploratory analyses outside of the DAP. When the primary outcome models were restricted to just drinkers (defined as either lifetime use or previous year’s use at baseline), drinking pupils in the intervention schools reported fewer harms than those drinking pupils in the control schools. However, it must be noted that when the lifetime use and previous year’s use subgroup effects were examined via interaction terms (on the full CC population), the interaction terms for harms were non-significant, as were the interaction terms for age at onset and unsupervised drinking.

A number of sensitivity analyses were conducted to estimate the impact of analytical decisions on the HED primary outcome. These all supported the chosen analysis, with the exception of the conservative case model (for which missing cases in the intervention arm were all assumed to have engaged in HED and missing cases in the control arm were set to non-drinking), in which the sign of the intervention coefficient changed while remaining significant. This indicated that the model results were robust (i.e. a school effect was found in four of the five tests), except when subject to extreme missing data assumptions (i.e. the conservative case model).

At baseline, 7.8% of control and 7.6% of intervention pupils reported HED (defined as drinking more than five drinks in one episode). The prevalence of HED (using the primary outcome definition) was 26% in the control group and 17% in the intervention group at final follow-up. It is difficult to directly compare these figures with other contemporaneous UK surveys of adolescent alcohol use because of the use of different questions, sampled age groups and recall periods,¹⁰⁶ but data from the control group seem broadly equivalent to other estimates. For example, 53.3% of 11- to 16-year-old pupils surveyed in the 2013 NI Young Persons Behaviour and Attitudes survey reported being drunk on at least one occasion in the previous month. The 2013 Scottish Schools Adolescent Lifestyle and Substance Use Survey¹⁰⁷ reported that 44% of 15-year-olds reported having ever felt ‘really drunk/drunk’ in their lifetime. In England, from the Smoking, Drinking and Drug Use among Young People in England survey,¹² 12.3% of 13-year-olds (equivalent to our baseline sample mean age) reported being drunk (self-defined) in the previous 4 weeks, and this increased to 53% of 15-year-old lifetime drinkers (comparable with our T3 sample). For comparison, in the 2011 pan-European Union ESPAD, 54% of 15- to 16-year-olds reported consuming more than five drinks in a single episode.

A Cochrane review of the effectiveness of universal school-based alcohol education programmes identified 53 studies for inclusion, including 11 alcohol-specific interventions (the remainder were generic approaches or targeted multiple substances).^47,108 No included studies were conducted in the UK. Of the 11 alcohol-specific interventions, six found beneficial effects of intervention. Outcome measures differed between those studies reporting significant findings, although three reported similar outcome measures to those included here (‘binge drinking’). In the original SHAHRP evaluation conducted in Australia, significant intervention effects were reported at 32 months and, although ORs were not presented, it was estimated that intervention pupils were 4.2% less likely to consume at risky levels [defined as reporting two (female)/four (male) standard drinks (10 g of alcohol) per occasion once per month or more often].⁵⁵ A German skills-based activity delivered over four interactive lessons to a similarly aged sample as STAMPP was associated with an OR of lifetime ‘binge drinking’ of OR 0.56 (95% CI 0.41 to 0.77) at 4 months and an OR of 0.74 (95% CI 0.57 to 0.97) at 12 months.¹⁰⁹ Finally, an Australian online alcohol harm reduction curriculum for 13-year-olds (CLIMATE Alcohol Program) reported significant intervention effects (no OR reported) for ‘binge drinking’ (a single occasion over the previous 3 months) for girls but not for boys 12 months after delivery.⁵⁷ Interestingly, and in contrast with STAMPP, self-reported harms were also significantly reduced in girls using the same assessment scale employed in the current study. Finally, for comparison, the earlier NI study of the SHAHRP⁶⁰ (32 months of follow-up), which employed a non-randomised design and used latent class growth modelling, intervention pupils were more likely to be in trajectories that reported fewer alcohol units consumed in the previous episode and fewer alcohol-related harms (no OR reported). Overall, the OR associated with STAMPP (OR 0.596, 95% CI 0.49 to 0.73) suggests that programme effects are comparable with those reported previously for similar interventions using similar outcome measures. However, caution is warranted when drawing comparisons between STAMPP and earlier studies conducted in Australia and NI. STAMPP used a different research design (CRCT vs. quasi-experimental designs in the Australian and NI SHAHRP studies), a younger age group and a different outcome measure at final follow-up. We also incorporated a parental intervention (albeit with low uptake). Furthermore, the earlier NI study was delivered against a higher population prevalence of drinking.

The total cost to deliver STAMPP was £85,900, equivalent to £818 per school and £15 per pupil. NICE published public health guidance⁵³ for use in primary and secondary schools on sensible alcohol consumption, which revealed the paucity of evidence from economic evaluations in this area. The authors of the review identified three studies that provided sufficient information on resource use to allow a cost per student to be calculated. The costs ranged from £20 to £150 (cost year 2005/6).^55,110,111 Thus, at a cost of £15 per pupil, STAMPP is a relatively low-cost intervention that successfully reduces HED. These costs reflect the cost of introducing and delivering STAMPP on one occasion. In reality, once teachers have been trained they will not need to be retrained on an annual basis, requiring only refresher training. Similarly, if STAMPP was delivered instead of existing alcohol EAN, which would likely be the case based on the findings of our process interviews, the cost per pupil would also fall.

The analysis of public service costs showed that there was an overall reduction in the use of public sector services over the 33-month study period for both groups. There were, however, no differences between groups and no differences in the use of the subcategories of education, health and criminal justice services. Costs were estimated from a public sector perspective, which was justified considering one of the principal objectives of STAMPP was to reduce alcohol-related harms in teenagers. It was hypothesised that this would in turn reduce the use of health and judicial services and the need for additional support within the school setting to address behavioural and emotional problems. The absence of statistically significant difference in public service costs between groups is in keeping with the analysis of the harms data from the trial [see Chapter 3, Drinking harms (T3)]. No differences were observed in the number of self-reported harms by pupils between groups and, indeed, both groups reported low levels of harms overall.

The primary CEA (using the number of pupils experiencing a heavy drinking episode in the previous 30 days) at T3 indicated that STAMPP weakly dominated EAN. At a notional WTP threshold of £15 (reflecting the cost of STAMPP per pupil observed in this study), the probability of STAMPP being cost-effective was 56%. This level of uncertainty reflects the considerable variability in the cost differences between groups. At T2, this probability of cost-effectiveness was considerably lower, at 35%, because of the additional variability in the effectiveness of the intervention. A similar pattern of results was observed in the secondary CEAs (using the number of heavy drinking episodes); the intervention was more cost-effective at T3 than at T2. However, the probability of cost-effectiveness was lower at each time point (45% and 32%, respectively, at a WTP of £15) than in the primary CEA. This was because of greater variability in both costs and outcomes. The greater variability in effectiveness is not surprising considering that the effectiveness measure used in the secondary CEAs was based on a continuous outcome, in contrast to the binary outcome used in the primary analysis. Overall, the implication is that STAMPP is more cost-effective in the longer term, as it has a greater impact on pupils when they are older, more likely to drink and drinking more. In light of the literature, which links heavy drinking in adolescence to alcohol dependence and poor health outcomes in adulthood (e.g. Bonomo et al.²), it is important to investigate if the (cost) effectiveness of STAMPP is sustained or even increases in the long term.

The sensitivity analyses indicated that the results of the CEA were robust to small changes in the parameters, that is, discounting and small increases in cost and effectiveness. However, when costs and effects were not adjusted for baseline covariates, the probability of STAMPP being cost-effective increased. This suggests that the cost-effectiveness of STAMPP may vary between subgroups and warrants further investigation to identify which pupils and/or schools might benefit the most from the receiving the intervention. Furthermore, when multiple imputation was used to impute missing cost and outcome data, the cost-effectiveness of STAMPP decreased. Considering multiple imputation is based on the assumption that data are missing at random, it is of some concern that the results of this sensitivity analysis differ somewhat from the primary analysis. The proportions of missing data in different groups were similar within all of the health economic analyses; thus, further investigation is warranted into the imputation model used as well as the pattern of missingness.

The process evaluation showed that clusters were successfully recruited into STAMPP, randomisation was successful, and pupils in schools were comparable across intervention arms at baseline. No adverse events were reported. The intervention was delivered with a good degree of fidelity and was enjoyed and/or acceptable to students, teachers, schools and other stakeholders. Overall, pupils thought that the intervention content was age appropriate, although from the comments received in some focus groups (and from the survey data), even within the same year group, pupils will have different drinking histories and there will be differences in rates of initiation and in establishment of more regular drinking patterns. The standard materials provided to pupils, in particular those focusing on the consequences of drinking, may therefore need to be adapted for some target groups if the intervention is delivered in routine practice. Furthermore, although participants valued the workbooks, pupils have changing expectations with regard to modes of delivery of learning materials, particularly with regard to new technologies and platforms, and so intervention materials may need to be adapted to keep pace with changes in learning platforms (e.g. electronic materials delivered through online platforms or tablets).

In contrast, there was very low uptake of the parental/carer component, and postal returns of the parent/carer survey, which were used as an indicator of implementation of mailed intervention materials, were also relatively low. This component was therefore not successfully delivered. We did not have a comprehensive response to the teacher survey assessing EAN practice in control schools, but reports from responders indicated that EAN was rarely delivered (n = 3 reports) and primarily consisted of single sessions of general alcohol awareness activities delivered by external organisations (e.g. police, alcohol charities). Therefore, we concluded that there was a clear differentiation between intervention and control schools.

Strengths and limitations

The key strengths of the trial were the large sample size (schools and pupils), low rates of attrition (no schools dropped out) and relatively high rates of matched data (> 80% depending on outcome) across survey waves. This means that the primary analysis on HED was sufficiently powered. This calculation used estimates of HED derived from the 2011 ESPAD study¹³ for the same age as the STAMPP pupils, with the estimated ICC derived from the Belfast Youth Development Survey.⁷⁵ Unfortunately, neither the Belfast Youth Development Survey nor the ESPAD contained comparable measures of drinking harms. The NI SHAHRP study,⁶⁰ which did use a self-report harm measure, was undertaken with an older age group of pupils, and so was also not an appropriate base for a sample size calculation. Therefore, as no formal sample size calculation was undertaken for alcohol-related harms, there is the possibility that the null result for this outcome was because of lack of power. However, given the large achieved sample size, the relatively low levels of subject attrition, the use of covariates within the models (providing additional power) and the relatively small observed differences between study arms, we do not suspect that the null finding was because of insufficient sample size. It is more likely that the self-reported harm measures were not sensitive enough to likely harms experienced by the participants and thus any potential benefit was hidden. The classroom component was delivered with acceptable fidelity and was positively received by both pupils and teachers. However, there was a higher dropout rate (i.e. non-matched data across T0 to T3) among pupils who were male (19%), who were in receipt of FSMs (25.8%) or who had used alcohol at baseline (25.4%). Although we controlled for these variables in our analyses, it is uncertain why there was a higher level of missing data in these groups. Sex, SES and previous alcohol use have been shown in other UK studies to be predictors of school non-attendance (e.g. truanting, exclusion),^12,112,113 although this would account for only some of the missing data. We were unable to identify previous studies examining predictors of retention or missing data in UK adolescent alcohol prevention interventions, but in the Australian and NI trials of the SHAHRP, there was an overall attrition of 24.1% and 12.8% at 32 months, respectively.^58,60 In the trial of the Dutch Preventing Heavy Alcohol Use in Adolescents (PAS) programme, from which our parental intervention was derived, overall attrition was 12.5%, and dropouts differed from completers in being older, drinking more and having parents with lower levels of education.⁶⁴ A systematic review of universal school-based interventions concluded that there was no difference between effective and non-effective interventions on the basis of attrition.¹⁰⁸ It would therefore be important for future work to determine why baseline drinking groups in particular produced more missing data, as this group would potentially benefit most from alcohol interventions, and inclusion in the data set may have adjusted our analyses.

A major limitation of the work was the failure to attract parents/carers to the brief intervention evening (9% in NI and 2.5% in Scotland), despite the support of many of the schools. Relatively low rates of return of the parental questionnaire (31% and 18%, respectively) also suggested that only a minority may have read the mailed information. Although we conducted an ITT analysis, which helped to preserve sample size, and achieved participation rates are likely to reflect family attendance in routine practice,^114–116 this meant that we were unable to draw any confident inferences about the combined impact of the school and parental intervention (see Koning et al.¹¹⁷) or the relative contribution of each component. In practical terms, this means that although the analyses presumed delivery of the combined intervention, discussions with stakeholders about research findings and future delivery are likely to focus on the classroom component (i.e. SHAHRP).

Failure to engage parents/carers in school-based substance use prevention is a consistent finding.^118–122 However, other trials have reported success at engaging family members. For example, in a recent feasibility study of the Welsh family-based alcohol prevention intervention Kids, Adults Together programme,¹²³ 50% of pupils (n = 158 intervention pupils in total) reported that at least one family member who was invited to a family event attended (although only 6.5% of eligible parents/carers returned a study questionnaire), suggesting that acceptable participation rates in the UK are achievable. These authors identified two key processes that they believe supported engagement. First, pupils were keen to attend the event with their parents/carers and, second, the family event was not marketed as an alcohol education event and was positioned around parents/carers wanting to attend the event to see their children’s school work and what activities they had been involved in. Similarly, in the Dutch PAS study there was a high level of parental retention in the parent only (75.9%) and combined parent and student (72.4%) intervention arms.⁶⁴ In keeping with the Kids, Adults Together programme, PAS parental events were part of regular school parents’ evenings, which a large number would have attended anyway. Future implementers of STAMPP should therefore consider such engagement approaches, which were not feasible in the current trial because of the timing of intervention delivery, the large number of schools involved which made co-ordination difficult, a lack of time within regularly planned parents evenings (which primarily focus on pupil progress) and education policy initiatives in one trial site that necessitated using parental evening time to introduce a new curriculum.

Our primary outcome assessments relied on self-report, which may have led to inaccurate reporting of alcohol use and associated harms through memory, social desirability and other biases.¹²⁴ Although adolescent self-reported alcohol questionnaires are generally reliable,^125,126 there may be differences in reliability between early and late adolescence⁶³ and studies of recanting in substance use surveys suggest that this may be an understudied bias in prevention research.¹²⁷ However, all pupils received the same questionnaire and pictorial prompts, and the recall period for the primary outcome used in this study was the previous 30 days, and so, if bias had existed, this would have been minimal and equivalent across trial arms. Less attention has been paid to the validity of assessments of alcohol-related harms, although similar social desirability and self-representation biases are likely to exist.¹²⁸ In this study, alcohol-related harms were measured using a 16-item scale, and previous work has shown the scale to have an internal consistency of α = 0.9. However, we do not know if pupils consistently interpreted the harms in the same way. Although some of the self-reported harms were likely to be interpreted in a straightforward manner (e.g. ‘did you vomit after drinking?’), similarly to differences in young people’s self-perception of drunkenness,¹²⁹ there may be individual differences in interpretation of the harms assessed in this study and different thresholds applied for an indicator being perceived as being a ‘problem’ (e.g. having a hangover after drinking). As mentioned previously, it is also possible that some of these harms were not age appropriate and thus these were low frequency in our population. Without a method for objectively verifying the level of harm in this population, it is difficult to know whether or not these self-reported data were biased.

The assessment of HED (or ‘binge drinking’) in adolescents, as used in our consumption primary outcome, is complicated by the lack of standardised definitions in both adults and adolescents.¹³⁰ We adapted the current CMO’s guidelines for adults⁶⁵ for this study, but regardless of this problem we were able to show a reduction in our measure of alcohol consumption. The introduction of the improved pictorial response sheet no doubt enhanced the accuracy of responses and reduced any potential bias given the problems with the concept of a ‘drink’ for not only the participants but for the research community as a whole.

There were a number of limitations to the economic evaluation. The study was not specifically powered to detect statistically significant differences in costs or cost-effectiveness. Although CEA does not typically make decisions based on significance rules,¹³¹ having a sufficiently powered study will allow decision-makers to be more confident in the value claim.¹⁰⁴ The resources used during the planning, preparation and delivery of the intervention were largely recorded retrospectively, and costs were obtained from invoices when these were available. We endeavoured to use plausible assumptions when actual data were not available, but the consistent and prospective collection of resource use and costs would lead to more robust data. The resource use questionnaire was completed by the pupils without any input from their parents or guardians. This was done because of resource limitations and to preserve confidentiality, and although definitions of the services were provided and the terminology simplified, it could be argued that more accurate costs would have been obtained with parental input. However, it was difficult to engage parents in the intervention, as reflected in the poor attendance to the parental evenings (see Chapter 4, Fidelity of implementation of STAMPP), so it is likely that response rates would have been poor.

We included pupils in the CEA only if they had complete cost and effect data. As a result, only two-thirds of the pupils were included in the T3 CEA and three-quarters in the T2, and the rest we assumed were missing at random. As discussed earlier, further investigation is required to establish whether or not this assumption is flawed.

The curriculum was delivered in most schools as part of their PSHE education (or local equivalent) curriculum and did not replace statutory activities. However, we did not assess spillover effects of STAMPP on other types of related school activity or curriculum, and so this must be considered a limitation of the trial.

Our approach to assessing fidelity of implementation and comparator bias was pragmatic in the context of the resources available and the large number of schools enrolled in the trial. Although our assessment of fidelity was based on an existing framework,¹³² and provided useful information, ideally, in addition to self-report we would have preferred to have recorded and/or observed some classroom and parent/carer sessions for independent rating of deliverer competencies, quality and completeness of delivery and target group responsiveness. Similarly, although we are confident that we identified other alcohol actions delivered to schools and we are able to conclude that delivery of competing interventions was very low, we were unable to make comparable assessments of exposure to community-based alcohol activities such as mass media campaigns and health promotion with an alcohol component. However, we were unaware of any major initiatives being delivered across the course of the trial, and the successful randomisation would have militated against some effects.

Other weaknesses and limitations of the research are identified in Self-assessment of risk of bias.

Further research

We found that STAMPP was effective in reducing self-reported HED T3, when pupils were aged 15–16 years. It will be important to assess whether or not these reductions are sustained and if effects on harms emerge as pupils get older, alcohol use behaviours become more frequent and patterns of use are established.^12,133,134 Booster interventions (e.g. brief interventions of proven effectiveness in adolescents and young adults) that build upon the skills developed in STAMPP may prove useful in sustaining behavioural change.¹³⁵ As the trial team included those who were responsible for the adaptation and development of the intervention, it is also important that any future replications are conducted independently.¹³⁶ The use of data linkage techniques to match the study cohort to additional educational, community and statutory service data sets (e.g. examination performance, hospital admissions, GP data, involvement with the criminal justice system) may also help to understand whether or not the changes reported in the current study lead to meaningful changes in health and well-being outcomes.¹³⁷ This recommendation also applies to other alcohol prevention research that predominantly uses simple proxy outcomes of use,⁴⁹ and this practice has been criticised for not providing useful information for commissioners who are tasked with funding interventions on the basis of demonstrable improvements in health and well-being.

In an earlier secondary analysis assessing implementation of the adapted SHAHRP in NI,¹³⁸ multilevel growth modelling was used to examine differential intervention impact when recipients were retrospectively classed according to alcohol use status (abstainer or existing drinker) and context of use (unsupervised or parental/carer supervised drinker) at baseline. Significant positive behavioural effects in terms of amounts consumed, frequency of drinking and self-reported alcohol-related harms were observed almost exclusively among pupils classed as baseline unsupervised drinkers. This was notable, as although the SHAHRP was delivered as a universal curriculum, it suggested that it might also have utility as a targeted intervention. In the current study, the preplanned subgroup analysis showed that there was no interaction between baseline drinking status and treatment effect. However, when exploratory analyses were undertaken, which examined drinking status groups independently, significant intervention effects emerged (HED and self-reported harms). These findings suggest that STAMPP may have had a differential impact on those pupils who would be considered most at risk from alcohol use (e.g. unsupervised baseline drinkers).¹³⁹

With respect to further development of STAMPP intervention, our rich data set means that it will be possible to conduct mediation analysis to further develop programme theory and to test the assumptions of our logic model (see Appendix 1). This may lead to a better understanding of which components of the intervention (e.g. specific lessons) were most successful and which require strengthening.¹⁴⁰ Identifying supportive or inert elements of the programme may lead to the development of a shorter optimised curriculum, which would reduce resource requirements and potentially increase the attractiveness of the intervention to funders. Similarly, analysis of moderation effects might identify local contextual and population factors that exert differential influences on outcomes.¹⁴¹

Extending this line of work, research examining the fidelity of implementation in more detail may help to refine delivery. Although we reported relatively high completeness of delivery with respect to content, there was variation in the number of lessons required to deliver, and it is uncertain what effect this may have had on programme outcomes. In a secondary analysis of the European Drug Addiction Prevention (EU-Dap) trial substance use prevention curriculum, another skills-based interactive prevention programme, class size, composition (e.g. sex ratio, academic ability) and social connectedness between pupils, were shown to be important predictors of programme implementation.¹¹⁴ Understanding these factors is important, because in routine practice, outside of the structures of a RCT, the intervention may not be delivered as intended, and formal and informal changes introduced by delivery staff may lead to a loss of programme integrity.⁹⁰ Furthermore, although based in the classroom, the adapted SHAHRP curriculum may not necessarily be optimally delivered by teachers,⁶⁰ and some pupils in the process evaluation suggested that their response to the lessons was dependent on pre-existing relationships with school staff. In future programme development and evaluation, different trial arms should include the assessment of alternative deliverers, such as trained prevention and youth service workers, who have specialist skills to help better engage young people in health programmes.

Considering our failure to recruit into the parental intervention, further research is required to better understand how to engage and retain parents/carers in prevention activities.^91,123 This is also important, as delivery of preventative activities outside of the structures of research trials frequently leads to lower implementation quality.⁹⁰ Universal interventions such as STAMPP require a range of recruitment strategies, as there will be different barriers to, and facilitators of, attendance in parental/carer-based actions. Research is therefore needed to assess the relative efficacy of recruitment strategies such as incentives, mass media campaigns, the removal of barriers to attendance (e.g. providing transport and childcare) and the use of key community recruiters (influential individuals and organisations).⁹¹ Furthermore, it is also important to understand if some parent/carer subgroups (e.g. differentiated according to child drinking risk) are more likely to respond to particular recruitment strategies and if this will lead to recruitment biases.

Broadening our research recommendations to the wider prevention field, and drawing upon our stakeholder interviews, it is clear that although universal interventions are valued, they compete for resources and must sit alongside other alcohol-related community actions and policy initiatives, which may moderate observed effects.⁹² By-laws and licensing decisions can affect local alcohol environments (e.g. density of alcohol outlets, opening hours and local marketing), and actions such as industry-driven corporate social responsibility initiatives, marketing and packaging regulations and community-based initiatives may complement or disrupt school-based actions.^93–97 There is also increasing interest in schools as environments for health promotion through actions that modify the physical and social environment.⁹⁸ Interventions and curricula, such as STAMPP, are therefore not being delivered in isolation, and, although study design characteristics such as randomisation control for internal biases, there is a need to better understand how interventions complement each other, the dynamic interplay between intervention components at different levels (e.g. the interaction between norms correction activities in a prevention curriculum and the placement of alcohol advertisements in the local area), and how interventions can be optimised within such complex health systems.⁹⁹ The use of systems mapping exercises and the study of alcohol prevention as a complex system may be one means to maximise effects from combinations of different prevention types.¹⁰⁰

Implications for practice

Our findings suggest that STAMPP is a candidate for inclusion in local strategies to reduce alcohol-related harm where reduction in HED is a stated aim. Although the reported outcomes were relatively modest, STAMPP is one of the few UK school-based alcohol prevention programmes to show effectiveness in reducing HED in adolescents. Although any universal prevention programme on its own is unlikely to lead to sustained changes in population levels of alcohol use,^92,101 the harm reduction focus of the classroom component of STAMPP may complement national and local actions targeting price, availability and affordability of alcohol and the licensing of alcohol outlets. Furthermore, although we have not yet conducted such an analysis, previous studies of universal prevention programmes have also shown benefits for participants considered at greater risk of harm, whether as a result of their substance use behaviour or population characteristics (e.g. Vigna-Taglianti et al.¹⁰² and Spoth et al.¹⁰³). With appropriate adaptation (e.g. language and comprehensibility, delivery outside mainstream classrooms), STAMPP may be a useful form of alcohol education for higher-risk and vulnerable groups.

The classroom intervention was easily implemented by teachers (conditional on support by school management) after suitable training and support materials were relatively inexpensive (workbooks and supporting CD/electronic material). Schools that have identified alcohol education as a priority would be in a good position to offer the classroom curriculum as part of existing PSHE provision, and teachers in this subject are likely to have the required general professional skills to facilitate delivery. Although we acknowledge curriculum pressures, considering the current poor state of substance use education in the UK,^104,105 STAMPP would make a useful contribution to health and social education in schools.

However, in the current trial, uptake of the brief intervention was poor, and although all intervention students parent(s)/carer(s) received the intervention leaflet, the return of materials to indicate reading/acknowledgement was low. It is, therefore, difficult to make any practice recommendations about this component of the programme. If commissioners or providers decide to implement STAMPP in future, then they must decide whether the full programme or only the classroom intervention is delivered. Relying on passive mechanisms (e.g. advertisements and information) is unlikely to encourage attendance and, although some of the techniques to encourage participation discussed in this section are likely to be costly (e.g. providing transport and childcare), others rely more on planning than resources (e.g. scheduling events as part of regular school parents’ evenings).

Generalisability

Although we are mindful of differences in school autonomy, governance and oversight, and we acknowledge regional variability in alcohol use behaviours (e.g. Public Health England⁹), we believe that the findings of this trial are generalisable to other geographies of the UK. However, as discussed throughout this report, parental/carer engagement was poor, and so generalisability may only be relevant to the classroom curriculum. Schools enrolled in the trial were drawn from urban and more rural areas, and from across the socioeconomic gradient. Furthermore, subgroup analysis showed that there were no differential intervention effects on the basis of school geography (i.e. NI vs. Glasgow/Inverclyde). Interviews with stakeholders, including local commissioners, did not identify any significant barriers to future delivery, beyond those expected relating to funding and local priorities. Similarly, teachers and senior school staff believed that STAMPP would help them to achieve their health and well-being aims in line with wider school policies.

Self-assessment of risk of bias

We conducted a self-assessment of bias in accordance with criteria adapted from the Cochrane Collaboration methodological handbook.¹⁰⁶ For each domain, TMG members assessed the level of bias with respect to three outcomes: unclear risk, which is interpreted as plausible bias that raises some doubt about the results; low risk, which is interpreted as plausible bias that is unlikely to seriously alter the results; medium risk, which is interpreted as plausible bias that moderately weakens confidence in the results; and high risk, which is interpreted as plausible bias that seriously weakens confidence in the results. Final ratings were achieved through consensus. We identified a medium risk of bias in relation to performance bias, as this was not a double-blind study. However, using an EAN comparator, it was not possible to conceal intervention allocation from teachers, who received specialised training and curriculum materials, or from pupils, who would typically receive little or no alcohol education in their usual school year. We self-rated detection bias as having a medium risk. This was because of resource constraints and because some of the data collection was undertaken by members of the trial team. We self-assessed conflict of interest bias as having a medium/high risk for two main reasons. The first reason was that the printing of curriculum workbooks in one intervention site (Glasgow only) was funded through dedicated money awarded by the alcohol industry. However, this funder was not involved in any aspects of research or intervention design and did not have any subsequent involvement in the trial, its management, analysis or write up. The second reason underpinning this rating was related to the involvement of three members of the trial team (including the principal investigator) in the original NI adaptation of the school curriculum, and all members of the TMG contributed to the development of the parental component. Finally, two members of the TMG reported that their university departments had historically received research grants from parts of the alcohol industry for unrelated research. Neither had individually directly benefited from this funding and it is reported here for transparency.