Home > Executive Summaries > The effectiveness and cost-effectiveness...

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.

NIHR Health Technology Assessment programme: Executive Summaries. Southampton (UK): NIHR Journals Library; 2003-.

NIHR Health Technology Assessment programme: Executive Summaries.

The effectiveness and cost-effectiveness of behavioural interventions for the prevention of sexually transmitted infections in young people aged 13–19: a systematic review and economic evaluation

J Shepherd, J Kavanagh, J Picot, K Cooper, A Harden, E Barnett-Page, J Jones, A Clegg, D Hartwell, GK Frampton, and A Price.

Author Information

J Shepherd,1,* J Kavanagh,2 J Picot,1 K Cooper,1 A Harden,2 E Barnett-Page,2 J Jones,1 A Clegg,1 D Hartwell,1 GK Frampton,1 and A Price3.

1 Southampton Health Technology Assessments Centre (SHTAC), UK
2 The Evidence for Informed Policy and Practice Information and Co-ordination Centre (EPPI-Centre), Social Science Research Unit, Institute of Education, University of London, UK
3 Wessex Institute, University of Southampton, UK
* Corresponding author

Published: 2010.

Background

Rates of sexually transmitted infections (STIs) continue to increase, particularly amongst young people. STIs can be either bacterial (e.g. chlamydia, gonorrhoea) or viral [e.g. human immunodeficiency virus (HIV), genital herpes, human papillomavirus]. Interventions to encourage young people to adopt and maintain safer sexual behaviour are one approach to preventing STIs and promoting sexual health. The prevention of STIs and teenage pregnancy is a high priority for health policy because of the adverse impact on individuals and on health service resources. We conducted a systematic review and economic evaluation to assess the effectiveness and cost-effectiveness of behavioural interventions for the prevention of STIs in young people.

Methods

Systematic review of effectiveness

A two-stage process was followed: (1) development of a descriptive map of the key characteristics of studies evaluating behavioural interventions, followed by (2) a detailed systematic review of a subset of interventions.

Search strategies Electronic bibliographic databases (for example, MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, PsycINFO and CINAHL) were searched for the period 1985 to March 2008. Bibliographies of systematic reviews and related papers were screened and experts contacted to identify additional published and unpublished references.

Study selection Titles (and abstracts, where available) were screened for eligibility by one reviewer using a priori inclusion criteria. Studies eligible for inclusion in the descriptive map were: controlled trials, evaluating a behavioural intervention (defined as any activity to encourage young people to adopt sexual behaviours that would protect them from acquiring STIs), in young people aged 13–19 years, which reported a sexual behavioural outcome. Full papers were obtained for those abstracts and/or titles that appeared relevant, and these were screened by two reviewers independently.

Descriptive map Keywords were developed and systematically applied to included studies to produce a detailed map of the evidence base that was used to prioritise a subset of studies for inclusion in the systematic review in consultation with stakeholders.

Data extraction and quality assessment Two reviewers independently quality assessed the studies included in the systematic review. Differences in judgement were resolved by discussion and involvement of a third reviewer if necessary. Outcome data from the studies that were judged to be methodologically sound were extracted by one reviewer and checked by a second. Process evaluation data were coded by two reviewers and classified into higher-order themes.

Data synthesis Studies were synthesised in both a narrative synthesis and meta-analysis.

Process evaluation Findings from process evaluations that had been conducted alongside the included original randomised controlled trials (RCTs) were summarised narratively.

Economic evaluation

A systematic review was conducted of economic evaluations of behavioural interventions for the prevention of STIs in young people. A number of electronic bibliographic databases (for example, CCRCT, MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, NHSEED and DARE) were searched for the period 1990 to February 2008, and references screened according to a priori inclusion criteria.

An economic model was developed to compare the costs and consequences of behavioural interventions for the prevention of STIs in young people. The cost-effectiveness of two types of behavioural intervention (teacher- and peer-led school-based intervention) compared with standard sexual health education was assessed.

A Bernoulli statistical model was constructed, which described the probability of STI infection based upon STI prevalence; single-act transmission probability; condom effectiveness and condom use; number of sexual episodes; and number of sexual partners. The parameters for the model were derived from a systematic search of the literature on the natural history and epidemiology of STIs; sexual behaviour and lifestyles; health-related quality of life; and costs. Costs were derived from primary data from previous studies, and national and local NHS unit costs. The analysis was conducted from the perspective of the NHS and Personal Social Services. In the model, the intervention effects last for 1 year, on the basis that the majority of the trials included in our systematic review assessed outcomes up to 1 year. The model estimates the lifelong costs and benefits from averted STI cases.

The model estimates the probability of becoming infected for the intervention and comparator groups according to changes in parameters that may be affected by the intervention (i.e. condom use, number of sexual partners, number of sexual episodes). The number of cases averted is estimated by multiplying the reduction in risk of STI infection by the number of people who receive the intervention. The total number of STI cases averted, and consequent quality-adjusted life-year (QALY) gain and saving in medical costs is estimated for males and females for all STIs for 1 year.

Results of the systematic review of effectiveness

A descriptive map of 136 studies meeting the inclusion criteria was produced. The results illustrated the predominance of North American trials of educational interventions conducted in schools with young people targeted primarily because of their age.

Discussion with the project’s advisory group enabled the prioritisation of a policy-relevant subset of studies for systematic review. To be included studies had to be an RCT; evaluate a behavioural intervention including factual information on STIs, in addition to an element of skills development for negotiation of safer sex; be delivered in a school; and report a sexual behavioural outcome (in addition to other outcomes).

A total of 15 RCTs met the inclusion criteria for the systematic review. The majority were conducted in the USA, with only two in the UK. Of the 15 RCTs, 12 were judged to be methodologically sound and were included in the analysis of effectiveness. Studies reported on five main types of behavioural outcome: initiation of sexual intercourse; condom use; sexual intercourse; contraception and pregnancy; and sexual partners. Outcome data for variables that may mediate behavioural change were also often reported: knowledge; skills and self-efficacy; attitudes; and behavioural intentions. Rates of infection were not reported.

Five studies contributed data on sexual initiation. Three of the five studies found that there was no significant difference for this outcome between the intervention and the comparison group. Two studies reported a statistically significant difference in favour of the intervention, although in one the difference was only observed for girls in the peer-led group, which was compared with a teacher-led group. Data from four of these studies could be combined in a meta-analysis, the odds ratio (OR) of 1.03 [95% confidence interval (CI) 0.74 to 1.43] indicated no overall significant difference between groups.

A variety of condom use outcomes were reported. Statistically significant effects in favour of the intervention group over the comparison group were only reported by two of the studies, with one further study reporting a statistically significant effect in favour of the intervention for a subgroup of participants. A meta-analysis was conducted for the general outcome of condom use (an outcome incorporating some of the various measures of condom use). The fixed-effect OR for the combined effect was 1.07 (95% CI 0.88 to 1.30), again indicating no significant difference overall between groups.

For the remaining behavioural outcomes of sexual intercourse, contraception and pregnancy, and number of sexual partners there were very few statistically significant differences between intervention and comparators. The interventions did not lead to a significant increase in initiation of sexual activity by young people, or to an increase in the number of their sexual partners.

The success of the skills component of interventions was generally assessed by self-efficacy measures. Eight of the 12 studies reported a self-efficacy measure, most commonly condom use self-efficacy which was reported by seven studies. Refusal or abstinence self-efficacy (n = 6 studies), communication/negotiation self-efficacy (n = 4 studies), and situational self-efficacy (n = 2 studies) were also reported by some studies. Statistically significant effects were reported for some, but not all, of the self-efficacy measures assessed. All the methodologically sound studies included a knowledge outcome measure, and statistically significant effects in favour of the intervention group over the comparison group were found by all but two of the studies.

Eight of the methodologically sound studies included an assessment of attitudes among their outcomes, and six studies reported participants’ intentions, with a variety of different attitudes and intentions being assessed. However, few studies reported statistically significant effects in favour of the intervention group for these outcomes.

Nine of the 12 methodologically sound RCTs conducted a process evaluation. Synthesis of the process findings to explore reasons for the limited impact of school-based skill-development interventions revealed two sets of factors. Firstly, interventions were not always implemented as intended. Variation in implementation was affected by whether or not there was a supportive school culture, flexible school administration, and enthusiasm and expertise from teachers and peers for delivering interactive sexual health sessions such as role plays. Secondly, not all young people found the interventions as engaging or as acceptable as they might have done. The qualities of the intervention providers – namely enthusiasm, credibility and expertise (in content and in managing classrooms) – was one factor that influenced whether or not young people found the interventions to be acceptable and engaging. Other factors were whether the interventions met young people’s own needs in relation to sexual health, including sexual feelings, emotions and relationships, the operation of gendered norms, the age appropriateness of the intervention and the level of discomfort felt in the classroom setting.

No conclusions could be drawn about the impact of the interventions on sexual health inequalities due to a lack of relevant data in the primary studies on factors, such as socioeconomic status (SES), gender and ethnicity.

Results of the economic evaluation

Systematic review of cost-effectiveness studies

Five economic evaluations of behavioural interventions for the prevention of STIs in young people were included. The studies were conducted in the USA and focused on the prevention of HIV. All studies used mathematical models extrapolating the changes in sexual behaviour. All interventions were effective at encouraging safer sexual behaviour in the study groups and thus led to cases of HIV averted. As the studies used a wide range of assumptions and parameter values in the mathematical models, substantial differences in the estimated cost-effectiveness of the behavioural interventions were reported.

Modelled cost-effectiveness analysis

An economic model was developed to assess the cost-effectiveness of behavioural interventions for preventing STIs in young people. However, as our meta-analysis did not show a statistically significant intervention effect, the results presented should be treated with caution and only be regarded as illustrative. The costs of teacher- and peer-led behavioural interventions, based on the resources estimated from the relevant RCTs in our systematic review, were £4.30 and £15 per pupil, respectively. We assumed the same benefit for teacher- and peer-led interventions. The teacher-led interventions were cheaper because of the need to train a new cohort of peers each year, whereas the teachers are only likely to need retraining after a number of years.

For a cohort of 1000 boys and 1000 girls, aged 15 years, the model estimated that the behavioural interventions would avert three STI cases and save 0.5 of a QALY. The incremental cost-effectiveness of the teacher- and peer-led interventions was £20,223 and £80,782 per QALY gained, respectively. Sensitivity analyses show the results were most sensitive to the intervention effect (condom use), the STI transmission probability, and the number of sexual partners in the base-case analysis. The model results were also sensitive to changes to the model parameters for chlamydia and especially for parameters related to tubal infertility. In a probabilistic sensitivity analysis, the probability of the teacher-led intervention being cost-effective was 46%, where a decision-maker is prepared to pay £20,000 per QALY and 54% at a threshold of £30,000 per QALY.

At a cost-effectiveness threshold of £20,000 per QALY, the population expected value of perfect information (EVPI) is £12.5M, assuming a 10-year lifetime for the intervention (i.e. the time until a new intervention supersedes or replaces it). This constitutes an upper limit on research expenditure to reduce decision uncertainty. An analysis of the individual parameters used in the model revealed that research would be best funded to assess the intervention effect for condom use from a school-based behavioural STI intervention.

Conclusions

School-based behavioural interventions which provide information and teach young people sexual health negotiation skills can bring about improvements in knowledge and increased self-efficacy. However, in this systematic review there were few significant differences between the interventions and comparators in terms of changes in behavioural outcomes, such as condom use. The studies conducted relatively short follow-up assessments at a time when many young people were only just becoming sexually active. It is possible that favourable behaviour change may have occurred with time, particularly as sexual activity becomes more routine in young people’s lives.

The results of the economic evaluation are considered illustrative primarily due to the uncertainty around the effect of intervention on behavioural outcomes, but also due to limitations in the data for other input parameters. The results were most sensitive to changes in parameter values for the intervention effect, the transmission probability of STIs and the number of sexual partners. Teacher-led interventions are likely to be cheaper than peer-led interventions due to less frequent need for retraining. Behavioural interventions for young people potentially may become more cost-effective as they get older and a greater proportion become sexually active.

Implications for practice

Policy-makers and practitioners should be cautious in their expectations about the impact of such interventions on sexual behaviour and incidence of infection. Nonetheless, school-based skills-building behavioural interventions can be effective in influencing behaviour-mediating outcomes, such as knowledge, attitudes and self-efficacy. This is in accordance with current UK government health policy, which stresses the need to provide high-quality information to enable young people to make informed decisions. Interventions need to be culturally relevant and context specific, taking into account the needs of subgroups of young people (e.g. young men, young women) and, where possible, be part of a whole school approach to sexual health promotion. Young people will benefit from being involved as equal stakeholders in the design and delivery of interventions. Providers of school-based interventions need to be enthusiastic and credible, with considerable expertise in classroom management and the delivery of skills-building activities, such as role plays and group discussions. A supportive school culture is also important.

Implications for research

If further primary evaluation of behavioural interventions is to be conducted there should be long-term follow-up to assess the extent to which safer sexual behaviour is adopted and maintained into adulthood. The impact of booster sessions should be further evaluated. All trials should be accompanied by rigorous process evaluation to assess the factors that contribute to success or failure, and economic evaluations to assess cost-effectiveness. Where appropriate, trials should collect, analyse and report data on the likely effects of the intervention on sexual health inequalities. Other markers of risk reduction (e.g. STI testing) should be measured.

For many of the parameters for the economic evaluation there were no available data for the < 16-year-old age group and we have had to make assumptions to extrapolate data from older age groups. Data on the sexual behaviour of under-16s is therefore needed. Furthermore, there is a need for prospective cohort studies to determine the parameters that describe the transmission of STIs between partners. The analysis of EVPI suggested an upper limit of £12.5M on funding for further research to reduce decision uncertainty, which should focus on the effectiveness of interventions on changing behaviour (e.g. increasing condom use).

Publication

  • Shepherd J, Kavanagh J, Picot J, Cooper K, Harden A, Barnett-Page E, et al. The effectiveness and cost-effectiveness of behavioural interventions for the prevention of sexually transmitted infections in young people aged 13–19: a systematic review and economic evaluation. Health Technol Assess 2010;14(7). [PubMed: 20178696]

NIHR Health Technology Assessment programme

The Health Technology Assessment (HTA) programme, part of the National Institute for Health Research (NIHR), was set up in 1993. It produces high-quality research information on the effectiveness, costs and broader impact of health technologies for those who use, manage and provide care in the NHS. 'Health technologies' are broadly defined as all interventions used to promote health, prevent and treat disease, and improve rehabilitation and long-term care.

The research findings from the HTA programme directly influence decision-making bodies such as the National Institute for Health and Clinical Excellence (NICE) and the National Screening Committee (NSC). HTA findings also help to improve the quality of clinical practice in the NHS indirectly in that they form a key component of the 'National Knowledge Service'.

The HTA programme is needs led in that it fills gaps in the evidence needed by the NHS. There are three routes to the start of projects.

First is the commissioned route. Suggestions for research are actively sought from people working in the NHS, from the public and consumer groups and from professional bodies such as royal colleges and NHS trusts. These suggestions are carefully prioritised by panels of independent experts (including NHS service users). The HTA programme then commissions the research by competitive tender.

Second, the HTA programme provides grants for clinical trials for researchers who identify research questions. These are assessed for importance to patients and the NHS, and scientific rigour.

Third, through its Technology Assessment Report (TAR) call-off contract, the HTA programme commissions bespoke reports, principally for NICE, but also for other policy-makers. TARs bring together evidence on the value of specific technologies.

Some HTA research projects, including TARs, may take only months, others need several years. They can cost from as little as £40,000 to over £1 million, and may involve synthesising existing evidence, undertaking a trial, or other research collecting new data to answer a research problem.

The final reports from HTA projects are peer reviewed by a number of independent expert referees before publication in the widely read journal series Health Technology Assessment.

Criteria for inclusion in the HTA journal series

Reports are published in the HTA journal series if (1) they have resulted from work for the HTA programme, and (2) they are of a sufficiently high scientific quality as assessed by the referees and editors.

Reviews in Health Technology Assessment are termed 'systematic' when the account of the search, appraisal and synthesis methods (to minimise biases and random errors) would, in theory, permit the replication of the review by others.

The research reported in this issue of the journal was commissioned by the HTA programme as project number 06/72/02. The contractual start date was in February 2008. The draft report began editorial review in February 2009 and was accepted for publication in June 2009. As the funder, by devising a commissioning brief, the HTA programme specified the research question and study design. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the Department of Health.

Editor-in-Chief: Professor Tom Walley CBE

Series Editors: Dr Martin Ashton-Key, Dr Aileen Clarke, Professor Chris Hyde, Dr Tom Marshall, Dr John Powell, Dr Rob Riemsma and Professor Ken Stein

© 2010 Crown Copyright.

Included under terms of UK Non-commercial Government License.

PMID: 20178696

PubMed Health Blog...

read all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...