NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Free C, McCarthy O, French RS, et al. Can text messages increase safer sex behaviours in young people? Intervention development and pilot randomised controlled trial. Southampton (UK): NIHR Journals Library; 2016 Jul. (Health Technology Assessment, No. 20.57.)

Cover of Can text messages increase safer sex behaviours in young people? Intervention development and pilot randomised controlled trial

Can text messages increase safer sex behaviours in young people? Intervention development and pilot randomised controlled trial.

Show details

Chapter 4Pilot trial

Objectives

The pilot trial aimed to assess the feasibility of a main trial and to test all trial procedures.

The objectives were to:

  • recruit 200 participants within 3 months
  • successfully deliver ≥ 93% of messages from the aggregator to participants’ mobile phones (data provided by the aggregator)
  • complete follow-up of ≥ 80% for the proposed primary outcome for the main trial (the cumulative incidence of chlamydia).

Methods

Description of trial design

This was a pilot, parallel-arm randomised controlled trial with an allocation ratio of 1 : 1, conducted in multi-geographical areas of the UK.

Important changes to methods after trial commencement

There were no changes to the methods after the trial commenced.

Participants

Eligibility criteria for participants

People aged 16–24 years with a positive chlamydia test result or who had had unsafe sex in the last year (defined as more than one partner and at least one occasion of sex without a condom) and who owned a mobile phone were eligible. People who satisfied these requirements were ineligible if they were non-English-language speakers or were unable to provide informed consent (e.g. people with severe learning difficulties).

Settings and locations where the data were collected

This trial identified potential participants through sexual health services in six geographical locations in the UK: London, Cambridgeshire (rural and urban), Manchester, East Anglia, Kent and Hull. Research staff recruited participants on site at the London and Manchester services. Staff at the Cambridgeshire, East Anglia, Kent and Hull services identified eligible potential participants and referred them to the trial centre (LSHTM) for recruitment.

Intervention

Intervention delivery and timing

The bespoke texting software delivered the intervention messages automatically, directly to the mobile phone number given by each participant at enrolment. The messages were tailored according to sex and infection status at enrolment. Additional tailoring enabled participants to choose a daily time period when they did not want the messages delivered (‘embargoed time’). We collected embargoed time preference data at enrolment, which were automatically fed into the texting software during baseline data entry (see Data collection and entry). The automatic embargoed time was from 2300 to 0900. Each day, the system delivered the messages from the start of the non-embargoed period, every 4 hours. For shorter non-embargoed periods, the system delivered the messages in proportionally shorter intervals. The computer automated mode of delivery ensured standardisation of the intervention.

Message frequency and spacing

For men and women testing positive for chlamydia the intervention included four messages per day for the first 3 days, reducing to one to two messages per day for the first 2 weeks. The number of messages was then reduced to one per day for the first month followed by between one and nine per month until 12 months. For men and women who tested negative for chlamydia the intervention included one to two messages per day for 1 month and then one to three messages per week for up to 12 months.

Intervention for chlamydia-, non-specific-urethritis- and gonorrhoea-positive participants

The intervention for female participants who received a chlamydia-, NSU- and gonorrhoea-positive result at the time of enrolment (‘positive participants’) consisted of 63 text messages sent over 1 year, starting from the point of randomisation. The intervention for positive male participants consisted of 61 text messages sent over 1 year, starting from the point of randomisation.

The message set was tailored according to sex and infection status at enrolment (no infection, chlamydia, gonorrhoea or NSU). The message sets for those diagnosed with a STI were similar to each other, except that the information provided was specific to the STI diagnosed. The numbers of messages targeting each behaviour and the numbers of messages employing specific intervention functions and BCTs are described in Table 6.

TABLE 6

TABLE 6

A summary of the final intervention: the number of messages targeting each behaviour, employing each intervention function and behaviour change technique

For those diagnosed with a STI, messages over the first 3 days focused on engaging with the study, getting treatment, taking treatment and providing information about the infection. Over the next week messages targeted telling partner(s) about an infection. The messages provided non-judgemental, non-stigmatising information covering how common infections are, that an individual may not have symptoms and so therefore be unaware that they have a STI; that many people diagnosed with a STI have had only one sexual partner in the previous year, and that infections are easy to treat. Messages provided information about how to prevent infections. They also provided suggestions about when, where and how to tell a partner about an infection and examples of how others had told partners, covering a range of different types of relationship (e.g. casual, long term). The messages then provided links to services that could inform partners and links to support for anyone concerned about violence in their relationship after telling a partner about an infection. Messages also aimed to provide social support, acknowledging people’s experiences.

For those diagnosed with an infection, after day 14 the messages targeted condom use and testing for STIs before having unprotected sex with a new partner, employing the same messages as for those who were not diagnosed with an infection.

Intervention for chlamydia-, non-specific-urethritis- and gonorrhoea-negative participants

The intervention for female participants who did not have a positive test result at the time of enrolment (‘negative participants’) consisted of 51 text messages, starting from the point of randomisation (see Table 6). The intervention for negative male participants consisted of 49 text messages, starting from the point of randomisation. Participants who had not been diagnosed with an infection were sent the messages about safer sexual behaviours (condom use and testing for STI) starting on day 1. Over the following 30 days messages were sent providing information on how to prevent infections and how you cannot assess risk according to how well you know someone or by their appearance. Messages included instructions on how to use condoms, emphasised positive aspects of condom use and provided tips on preventing condom problems and examples of how others resolved condom use problems. Participants were prompted to think about risks that they had taken and what they could do differently in the future and also to consider how they had carried out safer sexual behaviours in the past. Text messages included advice regarding getting tested before unprotected sex with a new partner. Participants were also sent links to further web-based information regarding contraception, alcohol and sexual risk, how to use a condom and general communication about sex. Women were sent messages covering how other women had negotiated condom use. The messages were designed to provide social support for safer sexual behaviours.

Control

Control messages were also delivered through the bespoke texting software, that is, during the chosen non-embargoed time period. All participants (including participants randomised to the intervention) received the control messages. The set of control messages consisted of 13 messages in total, which were spaced 30 days apart starting from the point of randomisation (see Table 6). The control messages contained no BCTs or information regarding sexual health. The messages were intended to help keep participants engaged in the study and to remind them of their participation, for example ‘Young people can experience health inequalities. Taking part in the texting study can help things to be equal. Thanks for taking part’. The messages expressed our appreciation of their involvement in the study and suggested that participation in research can be personally beneficial: ‘Taking part in the texting study is a way to help you be actively involved in things that affect your life. Thank you for taking part’.

Recruitment

We identified participants from seven sexual health services located in inner-city Manchester, south-east London, Cambridgeshire, Norfolk, Maidstone, Hull and London Brook services. Recruitment staff recruited participants on site at the service or staff referred eligible participants to OM at LSHTM for telephone recruitment.

Recruitment staff on site assessed potential participants for eligibility, provided detailed verbal and written information and gave potential participants the opportunity to ask any questions. Those who agreed to participate were asked to provide written informed consent by filling out a paper-based version of the consent form.

Staff at other services identified eligible participants (aged 16–24 years and who had recently received a positive chlamydia test result) and asked those who were interested in participating for their permission to pass their mobile phone number to OM. OM telephoned the referrals and provided detailed verbal information. She then texted or e-mailed the link to the online participant information sheet [see https://text4health.lshtm.ac.uk/registration/information_sheet.aspx (accessed 15 March 2016)] and gave participants the opportunity to ask any questions. Participants who agreed to participate provided informed consent through the secure online trial consent form [see https://text4health.lshtm.ac.uk/registration/Consent_Form.aspx (accessed 15 March 2016)].

Sample size

The aim of the pilot trial was to estimate the likely rate of recruitment and rate of follow-up at 12 months to assess the feasibility of the main trial. With a pilot sample size of 200, we would be able to estimate a loss to follow-up rate of 20% to within a 95% confidence interval (CI) of ± 6% (i.e. a 95% CI of 14% to 26%). The width of the CI was calculated by 1.96 × [p × (1 – p)/n], where p is the percentage dropout that we expect to see and n is the intended sample size. If the loss to follow-up was 10%, this would be within 4% of the true rate (i.e. a 95% CI of 6% to 14%).

Interim analyses and stopping rules

There were no interim analyses. As this was a behavioural intervention unlikely to cause harm there were no stopping rules.

Outcomes

Primary outcomes

The primary outcomes for the pilot trial were the recruitment rates and completeness of follow-up for the proposed primary outcome for the main trial (cumulative incidence of STIs at 12 months).

Our target was to recruit 200 participants over 3 months and to achieve an 80% response rate at each follow-up point. We assessed the numbers recruited by the number randomised during the 3-month time period. We assessed the follow-up response by the numbers completing the questionnaire at months 1 and 12 and returning a chlamydia test sample at months 3 and 12.

Secondary outcomes

Secondary outcomes were:

  • the proportion of messages successfully delivered (measured using the IT system metrics)
  • at month 1 we assessed:
    • process outcomes:
      • number of text messages read (all/some/none)
      • Knowledge of someone else who took part in the study (yes/no) – if yes: if the other person read the messages we sent the participant; how many they read; if the participant read the messages we sent the other person and how many they read
      • If anyone read messages sent to the participant and if yes, how the participant felt about this
    • behavioural outcomes:
      • for participants testing positive for chlamydia, gonorrhoea or NSU at the time of recruitment: if they took the treatment (yes/no); if they avoided sex for 7 days after treatment (yes/no/unsure); if they told the last person they had sex with before the test that they needed to get treatment (yes/no/sort of); if they avoided sex with this person for 7 days after they took the treatment (yes/no/not applicable); and if there was someone else they were having sex with around the time they tested positive, if they told this person to get treatment (yes/no/there wasn’t anyone else)
      • condom use at last sex (yes/no/unsure)
      • sex with someone new since joining the study (yes/no/unsure)
      • condom use at last sex with someone new (yes/no/unsure)
      • STI testing for self prior to sex with someone new (yes/no/unsure)
      • participant’s report whether their last new partner was tested for STIs prior to sex with them (yes/no/unsure)
      • number of sexual partners since joining the study (0/1/2+)
  • at month 3 we assessed:
    • infection (urine test for men and self-taken vulvovaginal swab for women, according to their preference) (positive/negative)
  • at month 12 we assessed:
    • infection (urine test for men and self-taken vulvovaginal swab for women, according to their preference) (positive/negative)
    • behavioural outcomes:
      • for participants testing positive for chlamydia, gonorrhoea or NSU after joining the study: if they took the treatment (yes/no); if they avoided sex for 7 days after treatment (yes/no/unsure); if they told the last person they had sex with before the test that they needed to get treatment (yes/no/sort of); if they avoided sex with this person for 7 days after they took the treatment (yes/no/not applicable); and if there was someone else they were having sex with around the time they tested positive, if they told this person to get treatment (yes/no/there wasn’t anyone else)
      • condom use at last sex (yes/no/unsure)
      • sex with someone new since joining the study (yes/no/unsure)
      • condom use at last sex with someone new (yes/no/unsure)
      • STI testing for self prior to sex with someone new (yes/no/unsure)
      • participant’s report whether their last new partner was tested for STIs prior to sex with them (yes/no/unsure)
      • number of sexual partners since joining the study (0/1/2+)
      • number of text messages read (all/some/none)
      • car accident in which the participant was the driver in the past 12 months (yes/no).

We also report participants’ views regarding the intervention messages (agree/unsure/disagree):

  • the text messages made me take action
  • the text messages made me think
  • the text messages were from someone I could trust
  • the text messages were respectful
  • the text messages talked down to me
  • the text messages were easy to understand
  • there were too few text messages each day
  • there were too many text messages each day
  • I would have liked the text messages to stop sooner
  • the text messages came at the right time of day.

There were no changes to the trial outcomes after the trial commenced.

Data collection and entry

We collected self-reported data using the trial baseline and follow-up questionnaires. Participants recruited on site completed a paper version of the baseline questionnaire with the recruitment staff. Participants who were enrolled by telephone referral provided baseline data to OM over the telephone. If participants had a positive chlamydia or gonorrhoea test result or NSU diagnosis at enrolment, recruiting staff at the clinic entered the baseline data onto the secure online trial database system within 24 hours. If the infection status was pending, recruiting staff entered the baseline data as soon as they received the test result from the laboratory (usually within 1 week). All participants enrolled by telephone referral had received a positive chlamydia test result and OM entered their data on the day that they were recruited.

In addition to sexual behaviour data, the baseline questionnaire also collected the following contact information: first name; surname; main mobile phone number; alternative phone number; e-mail address; alternative e-mail address; primary postal address; alternative postal address; and name and contact details of someone to contact if the participant could not be reached and his or her relationship with this person (optional) (see Appendix 5). We also collected the following demographic data: date of birth, sex, ethnicity and sexual orientation. Finally, the baseline questionnaire collected data on participants’ preferences regarding message delivery times and whether they preferred to test for chlamydia using a postal test kit or by attending the clinic.

Randomisation

Sequence generation

An independent online randomisation system [see www.sealedenvelope.com/ (accessed 22 July 2016)] generated the 1 : 1 allocation sequence, stratified by site, using random permuted block sizes of 2, 4 and 6. Staff were not aware of the block sizes.

Allocation concealment

The online randomisation system generated the allocation sequence, which meant that staff enrolling participants into the trial could not have known in advance which treatment allocation the next participant would receive.

Implementation

The online randomisation system randomised participants immediately after the recruiting staff entered their baseline data onto the online trial database system (see Recruitment and Data collection and entry).

Masking

Because of the nature of the intervention, participants could have been aware of their treatment allocation; they would have expected frequent text messages (intervention) or one text message a month (control). Thus, the participants were unmasked. The trial manager (OM) required access to treatment allocation to monitor the incoming texts and identify intervention participants for the qualitative interviews. However, the risk of bias associated with this unmasking is low as the intervention was prescribed and delivered by the bespoke texting software, directly to participants’ mobile phones; OM was not involved in the delivery of the intervention. Laboratory staff assessing chlamydia infection and researchers assessing the outcomes were masked to treatment allocation. Staff performing the statistical analysis were also masked to treatment allocation. Data were double entered with one researcher masked to allocation. The treatment allocation variable in the data set was coded 1 or 2 and this was kept undisclosed until the full analysis was complete.

Statistical methods

We estimated the follow-up rate for the primary outcome proposed for the main trial (cumulative incidence of chlamydia at 12 months) with a 95% CI. For all other outcome measures we estimated the relative risk with a 95% CI and a p-value using log binomial regression with robust standard errors. We analysed by randomised arm and conducted a complete case analysis only. This is a behavioural intervention unlikely to produce adverse effects and so the analysis by the research team was undertaken once, at the end of the trial.

Pooling of sites

Data were pooled across all sources of recruitment.

Time points for analysis

This is a behavioural intervention unlikely to produce adverse effects and so the analysis by the research team was undertaken once, at the end of the trial and after the data set had been locked.

Methods for dealing with missing data

We conducted a complete-case analysis only.

Adjustments for covariates

We did not adjust for covariates in the primary analysis.

Multiple comparisons

We did not adjust for multiple comparisons.

Examination of subgroups

We did not conduct a subgroup analysis.

Analysis of primary and secondary end points

The primary outcomes were the recruitment rate and completeness of follow-up for the proposed primary outcome for the main trial and the proportion of messages successfully delivered to participants’ mobile phones. We report the cumulative incidence rate of chlamydia infection in the control group to inform the sample size calculation for the main trial. For each binary outcome we report relative risks with 95% CIs and give a two-sided p-value for statistical significance.

Adverse events

Involvement in a road traffic accident is the only plausible adverse event that might be caused by a mobile phone-based texting intervention. We also report the proportion of participants who experienced someone else reading their text messages and the participants’ response to this (happy/unhappy/unsure).

Results

We assessed 470 people for eligibility (Figure 1) of whom 270 were excluded (n = 169 did not meet the inclusion criteria, n = 101 declined to participate); therefore, 200 participants were recruited within 3 months. In total, 66% (n = 131) of participants were recruited face to face at the Camberwell Sexual Health Centre at King’s College Hospital in London (see Appendix 7, Table 13), 4% (n = 8) were recruited from the Palatine Contraception and Sexual Health Service in Manchester, 6% (n = 11) were recruited face to face at Cambridge Regional College and Huntingdonshire Regional College and 25% (n = 50) were recruited by telephone referral. Recruitment stopped once we had achieved our target recruitment number. Participants were randomised from 9 September 2013 to 29 November 2013 and were followed up between October 2013 and the end of February 2015. In total, 99 participants were allocated to the intervention and 101 were allocated to the control (see Figure 1).

FIGURE 1. Pilot trial Consolidated Standards of Reporting Trials (CONSORT) diagram.

FIGURE 1

Pilot trial Consolidated Standards of Reporting Trials (CONSORT) diagram.

The baseline demographic and sexual behaviour data of the randomised participants are presented in Table 7.

TABLE 7

TABLE 7

Baseline demographic and sexual behaviour characteristics

Primary outcomes

Our primary outcomes were full recruitment within 3 months and follow-up rate for our proposed primary outcome for the main trial. We fully recruited within 3 months and 97% of messages were successfully delivered. We obtained an 81.0% (162/200) follow-up rate for the cumulative incidence rate of chlamydia, with a rate of 81.2% (82/101) in the control group and 80.8% (80/99) in the intervention group (Table 8).

TABLE 8

TABLE 8

Pilot trial primary outcome data

In total, 92% (183/200) provided questionnaire outcome data at 1 month. Of the 183 providing data, 80% (146/183) returned the postal questionnaire, 13% (23/183) completed the questionnaire online, 4% (7/183) completed the questionnaire at the clinic, 0.5% (1/183) responded to the key questions by e-mail and 3% (6/183) responded to the key questions by text message. In total, 86% (171/200) provided a chlamydia test sample at 3 months. Of the 171, 98% (167/171) returned the sample by post and 2% (4/200) provided the sample at the clinic. In total, 82% (163/200) provided questionnaire outcome data at 12 months. Of the 163, 94% (153/163) returned the postal questionnaire, 3% (5/163) completed the questionnaire online, 2% (3/163) completed the questionnaire at the clinic, 0% (0/163) responded to the key questions by e-mail and 1% (2/163) responded to the key questions by text message. Finally, 80% (160/200) provided a chlamydia test sample at 12 months (see Table 8). Of the 160, 98% (157/160) returned the sample by post and 2% (3/160) provided the sample at the clinic.

Withdrawals and requests to stop the intervention

Three participants withdrew from the study, all of whom were randomised to the intervention arm. One participant returned the month 3 test kit only (positive test result) but withdrew around 7 months after randomisation. Another participant returned the month 1 questionnaire only and requested to be withdrawn 3.5 months after randomisation. The third participant withdrew 11.5 months after randomisation, never requested that the messages stop and did not respond at any follow-up point. Of the 99 participants receiving the intervention, 15 participants (15%) requested that the messages stop; two of these also withdrew from the study (both stopped the messages 1 day after enrolment).

Secondary outcomes

Process outcomes

At 1 month 82% (72/88) of respondents in the intervention group had read all messages and at 12 months 74% of respondents in the intervention group (56/76) had read all messages (Table 9). There were three documented cases at 12 months in which participants in the control group reported reading messages sent to other trial participants.

TABLE 9

TABLE 9

Process outcomes

Intervention group participant views regarding the messages at month 1 are presented in Table 10. Over 80% of intervention recipients reported that the text messages ‘made me think’, were ‘respectful’ and were ‘easy to understand’. Just over one-third of recipients reported that the messages ‘made me take action’. A significant minority thought that the messages talked down to them (11/85; 13%); of these, seven out of 11 did not have an infection at the outset of the trial. About two-thirds of participants reported that the messages ‘were from someone they could trust’ and ‘came at the right time of day’. About one-fifth of respondents thought that there were too many messages and about one-fifth thought there were too few.

TABLE 10

TABLE 10

Intervention group participant views regarding the messages at month 1

There were 11 behavioural outcomes collected for all participants and 16 collected for participants testing positive for a STI at the start of the trial. There were no statistically significant changes in behaviour or infection using a cut-off of 0.005 (which uses a Bonferroni correction for multiple comparisons). The secondary behavioural and STI outcomes are reported in Appendices 810.

Contact problems

Losses to follow-up were mainly the result of mobile phone numbers no longer being current. Six out of the 17 non-responders to the month 1 questionnaire had a problematic mobile phone number. Three mobile phone numbers went straight to voicemail and two numbers were not in service after many attempts. We could not reach one participant at the number provided. Five of the 29 non-responders to the month 3 chlamydia test had a problematic mobile phone number. Two mobile phone numbers went straight to voicemail, two numbers were not in service and we could not reach one participant at the number provided.

Discussion

Key findings

The pilot trial demonstrated the feasibility of the trial procedures for a main trial. We fully recruited early and achieved an 81% follow-up rate for our proposed primary outcome of cumulative incidence of chlamydia at 12 months. There was no differential follow-up between groups. The IT system delivering the messages was effective, with 97% of messages sent being successfully delivered to participants’ mobile phones. Participant views of the intervention suggest that it is acceptable to the majority of participants. Some participants shared messages with friends and there were three cases of control group participants (3%) reporting reading other participants’ messages.

Strengths and weakness

In the pilot trial we achieved a high follow-up rate, allocation was concealed and laboratory staff and those analysing data were blind to allocation. Only one of the researchers double entering data was masked to allocation; however, in a main trial there would be sufficient staff for all staff entering follow-up data to be masked to allocation. Given the small sample size and the large number of variables assessed, the baseline characteristics of participants were reasonably well balanced. There were some differences in baseline infections and ethnicity between groups. Nonetheless, as the primary outcomes for the pilot were recruitment and follow-up, the allocation balance would not impact on this. Although the response rate overall was high, the response rate for some questions, such as partner notification, was low. A larger sample size in a main trial would allow for better balance between the arms. The pilot trial was not powered for behavioural or STI outcomes and so 95% CIs for these outcomes were large and it is unsurprising that no outcomes were statistically significant when the multiple comparisons were taken into account.

Discussion in relation to the existing literature

This pilot trial’s follow-up response rate for return of the chlamydia test samples was considerably higher that that seen in similar trials, screening initiatives and surveys.37 The Natsal study in the UK achieved a 57.7% response rate for face-to-face interviews and a 60% response rate for urine samples requested.35 The ClaSS project reported an uptake of chlamydia postal screening of 31.5% in people aged 16–24 years,36 whereas the Sexunzipped trial achieved a 45% follow-up rate using chlamydia postal test kits and a 72% follow-up rate for self-reported data at 3 months.37

The ClaSS project36 employed some elements of our approach to follow-up described in Chapter 3, such as choosing a postal testing kit that fit through a ‘standard’ letterbox and testing the kit with the target group.48 In the ClaSS project the researchers evaluated interventions to increase follow-up after the project commenced and later adopted pre-notification in the form of an invitation letter sent in advance of the kits. Adopting our approach would have identified methods known to increase postal follow-up before the study commenced, which may have increased the response. Our response rate may be higher than that achieved by the ClaSS project because our participants had agreed to provide follow-up data when they were recruited, we offered unconditional incentives and we included only essential test kit components.

Generalisability

The pilot trial was designed to demonstrate the feasibility of a main trial. It was not designed to generate reliable estimates of the intervention effect and thus the intervention effects are neither accurately estimated nor generalisable. The proportion of eligible participants agreeing to take part was 66% (200/301). The intervention has been designed to be accessible across socioeconomic and ethnic groups.

The implications of the pilot trial results for the design of the main trial are discussed in Chapter 6.

Copyright © Queen’s Printer and Controller of HMSO 2016. This work was produced by Free et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Included under terms of UK Non-commercial Government License.

Bookshelf ID: NBK378905

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (3.6M)

Other titles in this collection

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...