Send to

Choose Destination
J Epidemiol Community Health. 2018 Feb;72(2):173-178. doi: 10.1136/jech-2017-209976. Epub 2017 Nov 28.

Generalisability of an online randomised controlled trial: an empirical analysis.

Wang C1,2,3,4, Mollan KR5, Hudgens MG5,6, Tucker JD2,3,4, Zheng H2,3, Tang W2,3,4, Ling L1.

Author information

Faculty of Medical Statistics and Epidemiology, School of Public Health, Sun Yat-sen University, Guangzhou, China.
Dermatology Hospital of Southern Medical University, Guangzhou, Guangdong, China.
Guangdong Provincial Dermatology Hospital, Guangzhou, China.
Project China, University of North Carolina, Guangzhou, China.
Center for AIDS Research, School of Medicine, University of North Carolina, Chapel Hill, North Carolina, USA.
Department of Biostatistics, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, North Carolina, USA.



Investigators increasingly use online methods to recruit participants for randomised controlled trials (RCTs). However, the extent to which participants recruited online represent populations of interest is unknown. We evaluated how generalisable an online RCT sample is to men who have sex with men in China.


Inverse probability of sampling weights (IPSW) and the G-formula were used to examine the generalisability of an online RCT using model-based approaches. Online RCT data and national cross-sectional study data from China were analysed to illustrate the process of quantitatively assessing generalisability. The RCT (identifier NCT02248558) randomly assigned participants to a crowdsourced or health marketing video for promotion of HIV testing. The primary outcome was self-reported HIV testing within 4 weeks, with a non-inferiority margin of -3%.


In the original online RCT analysis, the estimated difference in proportions of HIV tested between the two arms (crowdsourcing and health marketing) was 2.1% (95% CI, -5.4% to 9.7%). The hypothesis that the crowdsourced video was not inferior to the health marketing video to promote HIV testing was not demonstrated. The IPSW and G-formula estimated differences were -2.6% (95% CI, -14.2 to 8.9) and 2.7% (95% CI, -10.7 to 16.2), with both approaches also not establishing non-inferiority.


Conducting generalisability analysis of an online RCT is feasible. Examining the generalisability of online RCTs is an important step before an intervention is scaled up.




epidemiological methods; hiv; randomised trials

Supplemental Content

Full text links

Icon for HighWire Icon for PubMed Central
Loading ...
Support Center