• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of bmjBMJ helping doctors make better decisionsSearch bmj.comLatest content
BMJ. Apr 22, 2006; 332(7547): 969–971.
PMCID: PMC1444839
Research methods

Reporting attrition in randomised controlled trials

Jo C Dumville, research fellow,1 David J Torgerson, director,1 and Catherine E Hewitt, PhD student1

Short abstract

Loss to follow-up can greatly affect the strength of a trial's findings. But most reports do not give readers enough information for them to be able to understand the potential effects

The main evaluative strength of randomised controlled trials is that each group is generally balanced in all characteristics, with any imbalance occurring by chance. However, during many trials participants are lost to follow-up. Such attrition prevents a full intention to treat analysis being carried out and can introduce bias.1,2 Attrition can also occur when participants have missing data at one or more points. We argue that researchers need to be more explicit about loss to follow-up, especially if rates are high.

Effects of attrition

Attrition can introduce bias if the characteristics of people lost to follow-up differ between the randomised groups. In terms of bias, this loss is important only if the differing characteristic is correlated with the trial's outcome measures. However, attrition is not a black and white issue—there is no specific level of loss to follow-up at which attrition related bias becomes acknowledged as a problem. Schulz and Grimes argue that loss to follow-up of 5% or lower is usually of little concern, whereas a loss of 20% or greater means that readers should be concerned about the possibility of bias; losses between 5% and 20% may still be a source of bias.3 For the purposes of this article we will not differentiate between loss to follow-up and missing data. We have also not considered exclusions by trial investigators. Although exclusion is justified in some cases,3 generally it is ill advised.1,2

Reporting of attrition

In a review of trials published in four general medical journals in 2002, 54% (71) of the 132 trials had some loss to follow-up for the main analysis.4 Among these trials the median percentage loss was 7% (minimum 0.08%, maximum 48%; interquartile range 2-18%). These data suggest that potentially problematic loss to follow-up occurs in many trials, even those published in high quality general journals.4

The standard practice for reporting trials, as encouraged by CONSORT,5 is to include a table describing the baseline characteristics of the trial participants. This table provides useful information on all participants and confirms the success of the randomisation process. However, if there has been loss to follow-up, information from the whole sample may not adequately describe the analysed sample nor accurately reflect the comparability of the trial groups. Thus, we suggest it is informative to present baseline characteristics for the participants for whom data have been analysed and those who are lost to follow-up separately. This would provide a clearer picture of the subsample not included in an analysis and may help indicate potential attrition bias. As an example we have taken data from a recently published randomised trial.trial.

Figure 1
Is your trial fall proof?

Example trial

A trial of hip protectors for preventing hip fracture is typical of many studies in that about 20% of participants were lost to follow-up from routine data collection.6 The authors dealt with this problem for the main outcome (hip fracture) by accessing the general practice records of non-responders. However, for secondary outcomes (such as quality of life) this was not possible. Therefore, reports of these secondary outcomes are at a high risk of bias from attrition. We can assess whether these outcomes may be affected by attrition bias by comparing rates of loss to follow-up between the arms of the trial as well as by examining the baseline characteristics of participants who were lost to follow-up and the characteristics of those remaining.

The trial had a small but significant difference in attrition rates between the two arms (372 (28%) in the intervention group and 619 (22%) in the control group, P = 0.001). This is the first indication of a potential problem (table). Because we have different numbers of participants leaving the trial arms, the likelihood that participants in one group are not balanced with similar participants in the other trial arm is increased.

Table 1
Baseline characteristics of all participants in a trial evaluating hip protectors, those lost to follow-up, and those remaining in the trial to the end.6 Values are percentages (numbers) unless stated otherwise

The table includes the baseline characteristics of all the participants. Not surprisingly, the baseline characteristics of the whole sample are well balanced, as we would expect through random allocation. However, these data cannot tell us whether the sample of women included in the analysis is balanced between the two treatment groups.

Of more interest is a comparison of the baseline characteristics of those who have left the study and those remaining. The table shows, as you might expect, that the between group differences in those lost to follow-up tend to be larger than any chance differences at baseline. For example, more volunteers, people with poor or fair health, and people with a previous fracture are lost from the control group than the intervention group. In these examples, the differential attrition leads to varying changes in the characteristics of those participants remaining in the trial when compared with all participants at baseline (table). This information is useful for the reader. Yet, providing information about the participants who have not contributed to the analysis may also be informative.

Interpreting results

The internal validity of a trial's results partly depends on the between group balance in prognostic characteristics of those who remain in the trial. In addition, important imbalances that are not readily apparent in the analysed groups may become apparent when we examine the between group characteristics of those lost to follow-up.

To clarify this point we have carried out some simulation work to assess the effect of attrition bias on baseline characteristics and the type 1 error rate. The simulations used a population of trials each with 630 participants and 10% biased attrition in one arm and 10% random attrition in the other. Although the type 1 error rate was substantially increased, such attrition does not always lead to an apparent imbalance of the baseline characteristics of the participants remaining in the trial. Thus, assessment of the characteristics of those lost to follow-up may be particularly important. Further work is needed to examine how this issue is influenced by sample size and other factors.

The data in the table would allow trialists to make a qualitative judgment about whether an important predictor variable has become more imbalanced since randomisation. They could then decide whether to carry out a sensitivity analysis treating a variable, such as previous fracture, as a covariate. However, since such judgment is subjective, especially for smaller sample sizes, a statistical test could be useful.

Statistical testing

The use of statistical tests in this context is complex, and there are arguments for and against testing. From current knowledge we suggest it is not useful to test for differences statistically. A further reason for avoiding statistical testing stems from arguments Altman and others have put forward against baseline testing of the total randomised group—that imbalance of a predictor variable may still bias the study results, even if the imbalance does not reach conventional levels of significance.7-10

Decisions about covariates are normally made before the start of the trial. However, because attrition bias cannot always be anticipated, information on differential attrition is relevant at the analysis stage. Nevertheless, adjusting for variables that have not been specified in advance is poor statistical practice and may introduce bias.11 As Altman suggests, if serious attrition bias is suspected, the analysis should be carried out as originally planned, with perhaps a second analysis adjusting for the new covariate.11 Furthermore, the data presented can only include observed baseline variables: unrecorded or unknown variables may also be imbalanced.

Implications

Baseline tables are a useful way of assessing the study sample, but the characteristics of the sample may change during the study, especially by attrition. As the hip protector example shows, the baseline characteristics of those lost to follow-up during a study can differ. And since attrition of data commonly occurs in trials,1,2,4 such differences could also be common. We know that missing data make it more difficult to carry out a true intention to treat analysis. Yet, since information on the baseline characteristics of those lost to follow-up and those for whom data are analysed is rarely reported, it is almost impossible to identify the effect of attrition on the study sample as a whole and therefore the result of the randomised controlled trial.

Questions arise about when these modified baseline tables could and should be used. We suggest that information on the participants included in the main analysis of a paper is of interest, especially if attrition is high. Missing data are more common for secondary outcome measures, as researchers often focus on collecting data on the primary outcome. Authors could consider presenting a table showing baseline characteristics of those who were and were not analysed when reporting secondary outcomes. The table might look different for different analyses of the same study. Although this table would require increased journal space, the information is arguably more useful than that in a standard baseline table. Since many journals publish online, these tables could be made available to interested readers in the electronic version only. Alternatively, the information might be incorporated into the flow diagram recommended by CONSORT.5

Summary points

Loss to follow-up can lead to bias in randomised trials

Imbalance resulting from this attrition is often hidden

Baseline characteristics of participants lost to follow-up and those included in the analysis should be reported separately

Assessment of the effect of differences between groups on the results is mainly subjective

Notes

Contributors and sources: JCD, DJT, and CEH are all involved in the design, implementation, and analysis of randomised controlled trials. JCD and DJT were responsible for the concept, design, and drafting of the article. CEH had statistical input. JCD is the guarantor.

Competing interests: None declared.

References

1. Tierney JF, Stewart LA. Investigating patient exclusion bias in meta analysis. Int J Epidemiol 2005;34: 79-87. [PubMed]
2. Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ 1999;319: 670-4. [PMC free article] [PubMed]
3. Fergusson D, Aaron SD, Guyatt G, Hebert P. Post-randomisation exclusions: the intention to treat principle and excluding patients from analysis. BMJ 2002;325: 652-4. [PMC free article] [PubMed]
4. Hewitt, CE, Hahn S, Torgerson DJ, Watson J, Bland M. Adequacy and reporting of allocation concealment: review of recent trials published in four general medical journals. BMJ 2005;330: 1057-8. [PMC free article] [PubMed]
5. Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised controlled trials. Lancet 2001;357: 1191-4. [PubMed]
6. Birks YF, Porthouse J, Addie C, Loughney K, Saxon L, Baverstock M, et al. Randomized controlled trial of hip protectors among women living in the community. Osteoporos Int 2004;15: 701-6. [PubMed]
7. Altman DG. Comparability of randomised groups. Statistician 1985;34: 125-36.
8. Senn S. Testing for baseline balance in clinical trials. Stat Med 1994;13: 1715-26. [PubMed]
9. Senn S. Covariate imbalance and random allocation in clinical trials. Stat Med 1989;8: 467-75. [PubMed]
10. Roberts C, Torgerson DJ. Baseline imbalance in randomised controlled trials. BMJ 1999;319: 185. [PMC free article] [PubMed]
11. Altman DG. Adjustment for covariate imbalance. In: Armitage P, Colton T, eds. Encyclopaedia of biostatistics. 2nd ed. Chichester: John Wiley, 2005: 1273-8.

Articles from BMJ : British Medical Journal are provided here courtesy of BMJ Group
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • Cited in Books
    Cited in Books
    PubMed Central articles cited in books
  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...