Home > Full Text Reviews > The Clinical Effectiveness and... > Assessment of clinical effectiveness

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.

Rodgers M, Asaria M, Walker S, et al. The Clinical Effectiveness and Cost-Effectiveness of Low-Intensity Psychological Interventions for the Secondary Prevention of Relapse After Depression: A Systematic Review. Southampton (UK): NIHR Journals Library; 2012 May. (Health Technology Assessment, No. 16.28.)

3Assessment of clinical effectiveness

The review of the evidence for clinical effectiveness was undertaken systematically following the general principles recommended in the Centre for Reviews and Dissemination (CRD)'s guidance for undertaking reviews in health care41 and the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement.42

Methods for reviewing clinical effectiveness

Search strategy

Literature searches were developed to systematically identify studies on the effectiveness of low-intensity psychological interventions to prevent relapse or recurrence after depression. The base search strategy was constructed using MEDLINE and then adapted for other resources searched. The search strategy included the following components:

  1. depression terms, and
  2. relapse terms, and
  3. low-intensity psychological intervention-related terms.

The searches were restricted to studies published after 1950. No language restrictions or study design filters were applied.

Search terms were identified by scanning key papers identified at the beginning of the project, through discussion with the review team and clinical experts, and the use of database thesauri.

Sources of information were identified by an information specialist with input from the project team. The following databases were searched during September 2010:

  • MEDLINE (via OvidSP)
  • MEDLINE In-Process & Other Non-Indexed Citations (via OvidSP)
  • PsycINFO (via OvidSP)
  • EMBASE (via OvidSP)
  • The Cochrane Library (via Wiley)

    CDSR (Cochrane Database of Systematic Reviews)

    DARE (Database of Abstracts of Reviews of Effects)

    CENTRAL (Cochrane Central Register of Controlled Trials)

    HTA (Health Technology Assessment Database)

  • Science Citation Index (via ISI Web of Knowledge)
  • Social Science Citation Index (via ISI Web of Knowledge)
  • BIOSIS Previews (via ISI Web of Knowledge and Dialog).

In addition, a range of resources were searched or browsed to identify guidelines on the treatment of depression. The bibliographies of relevant reviews and guidelines and included studies were checked for further potentially relevant studies.

Records were managed within an EndNote library, version X3 (Thomson Reuters, CA, USA). After de-duplication, 9112 records in total were identified.

The full search strategies and results for each database can be found in Appendix 1.

Inclusion and exclusion criteria

Two reviewers independently examined titles and abstracts for relevance; all potentially relevant papers meeting the inclusion criteria were ordered. All full papers were then independently screened by two reviewers, with disagreements resolved by consensus.


Studies of participants who have received treatment for depression were included. Studies establishing a diagnosis using a gold-standard structured interview for DSM or ICD criteria, such as the Structure Clinical Interview for DSM Disorders (SCID)43 were included, as were studies defining depression on the basis of a score above a cut-off point on a recognised psychometric measure or on the basis of unaided clinical diagnosis. The decision problem is concerned with the prevention of relapse or recurrence in patients who have received and responded to treatment. Consequently, studies of patients who were treated for an acute episode and then subsequently measured for relapse or recurrence were excluded; studies where patients had ‘recovered’ from their acute episode (responding to treatment or asymptomatic) and the aim was to prevent subsequent relapse or recurrence were included. Studies of participants with bipolar disorder were excluded, as were studies of children.


For part A (systematic review of efficacy), all evaluations of ‘low-intensity’ interventions as defined by the IAPT programme22,23 were considered relevant. Specifically, this incorporated any unsupported psychological/psychosocial interventions or any supported interventions that did not involve highly qualified mental health professionals. ‘Highly qualified professionals’ includes clinicians, who, in most instances, will have a core professional qualification (e.g. psychiatrist, clinical psychologist, mental health nurse) and have received formal, specialist training in the delivery of complex psychological interventions (e.g. 16+ session CBT, psychodynamic psychotherapy, systematic therapy, etc.).

Any interventions involving support from para-professionals, peer supporters, PWPs, physical trainers, case managers (as in collaborative care models) or no personal support at all (e.g. entirely computerised interventions) were included. ‘Para-professionals’ includes people who do not have a core professional qualification and do not have specialist training in complex psychological interventions, although may have some training in less complex interventions. Inclusion was not restricted by length of treatment, number of sessions or mode of delivery.

For part B (scoping review), all relevant evaluations of interventions involving qualified mental health professionals (e.g. clinician, CBT therapist) were included if they involved < 6 hours of contact per patient. For group treatment, contact estimates per patient were calculated by dividing treatment duration by the mean number of patients per group (with adjustments as necessary if there is > 1 therapist). Where the amount of contact time was unclear, study authors were contacted to obtain additional details. If authors could not be contacted or did not respond, clinical experts (ML, DM) were consulted as to whether or not the intervention was likely to be brief (i.e. < 6 hours per patient).

High-intensity psychological interventions requiring ongoing interaction with a mental health professional (e.g. CBT, behavioural activation, problem-solving therapy and couples therapy) were excluded. Studies evaluating interventions for the acute phase of treatment of an acute episode of depression were also excluded.

Studies evaluating pharmacotherapy alone [including TCAs, SSRIs, serotonin–norepinephrine reuptake inhibitors (SNRIs), anxiolytic medication, mood stabilisers and others] were excluded from the review of clinical effectiveness, as were studies of alternative and complementary treatment methods.


Study inclusion was not restricted by type of comparator treatment and could include no treatment (including waiting list control), placebo, psychological or pharmacological interventions.


Studies reporting outcomes related to relapse or recurrence (e.g. relapse rate, time to relapse, and severity of relapse episode) after initial treatment success were included. Other relevant outcomes such as social function and quality-of-life (QoL) measures were recorded where reported.

Study designs

Randomised, quasi-randomised and non-randomised studies with concurrent control patients were considered for inclusion. Animal models, preclinical and biological studies, reviews, editorials and opinions were excluded.

Translations of non-English-language papers and additional details of studies published only as meeting abstracts were obtained where time and budget constraints allowed.

Data extraction strategy

Data were extracted independently by one reviewer using a standardised data extraction form and checked by another. Discrepancies were resolved by discussion, with involvement of a third reviewer when necessary. Authors were contacted for any missing data or for clarification where necessary. Data from multiple publications of the same study were extracted as a single study. Extraction included data on patient characteristics, interventions, comparators, study design and outcomes.

Critical appraisal strategy

The quality appraisal checklist for quantitative intervention studies described in NICE's guide to methods for developing guidance in public health was obtained for assessing the internal and external validity of studies included in the systematic review of low-intensity interventions (part A).44 For the scoping review of brief therapy interventions (part B), formal critical appraisal of the included studies was not planned or conducted, with the exception of one study,45 in which the necessity of health professionals to deliver the intervention was unclear (see Assessment of effectiveness, Part B: brief therapy interventions for the prevention of relapse of depression, below).

Methods of data synthesis

Given the limited number of included studies and their clinical and methodological heterogeneity, a meta-analysis was not appropriate. Therefore, extracted data have been tabulated and discussed in a narrative synthesis.

Results of review of clinical effectiveness

Quantity and quality of research available

A total of 9112 unique records were identified from the searches and 129 articles were ordered for assessment. Figure 2 shows the flow of records through the review process, and the numbers included and excluded at each stage. Details of studies excluded at the full publication stage are presented in Appendix 2 (excluded studies).

FIGURE 2. Study selection process for clinical effectiveness.


Study selection process for clinical effectiveness.

Part A: low-intensity interventions for the prevention of relapse of depression

No studies met the main part A inclusion criteria for ‘low-intensity’ interventions that were delivered by para-professionals, peer supporters or PWPs as defined by the IAPT programme, without any restriction on length of treatment.

Part B: brief therapy interventions for the prevention of relapse of depression

Seventeen studies, reported in 27 publications, met the part B inclusion criteria for brief therapy interventions delivered by mental health professionals involving < 6 hours of contact per patient.4571 Fourteen of the studies were completed and published;4568 three are ongoing.6971

Table 1 provides details of the related publications for each of the included studies. In the following sections, reference will be made to the primary study only; the other linked publications provided additional information or results that are included in the data extraction tables (see Appendix 3).

TABLE 1. Primary and linked publications for included studies.


Primary and linked publications for included studies.

Completed studies

Of the 14 completed studies,4568 12 were parallel-group RCTs;45,46,51,52,5456,6365,67,68 the remaining two studies61,66 were non-randomised with concurrent control patients. Eight of the RCTs recruited participants from multiple centres,45,46,51,56,63,65,68,72 one of which used cluster randomisation.56

Ongoing studies

All three ongoing studies are RCTs.6971

Assessment of effectiveness

Part A: low-intensity interventions for the prevention of relapse of depression

No studies evaluating ‘low-intensity’ interventions that could be delivered by para-professionals, peer supporters, or PWPs as defined by the IAPT programme were identified.

Part B: brief therapy interventions for the prevention of relapse of depression

The following section provides a classification and description of studies identified which were identified as meeting the ‘part B’ inclusion criteria (i.e. they evaluated brief therapy interventions in which participants had up to 6 hours' contact with mental health professionals, such as clinicians or CBT therapists). As these studies fall outside the primary focus of this review, they are briefly described in an overview below, with key study characteristics presented in Appendix 3. In one study (Katon et al.45), the intervention could potentially be delivered by PWPs or equivalent practitioners, although in the retrieved evaluations it was delivered by mental health professionals. Given the potential relevance of this study to our part A question, it is discussed in greater detail below, and has been assessed for internal and external validity (see Appendix 4).

Completed studies

Ten of the completed studies evaluated interventions delivered in a group setting.46,51,54,55,61,63,64,6668 Of these, six specifically evaluated some form of MBCT.51,54,55,63,64,67 Three MBCT studies were based on an identical protocol that involved eight weekly sessions of 2 hours' duration, in which up to 12 participants met with experienced cognitive therapists to receive a programme based on the principles of CBT and mindfulness-based stress reduction.51,64,67 Participants in this programme attended two further meetings during the subsequent 52 weeks of follow-up. Other MBCT programmes were of a similar duration (typically 2 hours to 2 hours and 45 minutes, weekly for 8 weeks), although with larger groups of up to 15 or 17 participants.54,55,63 Three studies46,66,68 evaluated brief group CBT of a similar intensity to the MBCT interventions, but without any explicit mindfulness content. The one remaining group-based intervention was a brief 12-week ‘Coping with Depression’ (CWD) course, which was based on a multimodal psychoeducational approach, delivered by clinical psychologists and psychiatrists.61

Four studies evaluated brief therapy interventions delivered by mental health professionals to participants on an individual basis.45,52,56,65 One such intervention51 provided individuals with a brief CBT-based intervention (30 minutes every other week for 20 weeks) alongside ongoing pharmacotherapy. A second intervention56 incorporated a multimodal skills-based approach, providing support materials and general practitioner (GP) training to allow tailoring of evidence-based psychosocial strategies to individual patients in Australian primary care (‘Keeping The Blues Away’); this is a small pilot study for which it may be that the intervention could potentially be delivered by PWPs or equivalent but it is unclear from the detail provided what level of training is required. One study65 evaluated the effects of ‘continuation CBT’ (around 6 hours per patient) following initial CBT treatment in adolescents with depression. Another study45 evaluated a ‘multifaceted relapse prevention programme’ for patients who were at high risk of relapse, which is described in more detail below.

Eight of the 14 studies formally established the occurrence of relapse or recurrence using gold standard criteria, specifically Diagnostic and Statistical Manual of Mental Disorders – Third Edition-Revised (DSM-III-R) or DSM-IV criteria.45,46,51,54,61,63,64,67 Of these, seven explicitly stated that they established this outcome using SCID.45,46,51,54,63,64,67 Elsewhere, relapse was established using other criteria [Research Diagnostic Criteria (RDC)]52 or a variety of self-report and clinician-administered symptom scales [Beck Depression Inventory (BDI),55,66 Montgomery–Åsberg Depression Rating Scale (MADRS),68 Clinical Global Impression Improvement scale (CGI-I),65 Depression Anxiety Stress Scales (DASS)56].

The results of this diverse group of interventions in terms of preventing relapse or recurrence of depression are mixed. Even among MBCT studies following the same protocol, findings were inconsistent: two studies64,67 reported a statistically significant benefit for MBCT over treatment as usual (TAU) in patients with three or more previous episodes of depression at 14 months, but a third trial restricting inclusion to this subgroup of patients reported no overall difference in relapse between treatment groups over the same period.51 Other studies reported results that clearly favoured MBCT over TAU54 were of borderline significance63 or showed no difference between groups.55 One study68 suggested no significant benefit of brief CBT over TAU for preventing relapse, whereas another suggested any such benefit was restricted to participants with at least five previous depressive episodes.46 One observational study did not report relapse rates and found no significant difference in scores 1 year after the intervention.66 One study reported a statistically significant benefit of a multimodal psychoeducational approach over no intervention in terms of relapse prevention over 6 months, although this small observational study had several methodological limitations.61

The study evaluating a brief CBT-based intervention (alongside ongoing pharmacotherapy51) reported a statistically significant impact on relapse after 2 years, an effect that remained at 6 years' follow-up. Relapse rates were similar for the ‘Keeping The Blues Away’ programme and usual care in Australian primary care.56 The study of ‘continuation CBT’ in adolescents reported significant benefits of the intervention alone over both antidepressant medication treatment and combined continuation CBT/medication.65

Katon et al.

Five articles reported the findings of just one study (Katon et al.).45,5760 Although the practitioners in this study were predominantly mental health professionals, and therefore did not strictly meet our part A inclusion criteria, it was unclear whether or not delivery by a mental health professional was mandatory for the implementation of the intervention. Therefore, this study was critically appraised and is summarised in further detail below.

This study was a RCT that evaluated the effectiveness of a ‘multifaceted relapse prevention programme’ in a US primary care setting (see Appendix 3).45 This programme was provided to adult patients who had recovered from depression but who were at high risk of relapse and were encouraged to continue with antidepressant medication. The relapse prevention programme included aspects of patient education/self-help (patients were provided with a book and videotape developed by the trial investigators) alongside ongoing support from ‘depression specialists’. Each participant was scheduled two face-to-face sessions with a depression specialist (an initial 90-minute session and a 60-minute follow-up session), which were followed by three ‘telephone visits’ scheduled at 1, 4 and 8.5 months after the second face-to-face session. In addition, participants received ‘personalised mailings’ (at 2, 6, 10 and 12 months), containing a graph of participant BDI score over time and checklists on symptoms and medication adherence. The depression specialist alerted the primary care physician if the participant appeared to be symptomatic or had discontinued medication, based on data from participant feedback or from a monthly review of automated pharmacy data on antidepressant refills. Each depression specialist met with a supervising psychiatrist for 15–30 minutes each week to review cases and adjust treatment recommendations.

The focus of the relapse prevention intervention appeared to be largely on maintaining adherence to antidepressant medication. Meetings between patients and intervention ‘depression specialists’ integrated cognitive–behavioural and motivational interviewing approaches and provided information on the prevalence, course and efficacious treatment of depression. The depression specialist explained why each patient was at high risk of relapse, while acknowledging the individual's attitudes, beliefs and treatment choices. Depression specialists and patients discussed evidence illustrating the efficacy of pharmacotherapy for preventing relapse and recurrence, the perceived risks and benefits of long-term pharmacotherapy, approaches to manage specific medication side-effects and concerns of the patient. In addition, the depression specialist attempted to improve self-efficacy for preventing relapse and recurrence of depression through self-management behaviours such as monitoring depressive symptoms and scheduling pleasant activities.

In this trial, three different depression specialists were provided for 194 patients receiving the relapse prevention intervention programme. One depression specialist was a psychologist, one was a nurse practitioner with a master's degree in psychosocial nursing and the third was a social worker. Each of these had received a 60-page training manual and attended two half-day training sessions with a psychiatrist, a psychologist and a primary care physician before the start of the trial.

A total of 191 participants in the comparison group received ‘usual care’, which typically consisted of prescription for antidepressant medication (as in the intervention group), plus between two and four visits with a family physician over the first 6 months of treatment, with the option to refer to health maintenance organisation (HMO)-provided mental health services.

Relapse/recurrence was defined as either a current episode of depression according to the SCID (at 3, 6, 9 or 12 months) or incidence of an episode within each 3-month period according to the Longitudinal Interval Follow-up Evaluation.73 Other outcomes included depressive symptoms [measured by the 20-item Hopkins Symptom Checklist (SCL-20)], medication adherence, and number of primary care visits for reasons other than depression.

The authors reported significantly greater adherence to antidepressant medication in the relapse prevention intervention group than the usual-care group (adjusted odds ratio 1.91, 95% CI 1.37 to 2.65; p < 0.001). Depressive symptoms (as measured by the SCL-20) improved in both groups over time, with a small but significant greater reduction for the intervention group (p = 0.04). However, the rates of relapse/recurrence for the intervention and usual-care groups (35% vs 34.6%) are almost identical, suggesting that the intervention did not prevent relapse relative to usual care over 12 months' follow-up. The authors suggested that a more intensive programme might be needed to reduce relapse rates.

The internal and external validity of this study were assessed using the quality appraisal checklist for quantitative intervention studies described in NICE's guide to methods for developing guidance in public health (see Appendix 4).44 This appeared to be a reasonably well-conducted RCT, although with some important limitations. Given the nature of the interventions, blinding of participants and clinicians was not possible, although the authors did not state whether or not the outcome assessors were blinded to allocation (which may have led to bias). Other concerns raised by the assessment were the lack of a power calculation and the methods use to adjust findings to account for missing data. However, given other strengths of the study, the reported lack of benefit for the relapse-prevention programme is unlikely to be due to a type II error (i.e. a ‘false-negative’ finding), but is likely to be a reasonably valid finding for the studied population. However, as with any such study comparing an intervention against ‘usual care’, it is difficult to separate benefits of the treatment programme per se from benefits of the attendant increase in support, engagement and monitoring that the intervention involves. In terms of external validity, the study population was drawn from four primary care clinics of one HMO in western Washington, USA. Participants were predominantly female, white, college educated and in paid employment. The findings of this study may not therefore be directly generalisable to more socially or ethnically diverse populations or to a UK primary care setting.

Ongoing studies

Three of the identified studies are ongoing RCTs.6971 Two of these studies are evaluating MBCT approaches,69,71 one alongside cognitive psychoeducation without any mindfulness content.71 The third trial is evaluating the impact of cognitive training self-help in addition to TAU.70 The available details of these studies are presented in Appendix 3.

© 2012, Crown Copyright.

Included under terms of UK Non-commercial Government License.

Cover of The Clinical Effectiveness and Cost-Effectiveness of Low-Intensity Psychological Interventions for the Secondary Prevention of Relapse After Depression: A Systematic Review
The Clinical Effectiveness and Cost-Effectiveness of Low-Intensity Psychological Interventions for the Secondary Prevention of Relapse After Depression: A Systematic Review.
Health Technology Assessment, No. 16.28.
Rodgers M, Asaria M, Walker S, et al.
Southampton (UK): NIHR Journals Library; 2012 May.

PubMed Health Blog...

read all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...