Results: panoramic meta-analysis

Beth Fordham; Thavapriya Sugavanam; Katherine Edwards; Karla Hemming; Jeremy Howick; Bethan Copsey; Hopin Lee; Milla Kaidesoja; Shona Kirtley; Sally Hopewell; Roshan das Nair; Robert Howard; Paul Stallard; Julia Hamer-Hunt; Zafra Cooper; Sarah E Lamb

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Fordham B, Sugavanam T, Edwards K, et al. Cognitive–behavioural therapy for a variety of conditions: an overview of systematic reviews and panoramic meta-analysis. Southampton (UK): NIHR Journals Library; 2021 Feb. (Health Technology Assessment, No. 25.9.)

Cognitive–behavioural therapy for a variety of conditions: an overview of systematic reviews and panoramic meta-analysis.

Show details

Contents

< Prev Next >

Chapter 5Results: panoramic meta-analysis

The map of the CBT review evidence base included 494 reviews. Of these, 171 reviews included data suitable for inclusion in the PMAs. The majority of the reviews reported in the mapping exercise, but excluded from the PMA, were not suitable because we could not extract the CBT RCT-specific data in isolation for any of the four outcomes (n = 279). This could be for any one of the following reasons:

The review may not have performed a meta-analysis or reported any quantitative data from single RCTs.
The review may have looked for RCTs reporting on the outcome but not identified any evidence.
We may have been unable to extract CBT RCT data in isolation, for example a review that presented a subgroup analysis of 10 CBT trials, but one of these trials was not a RCT. We could not isolate the purely CBT RCT evidence; therefore, the data were not included in the PMA. To have included these RCTs, we would have needed to return to the original RCT and perform a new meta-analysis including only the RCT data.

Of the 126 reviews eligible for the end-point PMA, 71³²^,³⁷^,³⁹^,⁴⁶^,⁵⁰^,⁵⁹^,⁶³^,⁶⁸^,⁸⁹^,¹⁰²^,¹¹⁷^,¹²⁶^,¹³⁴^,¹⁴³^,¹⁴⁹^,¹⁶⁵^,¹⁶⁷^–¹⁶⁹^,¹⁷⁵^,¹⁸⁸^,¹⁹³^,¹⁹⁷^–¹⁹⁹^,²⁰⁵^,²⁰⁶^,²¹¹^,²¹⁹^–²²¹^,²²⁷^,²³¹^,²³⁴^–²³⁶^,²⁴⁶^,²⁴⁹^,²⁵¹^,²⁵⁹^,²⁶¹^,²⁶⁷^,²⁷⁵^,²⁸⁶^,²⁹¹^,²⁹⁹^,³¹⁵^,³¹⁷^,³⁴⁰^,³⁴³^,³⁴⁷^,³⁵⁷^,³⁶⁹^,³⁷¹^,³⁷³^,³⁹⁷^,⁴⁰¹^,⁴⁰⁵^,⁴⁰⁹^,⁴³²^,⁴⁴⁵^,⁴⁴⁶^,⁴⁴⁸^,⁴⁵⁰^,⁴⁵⁴^,⁴⁶⁴^,⁴⁶⁹^,⁴⁸⁰^,⁴⁸⁴^,⁵⁰⁷^,⁵¹⁸ were higher-quality reviews (i.e. ‘moderate’ or ‘high’ on AMSTAR-2); the primary analyses for each outcome were conducted using these 71 higher-quality reviews. The PRISMA flow diagram describing review selection for the PMAs from the mapping stage is presented in Figure 5.

FIGURE 5

The PRISMA diagram from mapping to PMAs.

For reviews reporting data as change scores or dichotomous outcomes (RR, OR), separate PMAs for each outcome were undertaken; these are presented in Appendices 9–12.

Health-related quality of life

We identified 24 higher-quality systematic reviews³²^,³⁹^,⁶³^,¹⁶⁵^,¹⁸⁸^,¹⁹³^,²¹⁹^,²²⁰^,²³¹^,²³⁵^,²³⁶^,²⁷⁵^,²⁸⁶^,²⁹⁹^,³¹⁷^,³⁴³^,³⁴⁷^,³⁷¹^,⁴⁰⁹^,⁴⁴⁵^,⁴⁴⁶^,⁴⁶⁴^,⁴⁶⁹^,⁵¹⁸ that met the eligibility criteria to be included in the HRQoL PMA. One review³⁴³ included two meta-analyses for different disorders; hence, the number of comparisons is 25. These reviews included 49 RCTs, 4304 participants and represent 12 out of 40 ICD-11 categories (30%), as presented in Box 4. The white rows represent those ICD-11 codes that are represented in the primary analysis, the purple rows represent those conditions that are represented in the sensitivity analyses (see Sensitivity analysis) and the orange rows represent those ICD-11 codes that are not represented.

BOX 4

The ICD-11 categories (not) represented in the primary PMA of higher-quality reviews for the HRQoL outcome

The most commonly used measure of HRQoL was the Short Form questionnaire-36 items (SF-36) (n = 6). Other measurements included the Quality of Life Inventory (n = 5), the EuroQol-5 Dimensions (n = 2), the WHO Quality of Life-BREF (n = 1), the Global Assessment of Functioning (n = 1) and the Modular System for Quality of Life-54 (n = 1). The remaining reviews used population-specific (e.g. KIDSCREEN-27; KINDL-R; Comprehensive Quality of Life Scale, Intellectual Disability) or condition-specific [e.g. Fibromyalgia Impact Questionnaire, ADHD (attention deficit hyperactivity disorder) Impact Module-Adult™, Quality of Life in Alzheimer’s Disease, Diabetes Quality of Life for Youths] quality-of-life measurements.

Some reviews included trials with mixed characteristics, for example one review could include trials with adults and with older adults; in such a case, we would record that there was one review with adult data and one with older adult data. Consequently, the counts do not always add to 24 reviews. The mapping results demonstrate that the majority of these meta-analyses were focused on adults (n = 21), with only three reviews of children/adolescents and one review of older people. A higher number of reviews (12/24) had samples that included more female than male participants than reviews with samples of more male than female participants (6/24). Only three reviews reported the ethnicity of their samples: two reviews had samples with < 25% non-white participants and one included a sample with > 75% non-white participants.

The majority of reviews reported on the management of clinical conditions (16/24), through high-intensity CBT (17/24), delivered in outpatient settings (16/24), and with short-term follow-up (19/24). Seven reviews shared these three contexts but were conducted across different conditions. The majority of the included RCTs were from Europe, North America and Australasia (21/24).

The number of reviews containing only one trial was 6 out of 25; for some conditions, the numbers in each trial were very small (Figure 6). Comparators were active (8/24), mixed (3/24) and non-active (13/24).

FIGURE 6

Primary analysis of the primary outcome: HRQoL (end-point scores) from ‘high-quality’ reviews. GAD, generalised anxiety disorder. Adapted from Fordham et al. © The Author(s) 2021. Published by Cambridge University Press. This is (more...)

Primary analysis

Within-condition heterogeneity (I²) varied between 0% and 56%, and across-condition heterogeneity was 32%; hence, the criteria for PMA were met. The pooled across-condition SMD between control groups and CBT intervention groups gave a modest effect in favour of CBT on outcomes of HRQoL (SMD 0.23, 95% CI 0.14 to 0.33) (see Figure 6). Variation in effects was observed across conditions; for example, in aggression, the estimate mean effect was almost zero, although it was estimated with considerable uncertainty (SMD –0.02, 95% CI –0.28 to 0.32), whereas, in anxiety disorders, the estimated effect was positive and was estimated with much greater certainty (SMD 0.42, 95% CI 0.20 to 0.64). This heterogeneity is reflected in the resulting prediction intervals, which, indicated for the overall effect (within any given condition), were between –0.03 and 0.50, indicating, at worst (and with little support in the prediction interval), a small negative effect of CBT for some conditions and, at best, a large positive effect for other conditions.

No publication bias was detected using funnel plots (Figure 7) and Egger’s test showed that there were no small-study effects (p = 0.18).

FIGURE 7

The HRQoL funnel plot with pseudo-95% confidence limits (end-point data from high-quality reviews).

Mean difference in health-related quality of life

We identified a standard deviation (10.93 points) of the SF-36 physical composite score from a trial,⁵²⁵ deemed to have a low risk of bias, in a higher-quality review.⁴⁶⁴ The SMD translated to an estimated mean difference on the SF-36 of 3 points (95% CI 2 to 4 points).

Subgroup analysis

The only interaction test that was statistically significant was between reviews of CBT compared with active comparators and reviews of CBT compared with non-active comparators. All other subgroup interaction tests were not statistically significant and are, therefore, consistent with the general effect of CBT on HRQoL outcomes.

Cognitive–behavioural therapy intensity

Overall, high- and low-intensity CBT reviews were distributed evenly across the different conditions and characteristics. High- and low-intensity CBT reviews both included populations diagnosed with 6B00-06 anxiety or fear-related disorders, 6A60-80 mood disorders, 6A00-06 neurodevelopmental disorders, 6B20-25 obsessive–compulsive disorders, 21 pain and 6B40-45 disorders specifically associated with stress. They included patients with chronic symptoms (6A60-80 mood disorders and 21 pain). The reviews included children, adolescents and adults, of both sexes, from all care settings in Europe, North America, Australasia and Asia. Reviews of both intensities included long-term follow-up data. Reviews of high-intensity, but not low-intensity, CBT included (1) populations diagnosed with 6C20-21 disorders of bodily distress, 08 diseases of the nervous system or 6A20-25 schizophrenia or other primary psychotic disorders; (2) older adults; and (3) CBT delivered in a preventative context. Reviews of low-intensity, but not high-intensity, CBT were conducted in populations diagnosed with 6C40-4H addiction and MB23 aggressive behaviour.

There was little difference between effect estimates in reviews of high-intensity and low-intensity CBT, although heterogeneity was substantially higher for low-intensity CBT (SMD 0.23, 95% CI 0.03 to 0.42; I² = 68%) than for high-intensity CBT (SMD 0.21, 95% CI 0.11 to 0.32; I² = 0%) (Figure 8). The interaction test between high- and low-intensity CBT reviews was not statistically significant (p = 0.99).

FIGURE 8

The HRQoL subgroup analysis (end-point data from higher-quality reviews): CBT intensity. Note that two reviews, that combined high- and low-intensity CBT are not included here. GAD, generalised anxiety disorder.

We identified three reviews³¹⁸^,³⁴³^,⁵²¹ (four RCTs, 243 participants; two reviews of lower quality and one review of higher quality) that directly compared high- with low-intensity CBT interventions on HRQoL outcomes in 6B00-06 anxiety or fear-related disorders and 6A60-80 mood disorders. One review provided separate data for both 6B01 panic and 6B04 social anxiety disorder populations, and so the PMA included four meta-analyses. In this subset of direct comparisons, there was no difference between high- and low-intensity CBT (SMD 0.15, 95% CI –0.10 to 0.40; I² = 0%) (Figure 9).

FIGURE 9

Health-related quality of life: high- vs. low-intensity CBT, direct comparison PMA. SAD, seasonal affective disorder.

The direct evidence (see Figure 9) comparing high- with low-intensity CBT in 6B00-06: Anxiety and 6A60-80: Mood disorders supports our indirect evidence (see Figure 8) from subgroup analyses of high and low intensity. In summary, we have found no direct or indirect evidence that high- or low-intensity CBT produced different effect sizes.

Type of comparators

The choice of comparator had a significant effect on the treatment estimates. Comparison to an active intervention was associated with a very small effect (SMD 0.09, 95% CI –0.01 to 0.19; I² = 0%) (Figure 10). The active comparators tested in these reviews were education, exercise, pharmacotherapy, physiotherapy, psychotherapy/counselling and relaxation. Comparison with a non-active control was associated with a larger effect estimate (SMD 0.31, 95% CI 0.18 to 0.45; I² = 40%). The interaction test was statistically significant (p = 0.04).

FIGURE 10

The HRQoL subgroup analysis (end-point data from high-quality reviews): type of comparators. Note that three reviews,, with mixed active and non-active comparators are not included here. GAD, generalised anxiety disorder.

Duration of follow-up

Effect estimates were higher in reviews reporting short-term follow-up (SMD 0.29, 95% CI 0.17 to 0.42; I² = 30%) than in reviews reporting long-term follow-up (SMD 0.11, 95% CI 0.02 to 0.20; I² = 0%) (Figure 11). However, the interaction test did not find a statistically significant difference between the groups (p = 0.06).

FIGURE 11

The HRQoL subgroup analysis (end-point data from high-quality reviews): duration of follow-up. Note that one review with combined short- and long-term follow-up is not included here. GAD, generalised anxiety disorder.

Age

Effect estimates were similar in reviews of children and adolescents (SMD 0.20, 95% CI –0.15 to 0.56; I² = 0%) and adults (SMD 0.23, 95% CI 0.14 to 0.33; I² = 39%) (Figure 12). However, the sample sizes were much smaller in the reviews of children and adolescents, and the consequent CIs crossed zero. The interaction test did not find a statistically significant difference between the children/adolescents and adult groups (p = 0.06). The effect size for older adults was larger (SMD 0.39, 95% CI –0.24 to 1.02), but was generated from one review, with one trial and only 39 participants, and, again, the 95% CIs crossed zero (see Figure 12).

FIGURE 12

The HRQoL subgroup analysis (end-point data from high-quality reviews): age. GAD, generalised anxiety disorder.

Sensitivity analysis

The sensitivity analysis was conducted with an additional 10 reviews that had been rated as low or critically low on the AMSTAR-2. Therefore, the sensitivity analysis was conducted with 34 reviews (76 RCTs, 7466 participants).³²^,³⁹^,⁶³^,⁸²^,¹⁶⁵^,¹⁸⁸^,¹⁹³^,²¹⁹^,²²⁰^,²³¹^,²³⁵^,²³⁶^,²⁷⁰^,²⁷⁵^,²⁷⁶^,²⁷⁹^,²⁸⁶^,²⁹⁹^,³¹⁷^,³²⁹^,³⁴³^,³⁴⁷^,³⁵⁶^,³⁷¹^,⁴⁰⁹^,⁴¹³^,⁴⁴⁵^,⁴⁴⁶^,⁴⁶⁴^,⁴⁶⁷^,⁴⁶⁹^,⁵¹³^,⁵¹⁸^,⁵²¹ Inclusion of lower-quality reviews increased the estimate of effect (SMD 0.28, 95% CI 0.17 to 0.38) and raised the levels of heterogeneity (I² = 71%) (see Appendix 9, Figure 15). This analysis included reviews from more physical conditions: 13: Digestive system, 02: Neoplasms, 08: Headaches and epilepsy and 21: Symptoms such as tinnitus and fatigue (see Box 4). All of the additional within-condition group estimates were consistent with the general effect, that is, an absence of inconsistent effects.

We re-ran the PMA replacing the physical component scores with the mental component scores from the SF-12/SF-36 in the two reviews that presented both the physical and the mental component scores.²²⁰^,⁴⁶⁴ The replacement did not change the overall effect or heterogeneity rating for the HRQoL outcome (SMD 0.24, 95% CI 0.14 to 0.33; I² = 38%) (see Appendix 9, Figure 16).

Health-related quality-of-life change scores and risk ratio data

Four reviews (four RCTs, 185 participants), two of higher and two of lower quality, presented HRQoL data as change scores.¹⁵⁸^,²⁴⁶^,⁴⁰⁶^,⁵²³ These included reviews of 6B00-06 anxiety and 6A60-80 mood disorders, 13 digestive system, 21 pain and 12 respiratory system disorders. The overall pooling reported acceptable heterogeneity and a moderate effect in favour of CBT (SMD 0.58, 95% CI 0.15 to 1.00; I² = 66%) (see Appendix 9, Figure 17).

One lower-quality review (two RCTs, 145 participants) presented HRQoL data as a RR.¹⁷⁰ This review identified a large effect (SMD 1.57, 95% CI 0.78 to 2.37; I², not applicable) in favour of CBT (see Appendix 9, Figure 18).

Discussion

From the highest-quality reviews, we found that CBT produced consistent, positive effects on HRQoL across 10 different conditions. Effect estimates suggest a modest, long-term improvement, compared with no intervention. These effects became very small when CBT is compared with other active treatments, including education, exercise, pharmacotherapy, physiotherapy, psychotherapy/counselling and relaxation. We did not find a difference between the effect sizes of reviews conducted with low-intensity CBT or high-intensity CBT.

The effect estimates were generated by synthesising data from samples of children, adolescents and adults, of both sexes, mainly living in countries in Europe, North America and Australasia. There is a lack of higher-quality evidence of CBT’s effectiveness for older adults.

We do not know if CBT will be effective when delivered preventatively or when delivered to patients with severe or subclinical symptoms. We do not know if CBT is equally effective across different ethnic groups nor do we know its effect for people living in countries in Africa, Asia or South America.

Depression

We identified 48 higher-quality systematic reviews³⁷^,³⁹^,⁴⁶^,⁵⁰^,⁵⁹^,⁶³^,¹¹⁷^,¹²⁶^,¹⁴³^,¹⁶⁷^–¹⁶⁹^,¹⁷⁵^,¹⁹⁷^–¹⁹⁹^,²⁰⁵^,²⁰⁶^,²²⁰^,²²¹^,²³¹^,²³⁴^,²³⁵^,²⁴⁶^,²⁴⁹^,²⁶¹^,²⁶⁷^,²⁷⁵^,²⁸⁶^,²⁹⁹^,³⁴⁰^,³⁵⁷^,³⁶⁹^,³⁷¹^,³⁷³^,⁴⁰¹^,⁴⁰⁵^,⁴⁰⁹^,⁴³²^,⁴⁴⁵^,⁴⁴⁶^,⁴⁴⁸^,⁴⁵⁰^,⁴⁵⁴^,⁴⁶⁹^,⁴⁸⁰^,⁴⁸⁴^,⁵⁰⁷ that met the eligibility criteria to be included in the primary depression PMA. One review included six meta-analyses for different disorders; hence, the number of comparisons is 53. These included 130 RCTs and 14,073 participants, and represent 16 out of 40 possible ICD-11 categories (40%). Box 5 includes the ICD-11 codes represented in the primary PMA (white cells) and those codes not represented (shaded cells).

BOX 5

The ICD-11 categories (not) represented in the primary PMA of higher-quality reviews for depression outcome

The most commonly used measure of depression was the Beck Depression Inventory (n = 22). Other measurements included the Hamilton Depression Rating Scale (n = 7), the Hospital Anxiety and Depression Scale (n = 4), the Patient Health Questionnaire-9 items (n = 2), the Montgomery–Åsberg Depression Rating Scale (n = 2), the Center for Epidemiologic Studies Depression Scale (n = 2), the Profile of Mood States (n = 1), the Depression Anxiety Stress Scale (n = 1) and the Hopkins Symptoms Checklist (n = 1). The remaining reviews used population-specific depression measures [Glasgow Depression Scale for People with a Learning Disability (n = 1), Children’s Depression Inventory-revised (n = 3)].

The majority of these meta-analyses were focused on adults (37/48), with seven reviews focusing on adolescents/children and one review focusing on older people. More reviews had samples that included more female than male participants (23/48) than samples with more male than female participants (9/48). Only seven reviews reported the ethnicity of their samples. Of these, four reviews had samples with < 25% non-white participants and one included a sample with > 75% non-white participants.

The majority of reviews reported on the management of clinical conditions (26/48), on interventions of high intensity (26/48), delivered in outpatient settings (27/48), and with short-term follow-up (39/48). The majority of included RCTs in the reviews were from Europe, North America and Australasia (33/48). Many of the reviews contained only one trial (54%, 26/48), and, for some conditions, such as personality disorders, the numbers in those trials were very small (see Appendix 10, Figure 19).