NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Pavey TG, Anokye N, Taylor AH, et al. The Clinical Effectiveness and Cost-Effectiveness of Exercise Referral Schemes: A Systematic Review and Economic Evaluation. Southampton (UK): NIHR Journals Library; 2011 Dec. (Health Technology Assessment, No. 15.44.)

Cover of The Clinical Effectiveness and Cost-Effectiveness of Exercise Referral Schemes: A Systematic Review and Economic Evaluation

The Clinical Effectiveness and Cost-Effectiveness of Exercise Referral Schemes: A Systematic Review and Economic Evaluation.

Show details

4Systematic review of the cost-effectiveness of exercise referral schemes

Introduction

A systematic review of the literature was conducted to identify economic evidence on ERS as defined in the earlier stages of this report, i.e. schemes that involved referral from a primary health-care professional due to an underlying condition and access to a structured programme of exercise. Both economic evaluations and existing systematic reviews of economic evidence on exercise referral were considered for inclusion. By adhering to a relatively narrow definition of what constitutes ERS, a number of studies exploring the cost-effectiveness of PA were excluded on the basis that (1) they did not include a referral from a health-care professional; (2) they did not consider a population with an underlying health condition; or (3) they did not comprise a structured programme of exercise. In this respect, the findings of this economic review are intended to mirror those of the effectiveness review presented in Chapter 3 of this report.

Methods

This review was conducted and reported in accordance with the PRISMA statement.36

Search strategy

Studies were identified using the methods described in Chapter 3. For inclusion in this economic systematic review, studies had to satisfy all the inclusion criteria outlined in Chapter 3 and also include cost and/or cost-effectiveness data. Studies for possible inclusion were initially identified by reviewing titles.

Study selection

As described in Chapter 3.

Data extraction and critical appraisal methods

A data extraction framework was established to abstract information from economic evaluations identified for inclusion. For each study, data were extracted on the following: study objective, population characteristics, nature of the intervention and comparator, cost and cost-effectiveness findings and methodological strengths and weaknesses. Primary economic studies considered for review were formally appraised against recognised appraisal criteria for economic evaluations74 and, where appropriate, decision-analytic models.75 Data extraction was conducted independently by one reviewer (NA) and checked by a second (PT). Discrepancies were resolved by discussion within the research team. Systematic reviews identified as part of the literature search were also considered for inclusion.

Data synthesis

The findings of both the economic evaluations and systematic reviews identified are presented descriptively in the form of detailed tabular summaries. Given that only a small number of primary studies were included in the review, a summary of each study, along with a commentary on the methods used, is provided below.

Results

Identification and selection of studies

The bibliographic searches identified three economic evaluations50,61,70 of ERS that met our inclusion criteria (UK, n = 2; non-UK, n = 1). In addition, we included a model-based economic evaluation of brief interventions designed to promote PA developed to inform public health guidance issued by NICE.76 This NICE evaluation considered ERS as one method of promoting PA in primary care. Although not published in a peer-reviewed journal, the full report of the study was available in the public domain (available at www.matrixknowledge.com/…/physical_activity_economic_modelling_report_april2006.pdf).

In addition to the primary economic evaluations, three systematic reviews of ERS were identified,40,41,77 which included consideration of cost-effectiveness. Findings from the reviews and primary studies are reported separately below. See Figure 13 for details.

FIGURE 13. Study inclusion process for ERS cost-effectiveness systematic review.

FIGURE 13

Study inclusion process for ERS cost-effectiveness systematic review. PHC, primary health care.

Findings of previous systematic reviews

Two systematic reviews of the effectiveness of ERS included consideration of the cost-effectiveness evidence on ERS.40,41 A quality appraisal of these systematic reviews is presented in Chapter 3. A third systematic review,77 conducted to inform the development of NICE guidance, specifically considered evidence on the cost-effectiveness of ERS.

Table 24 summarises the objectives, methods and findings of the systematic reviews and highlights notable differences in the definition of ERS and the inclusion criteria applied. All three studies considered referral to exercise by a health-care professional in primary care. However, the review conducted to inform the development of NICE guidance adopted a broader definition of interventions, including the use of pedometers and community-based interventions as well as exercise referral. Although the NICE review focused specifically on economic evidence, the other reviews considered economic evidence alongside the evidence on clinical effectiveness, including uptake levels of PA and other effectiveness outcomes.

TABLE 24. Summary of systematic reviews of cost-effectiveness of ERS.

TABLE 24

Summary of systematic reviews of cost-effectiveness of ERS.

The findings of the three reviews differ somewhat. The review conducted for NICE77 concluded that most brief interventions to promote PA are marginally more costly than a ‘do-nothing’ alternative, but generate improved long-term outcomes. The evidence relating to exercise referral was equivocal, with one study reporting that intervention was less costly and more effective (i.e. a dominant strategy) than the comparator, three studies reporting it to be more costly and more effective, and one study reporting it to be more costly and equally effective. On balance the authors indicate that the economic case for brief PA promotion interventions is largely positive, although the authors highlight concerns about the applicability of some of the evidence considered to the NHS.

The review by Sorensen et al.40 indicated ERS to be a cost-effective intervention compared with usual care. This finding appears to be based on a single economic study.78 Williams et al.41 examined three UK-based studies and concluded that there is little evidence to suggest that ERS improves outcomes. On this basis, they conclude that an ERS is marginally more costly than usual care, but that inadequacies in the evidence of effectiveness mean that it is not possible to determine whether or not it is a cost-effective use of resources.

The degree to which the conclusions of the reviews differ is, at least in part, due to differences in the inclusion criteria adopted by the reviews. Table 25 shows the lack of consistency in the studies included in the reviews.

TABLE 25. Studies included in previous systematic reviews of cost-effectiveness of ERS.

TABLE 25

Studies included in previous systematic reviews of cost-effectiveness of ERS.

Given the variation in the definition of ERS used, it is unsurprising that there were inconsistencies in the number of primary studies identified for inclusion in each of the reviews. This, together with the publication of recent trials of ERS, underscored the need for a de novo systematic review that used a standardised definition of ERS. The findings of this de novo review are presented in the following sections.

Findings of primary economic evaluations

Four economic evaluations were identified for inclusion in this systematic review. These comprised three trial-based economic evaluations of ERS50,61,70 and one model-based evaluation76 of the cost-effectiveness of brief interventions in primary care to promote PA, including ERS. Three of the studies were based on UK populations,50,61,76 whereas one trial-based analysis was conducted in Spain.70 Given the number of studies identified, a summary of each study is presented below along with a commentary on the quality of the study and the implications of the findings (detailed data extraction in Appendix 5).

Trial-based economic evaluations

Stevens et al.50 assessed the cost-effectiveness of a primary care-based intervention aimed at increasing levels of PA in inactive people aged 45–74 years (further details of the study design, population and interventions are available in Chapter 3). The study comprised an economic evaluation conducted alongside an RCT. A within-trial analysis was undertaken and no attempt was made to extrapolate the findings beyond the duration of the study (8 months). Although not explicitly stated, the perspective of the analysis appears to be that of the health service. Costs were derived in a top-down manner, i.e. the total costs of administering the ERS scheme were divided by the number of participants to generate a mean cost per participant. Some adjustment was made to exclude costs associated with the research, as differentiated from administration of the intervention. As a result, it was not possible to report disaggregated estimates of resource use and costs.

Evidence on costs was synthesised with evidence on effectiveness to generate cost-effectiveness estimates. A number of outcomes were considered in this process. The primary outcome in the analysis was the cost of promoting one sedentary person to undertake more PA. The cost of doing so was £623. A second analysis considered the cost involved in moving a moderately active individual to the minimum recommended level of PA. This was achieved at a cost of £2498. Finally, the cost of moving an individual to the next level of PA (defined as sedentary, low intermediate, high intermediate and active) was reported as £327.

One-way sensitivity analyses were conducted to explore parameter uncertainty. The findings were found to be sensitive to changes in the response rate, leading the authors to conclude that particular attention should be paid to recruitment strategies in setting up ERS. Furthermore, given the top-down approach to costing, the cost of the intervention is dependent on the number of recipients, and the authors point out that the marginal cost of the intervention is expected to fall if the number of recipients can be increased.

Isaacs et al.61 conducted an economic evaluation alongside the UK Exercise Evaluation Randomised Trial (EXERT), which compared the effectiveness of a leisure centre-based (ERS) programme, an instructor-led walking programme and advice only in patients referred for exercise by their GPs. (Further details of the trial design, study population and interventions can be found in the effectiveness review in Chapter 3.) A cost-effectiveness analysis was conducted alongside the trial. Outcomes were reported at 6 months and 12 months post intervention (determined by the trial duration) and a partial societal perspective to costing was adopted, capturing costs incurred by the NHS, local government and participants. Attempts were made to provide a detailed assessment of the costs involved in the provision of the interventions. Intervention costs included costs to the provider and the participant, as well as any equipment costs that might be incurred. In addition to this, the study also captured information on GP and hospital consultations and pharmaceutical use prior to the intervention and over the course of the study through a case note review, to determine whether or not PA had any influence on general health-care resource consumption. Detailed costs for the control group and both intervention groups derived from the study are presented in Table 26.

TABLE 26. Description of cost of ERS (adapted from ref. 59).

TABLE 26

Description of cost of ERS (adapted from ref. 59).

Commentary on Stevens et al.50

The study was reported to be based on the largest RCT trial of PA promotion conducted in the UK and, as such, provides a valuable source of economic evidence on ERS. Methodologically, the study is a reasonable attempt to estimate the cost-effectiveness of an intervention alongside a trial (see Table 28). However, there are some methodological weaknesses, some of which are acknowledged by the authors. The use of a top-down costing methodology is a limitation and raised challenges for the authors in deriving an accurate estimate of the cost per participant. In particular, there are challenges about whether or not recruitment can be increased at a modest additional cost once the programme is up and running. If this were possible, then it would be possible to reduce the cost per participant significantly by increasing the number of participants. A further challenge relates to the outcome measures considered in the analysis. Although these are perfectly legitimate and translate into meaningful measures of effectiveness, it would have been desirable to present the findings in the form of a cost–utility analysis, reporting an incremental cost per quality-adjusted life-year (QALY) or similar outcome. Best practice recommendations for cost-effectiveness analysis developed by NICE in England and Wales identify the use of cost–utility analysis based on preference-based outcome measures as the preferred end point for economic evaluations, as they allow for comparison between different interventions and populations. The absence of this makes interpretation of the findings somewhat challenging for a healthcare policy audience. Finally, the economic evaluation is essentially a within-trial analysis and, as such, adopts a relatively short time horizon. Previous research has indicated that the cost-effectiveness of public-health interventions is likely to be dependent not just on their short-term effect, but also on the degree to which any behaviour change is lasting. As such, an attempt to model the benefits over a longer time horizon may provide a richer source of information for health-care planners, acknowledging that this would introduce a greater degree of uncertainty.

TABLE 28. Summary of the findings of included ERS economic evaluations.

TABLE 28

Summary of the findings of included ERS economic evaluations.

The mean cost of the leisure centre ERS intervention over 12 months was estimated to be £186 to the providers, with a further £101 being incurred by participants.

Outcomes were measured using the SF-36. The authors' state that their intention was to convert SF-36 score into quality-adjusted life-years (QALYs); however, this was not possible owing to instability in the findings. Incremental cost-effectiveness ratios (ICERs) were generated in the form of the incremental cost per unit change in SF-36 score. A comparison of leisure centre-based interventions with controls resulted in an incremental cost of £19,500 per unit change in SF-36 score at 6-month follow-up.

Parameter uncertainty was explored through probabilistic sensitivity analysis (PSA). The findings suggest that there is a low probability of the leisure centre intervention being dominated by the control group.

The objective of Gusi et al.,70 the only non-UK-based study considered herein, was to examine the cost/utility of adding a supervised walking programme to standard ‘best care’ in individuals who are obese or depressed. The economic study was conducted alongside a study of the effectiveness of this intervention in four general practices in Spain. Although non-UK, the Gusi et al.70 paper highlights the ERS model and references other ERS studies for comparison.

A cost–utility analysis was undertaken adopting a health-care provider's perspective and a time horizon of 6 months. Costs considered included the costs of staffing the intervention, as well as the costs of medication and consultations. However, no difference was seen between the intervention group and the controls in the latter, so the incremental cost of the intervention group comprised only the staff costs involved in delivery. Outcomes were measured using the EQ-5D utility scale.

The findings show that the exercise programme led to an incremental QALY gain of 0.132 over a 6-month period, at an incremental cost of €41 per participant, generating an ICER of €311/ QALY. Sensitivity analyses, including PSAs, were presented. One-way sensitivity analysis showed the findings to be relatively robust to changes in parameter estimates, with the worst-case scenario ICER increasing to €811/QALY. PSA showed a high probability of the intervention remaining cost-effective when extreme parameter values were considered.

Commentary on Isaacs et al.61

The study is a useful complement to the existing evidence base on the cost-effectiveness of ERS. Particular mention should go to the effort put into generating detailed estimates of the cost of the intervention to providers and participants. (These estimates have been used in the modelling work presented in the later parts of this report.) The main limitation of the study appears to be the inability to convert the findings presented in the form of SF-36 scores into utility scores that might allow for the derivation of QALYs. The authors acknowledge this as a limitation, although there is relatively little explanation given for why this was not possible (e.g. this could be due to missing data in responses). The other major limitation of the study is the relatively short time horizon that was dictated by the trial design. However, this is true of many of the studies considered in this review and reflects the difficulties that are inherent in conducting long-term RCTs of interventions designed to change behaviour. Estimation of long-term outcomes is important as it allows us to verify the main differences among the alternative options with respect to costs and benefits.83 However, it is important to note that it is often difficult to extrapolate beyond the observed data on health gains because there is lack of evidence surrounding (1) post-intervention effects on PA behaviour (do participation levels stay constant, decline or increase?) and (2) the nature of the relationship between PA and health gains over time.84

Commentary on Gusi et al.70

This study performs well when considered in relation to critical appraisal checklists for economic evaluation and best-practice principles (see Table 5). Estimates of cost and outcomes are presented clearly and the study benefits from the use of the EQ-5D, allowing the authors to generate ICERs in the form of cost/QALY. This allows for comparison with other interventions both in the field of public health and beyond, with the findings suggesting the intervention is likely to be highly cost-effective when compared with accepted thresholds. For our own purposes, the main limitation appears to be the degree to which the intervention and the findings are relevant to a UK population. Given the relatively limited information available, it is difficult to determine whether or not this intervention could be easily reproduced in the NHS at a similar cost and effectiveness.

Economic modelling studies

Only one economic modelling study that attempted to estimate the longer-term costs and benefits of exercise referral was identified as part of this search. This NICE76 study comprised an evaluation of primary care-based interventions designed to promote PA, including exercise referral. The study was commissioned to help inform the development of NICE public health guidance on PA.

A cost–utility analysis was conducted using a decision-analytic model to examine the cost-effectiveness of four interventions. The model considers a cohort of individuals who enter the model in a sedentary state. The individuals are exposed to an intervention (exercise referral) which is assumed to affect their likelihood of becoming physically active.

Physical activity is assumed to have a long-term effect on an individual's likelihood of developing a number of chronic conditions. Conditions included in the model were selected on the basis that there was evidence of a strong causal relationship between PA and evidence on the magnitude of effect of PA on the incidence of these conditions. Conditions included in the analysis were CHD, stroke, type 2 diabetes mellitus and colon cancer.

Estimates of the RR of developing each of these conditions, depending on PA status, were derived from published sources. The conditions are assumed to be independent of one another and individuals are permitted to experience only one condition within the confines of the model. Estimates of mortality rates and life-years lost associated with each condition were derived from published sources and derived by assuming an average age at onset for each condition, dependent on the age of the population under consideration. Utilities and unit costs associated with each condition were synthesised from multiple published sources.

Outcomes are reported both as cost per person who moves from a sedentary state to a physically active state as well as in the form of cost per QALY. The cost of moving an individual from a sedentary state to a physically active state ranged from £90 to £4500, dependent on the cost of the intervention. The incremental cost per QALY ranged from around £20 to approximately £670, dependent on the cost of the intervention.

Further analyses considered the potential savings that may accrue from reductions in future health-care resource consumption as a result of being physically active. This analysis generated even more favourable cost-effectiveness ratios, which, in most cases, were dominant (that is ERS is cheaper and more effective than the control).

One-way sensitivity analysis explored changes in persistence with exercise (i.e. dropouts), intervention costs and effectiveness. The authors report that the intervention remains cost-effective under most scenarios considered in the analysis.

Commentary on the NICE study76

Unlike the primary studies conducted alongside trials presented above, this modelling study attempts to estimate the longer-term impacts of PA. Any model should be considered a simplification of the real world and the authors acknowledge many of the weaknesses inherent in their analysis. For example, the model considers only a small number of conditions that have been associated with physical inactivity, while excluding many others, such as musculoskeletal disease and respiratory illness. However, this can be justified on the basis of the available evidence on the relationship between PA and these conditions.

In addition to this, the model adopts a fairly simplistic approach to the long-term effectiveness of interventions designed to promote PA, assuming that around 50% of individuals fail to adhere to any intervention for a long enough period to experience reductions in the risk of future events. This rate is not explored in any depth and further attempts are warranted to estimate the degree to which behaviour change is lasting as this is likely to have a significant effect on the cost-effectiveness of interventions.

Other simplifications in the model include the approach to estimating life-years lost, the assumption of independence of the conditions considered and the assumption that individuals experience only one of the conditions. Clearly, these assumptions are unlikely to apply in real life, particularly the assumption that the incidence of CHD, stroke and diabetes are unrelated. However, as with any model, it is relatively easy to take issue with simplifications and assumptions which have been adopted due to the absence of data. In many of these instances, there are relatively few options for improving the model until further long-term evidence becomes available.

One consideration for future research might be whether or not the simple decision-analytic approach to modelling is warranted in this indication. Given that individuals' behaviours may change over time, it may be that a more dynamic approach to modelling the cost-effectiveness of PA is warranted, although once again this may be limited by the available evidence. In light of this, the model described above provides a useful contribution to the primary evidence on cost-effectiveness presented earlier in this section. The model has also provided a basis for the economic modelling presented in the later stages of this report, although some modifications have been made while further consideration has been given to issues such as uptake and adherence with interventions.

Quality assessment

Studies were reviewed against criteria laid out in critical appraisal checklists for economic evaluations. In general, the studies performed well, particularly with regard to clarity of presentation of the results. There were some deficiencies in relation to the reporting of input parameters, although in many cases these were identified as limitations by the authors. A summary of the characteristics of the economic evaluations is presented in Table 27.

TABLE 27. Quality assessment of included ERS economic evaluations.

TABLE 27

Quality assessment of included ERS economic evaluations.

Summary of the economic evidence and critical appraisal

A summary of the findings of the economic evidence considered above is presented in Table 28. All studies found the ERS interventions to be cost-effective compared with the controls. However, one study61 attempted to compare an alternative PA intervention with ERS and found that a walking-based intervention is likely to be relatively more cost-effective than leisure centre ERS intervention, with the former leading to a cost saving of £8750 per unit increase in HRQoL scores as measured by SF-36. It would be reasonable to surmise that the available economic evidence on ERS suggests that it appears to be a cost-effective use of health-care resources.

Only one of the economic studies adopted a decision-analytic approach that was suitable for review against best-practice principles for economic modelling. Table 29 highlights the aspects of the guidelines for decision-analytic modelling that were found not to have been addressed by the study.76 The problems mainly related to the lack of information on validation of the model against existing evidence and incomplete assessment of uncertainties. Regarding the latter, the study focused on parameter uncertainty tending to ignore the other types of uncertainty such as methodological and structural uncertainty.

TABLE 29. Quality assessment for included ERS decision-analytic model.

TABLE 29

Quality assessment for included ERS decision-analytic model.

Summary

  • Given the lack of standardisation of the ERS definition used by previous systematic reviews and the publication of further recent evidence, we undertook a de novo systematic review of cost-effectiveness of ERS.
  • Our systematic review identified only four primary economic evaluations that assessed the cost-effectiveness of ERS – three trial-based economic evaluations and a model-based analysis (commissioned by NICE as part of the development of guidance on brief interventions in primary care for the promotion of PA).
  • Broadly, the previous evidence base suggests that ERS is a cost-effective intervention in sedentary, but otherwise healthy populations. However, there is some significant uncertainty around the estimates of cost-effectiveness because of an absence of evidence on the long-term effectiveness of these interventions. Although modelling studies can go some way to exploring this, ultimately these issues can only be resolved through better evidence of effectiveness derived from RCTs or other well-designed observational studies. As such, any criticism of the economic evidence should be considered in light of the evidence on effectiveness available at the time of the analysis.
  • Each of the previous economic evaluations has its merits and makes a valuable contribution to the limited evidence base on the cost-effectiveness of ERS. The trial-based studies benefit from a high degree of internal consistency, deriving their estimates of effectiveness from the trial and, in some cases, detailed estimates of the cost of the interventions. Any weaknesses inherent in these analyses are also largely as a result of the limitations of the trials, particularly the degree to which the findings can be considered to be externally valid and the relatively short follow-up that was achievable in a trial setting.
  • The NICE economic modelling study overcomes the issue of the short-time horizon inherent in the trial-based analyses. This study allowed for an estimate of the longer-term costs and benefits of PA, taking into account the effects on a number of long-term conditions that are known to be associated with physical inactivity. There are many weaknesses associated with the model although many of these result from an absence of evidence on the effectiveness of ERS (e.g. on the relationship between physical inactivity and long-term conditions, long-term effectiveness of interventions, adherence to interventions etc.). It should also be remembered that any economic model can only ever be a simplification of reality. In an area as complex as PA and behaviour change, and an area characterised by limitations in the evidence base, the need for simplification may be great, leading to a model that fails to meet many of the best-practice criteria.
  • A further limitation of previous economic evaluations is their focus on a sedentary, but otherwise healthy population. Few of the studies explicitly consider whether or not ERS can contribute to improved outcomes in populations with underlying conditions (with the exception of Gusi et al.,70 which was conducted outside the UK).
  • In light of these findings, we decided to develop a de novo economic model to assess the cost-effectiveness of ERS. Our model builds on the principles of the NICE decision-analytic model, which includes some important further development of the methods and a more robust approach to the incorporation of ERS effectiveness evidence. The findings of this analysis are presented in Chapter 6.
© 2011, Crown Copyright.

Included under terms of UK Non-commercial Government License.

Bookshelf ID: NBK97781

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (4.8M)

Other titles in this collection

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...