Home > A cost effectiveness model for the...

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.

National Collaborating Centre for Chronic Conditions (UK). Chronic Heart Failure: National Clinical Guideline for Diagnosis and Management in Primary and Secondary Care. London: Royal College of Physicians (UK); 2003. (NICE Clinical Guidelines, No. 5.)

  • This publication is provided for historical reference only and the information may be out of date.

This publication is provided for historical reference only and the information may be out of date.

Appendix GA cost effectiveness model for the appropriate use of echocardiography in the diagnosis of heart failure

Summary

This paper represents a rapid attempt to use economic modelling to inform the recommendations on the most efficient use of echocardiography facilities in the diagnosis of heart failure. The question studied is:

Should all patients presenting to a GP with suspected heart failure get BNP & ECG & echocardiography as a matter of course, or should they initially undergo an ECG and BNP and only get an echocardiogram if the first two tests suggest an abnormality?

This question is considered by calculating the additional cost per life year gained of the first strategy on the assumption that a diagnosis of heart failure provides access to treatment not otherwise available to the patient (in this case ACE inhibitors).

Using our best estimates from UK data, the baseline cost per life year gained of immediate echocardiography is around £16,000. This would normally be considered cost effective in the context of NICE’s historical decisions regarding drug therapies. However, the cost effectiveness of diagnostic technologies, especially in chronic conditions, is rarely investigated and therefore it is difficult to compare this figure with anything else.

The issue of appropriate referral to echocardiography is key. If the proportion of patients being sent for echocardiography who actually turn out to have heart failure is low, then immediate use of echo does not look cost effective. If we can use other tests (in this case ECG and BNP) to filter out patients who do not require an echocardiogram, this could result in more efficient use of echo facilities.

Evidence from open access services should give us a reasonable estimate of the ‘prevalence’ in this population if echocardiography is made more readily available. If this proportion falls below 25%, the cost per LYG shoots up. For example, the Davie et al 306 study in Edinburgh suggests that 18% of patients referred by GPs for echocardiography actually turned out to have heart failure. This would result in a cost effectiveness ratio of > £30,000 using the model developed here.

In addition the sensitivity of BNP and ECG (combined) is also important; the higher the sensitivity the higher the cost per LYG of immediate echocardiography. The results are sensitive to key model parameters (see table opposite).

Introduction

In developing recommendations for the diagnosis of heart failure the guideline development group were aware that a policy of referring all cases of suspected heart failure for echocardiography would have substantial service implications, and create problems for GPs who do not have open access to echocardiography.

While it was generally agreed that an echocardiogram was necessary to confirm a diagnosis of heart failure and provide information on the underlying functional abnormality of the heart, it was also suggested that a diagnosis of heart failure could, in many cases, be ruled out by carrying out a full clinical examination and tests including a chest X-ray, 12 lead electrocardiogram and natriuretic peptide (BNP or NT proBNP) testing. If these tests do not reveal any abnormality then a diagnosis of heart failure is very unlikely.

In order to aid the guideline development process a model has been constructed to investigate the possible costs and benefits of two alternative recommendations relating to the use of echocardiography in the diagnosis of heart failure; on the one hand echocardiography could be carried out on all patients who present to the GP with possible symptoms of heart failure, or on the other hand echocardiography could be reserved for those patients who have an abnormality revealed by the other three diagnostic tests described above.

The model is a simplified version of the real clinical situation and is characterised by much uncertainty around some of the key parameters. However, no attempt at modelling the cost effectiveness of echocardiography in diagnosing heart failure has been attempted before, so the model represents a step forward in that respect. It is hoped that the model can provide some useful information to help formulate the guidelines on diagnosis.

A brief description of the modelling choices

The diagnosis modelling exercise is necessarily a simplified version of the real clinical situation. It is based on the assumption that there are three main diagnostic tests for heart failure; ECG, BNP and echocardiogram.*

One key question is formulated: Should all patients presenting to a GP with suspected heart failure get all three tests (including echocardiography) as a matter of course, or should they initially undergo an ECG and BNP and only get an echocardiogram if the first two tests suggest an abnormality?

Considering a person with suspected heart failure, there are four possible scenarios relating to the tests and their results:

  1. Carry out two tests (ECG, BNP), neither are abnormal.
  2. Carry out two tests, one is abnormal therefore carry out an echocardiogram.
  3. Carry out two tests, two are abnormal therefore carry out an echocardiogram.
  4. Carry out two tests plus an echocardiogram as a matter of course.

Scenarios 2, 3, and 4 all have the same costs (of the three tests) and detection rate, suggesting only two important alternative strategies:

  1. carry out ECG and BNP initially and only refer to echocardiography if an abnormality is identified by the two tests; or
  2. carry out all three tests initially.

If the problem is set up in this way the key issue becomes:

How many cases of heart failure would carrying out only ECG and BNP miss? That is, if these two tests show no abnormality but echocardiography would have picked up an abnormality? The answer to this rests on the sensitivity and specificity ECG and BNP in diagnosing heart failure. To simplify the model it is assumed that echocardiography is the diagnostic gold standard with both sensitivity and specificity of 100%. (The appropriateness of this assumption is discussed below.) We also assume that missed diagnoses are not ‘caught’ within the four-year period considered.

Costs of each strategy include the costs of the tests, the costs of treating heart failure with drugs and the hospitalisation costs for treated and untreated heart failure. The benefits will be expressed in terms of life years gained (LYG) from heart failure treatment, assuming that a diagnosis of heart failure means access to treatment that has positive effects in terms of life years gained and reduced hospitalisation. No attempt is made to adjust the life years gained with quality weights since there is no adequate source of information on quality of life in treated and untreated heart failure patients.

A simplified representation of the model

A simple schematic of the two strategies is shown in Figure 1: strategy A in which a patient only receives an echocardiogram if there is an abnormality on either or both of the first two tests; strategy B where all three tests are carried out as a matter of course.

Figure 1. Strategies for the diagnosis of heart failure.

Figure 1

Strategies for the diagnosis of heart failure.

In a very simple example, for 100 patients, say that strategy A (ECG and BNP) finds no abnormality, but strategy B detects heart failure in two patients via echocardiography. Then if strategy A is carried out two diagnoses will be ‘missed’, that is those patients will not have access to treatment for heart failure, whereas they would have done under strategy B.

The relative costs and benefits of these two strategies are summarised in Table G2 overleaf.

Table G2. Relative costs and benefits of strategies A and B.

Table G2

Relative costs and benefits of strategies A and B.

The model in more detail

The model traces a hypothetical cohort of 100 adults who present to the GP with possible symptoms of heart failure. The model traces this cohort for a period of four years. The time period is limited to four years as this is the maximum follow-up time available to compare outcomes for treated and untreated heart failure using data from the SOLVD trial.121 No attempt has been made, at this stage, to extrapolate findings beyond trial follow-up.

The parameter values and evidence sources for the baseline scenario show (Table G3) that the additional costs of carrying out an echocardiogram on all 100 patients as a matter of course, as opposed to using ECG and BNP as a filter, is £2,062. However the life years gained from this strategy are very small at only 0.13 life years for 100 patients. This results in a baseline cost per life year gained of £15,928.

Table G3. Baseline scenario.

Table G3

Baseline scenario.

Model parameters, baseline values and one-way sensitivity analysis

Prevalence of heart failure in this population

This is an estimate of the number of patients referred by a GP with possible heart failure who actually receive a diagnosis of heart failure after further investigation. The baseline estimate of 29% comes from the Hillingdon Heart Failure Study (Cowie et al 1999340). This study identified incident cases of clinical heart failure developing in a population of 151,000 people served by 81 GPs in 31 practices in the Hillingdon district, west London. The GPs agreed to refer all suspected cases of new heart failure to a rapid access study clinic, preferably before the patient had started treatment. Between April 1995 and July 1996, 122 patients were referred.

There is a large amount of uncertainty surrounding this parameter and estimates in the range 18% to 50% have been obtained in other studies on similar populations. In particular, the GPs in the Hillingdon study may have had higher than average awareness of heart failure. In one-way sensitivity analysis increasing the prevalence estimate results in a reduction in the cost per LYG. The cost per LYG is very sensitive to this parameter especially at high values (see below for more detail on key parameters).

In the simple model the patient cohort is assumed to be approximately 50% male and prevalence rates are assumed to be equal between men and women. However, there is evidence from the Hillingdon study and a study in Finland (Remes et al)27 that prevalence rates may differ substantially between men and women. In the Hillingdon study only 17% of the women referred had a diagnosis of heart failure confirmed by further investigation, whereas this figure was 41% for men. A possible extension to the model would be to divide the cohort into men and women with different prevalence rates and possible different mortality rates.

Survival probabilities

Treated heart failure – The baseline values for the cumulative survival probabilities for treated heart failure over the four-year period are taken from the treatment arm of the SOLVD trial, which assessed the effect of the ACE inhibitor enalapril on mortality in patients with mild to moderate heart failure.

Untreated heart failure – The baseline values for the cumulative survival probabilities for untreated heart failure come from the control arm of the SOLVD trial.

Cumulative survival probabilities for both treated and untreated heat failure were estimated from the SOLVD data using the actuarial method.307 Both sets of probabilities may be higher than those seen in clinical practice due to the strict protocols adhered to in a clinical trial.

In one-way sensitivity analysis as the mortality differential between treated and untreated heart failure increases, that is as the gains from treatment increase, the cost per LYG is reduced. However, the cost per life year gained is not very sensitive to changes in the mortality differential. Increasing the mortality gains from treatment by 10% (in each of the four years) reduces the cost per life year gained to £6,459.

Non-heart failure, but with clinical symptoms

The cumulative survival probabilities for patients who present with possible symptoms of heart failure but who do not receive a diagnosis of heart failure after further investigation were taken from the Hillingdon study (de Giuli, personal communication).

The mortality rate in the first 12 months is higher than for the untreated arm of the SOLVD trial,121 but it is lower in subsequent years. This may reflect the unrealistic context of the trial compared to the Hillingdon study that more closely reflected normal clinical practice.

Changes in the cumulative survival probabilities for this group have no effect on the cost per life year gained.

In this model, given that echo is assumed to be the gold standard with sensitivity and specificity of 100% no one will receive a confirmed diagnosis of heart failure who does not have the disease. Any abnormality in the first two tests will be referred for echocardiogram and this is assumed to give a definitive diagnosis. This is not a true reflection of the clinical situation and a possible extension to the model would be to reduce the sensitivity and specificity of echocardiography to below 100%. One implication of this is that some patients will have a diagnosis of heart failure confirmed when in fact they do not have the disease and this will result in them receiving inappropriate treatment.

Sensitivity of the tests

The sensitivity of a diagnostic test for heart failure refers to the proportion of patients correctly identified with heart failure by the test, ie the proportion of patients with the disease who also get a positive test result.

The specificity of a diagnostic test refers to the proportion of patients who do not have the disease who also get a negative test result.

The assumption employed here is that echocardiography is the diagnostic gold standard with both sensitivity and specificity of 100%.

For ECG and BNP there are a number of studies that estimate the sensitivity and specificity of the individual tests in patient groups similar to that considered here. However, there is little information on the combined diagnostic sensitivity and specificity of these two tests. A literature search for studies that reported sensitivity and specificity of ECG and/or BNP in diagnosing heart failure in a relevant patient group produced six papers in relation to BNP and seven for ECG; these are summarised in tables G4 and G5 (at the end of this appendix).

Table G4. Statistics for BNP in the diagnosis of heart failure.

Table G4

Statistics for BNP in the diagnosis of heart failure.

Table G5. Statistics for ECG in the diagnosis of heart failure.

Table G5

Statistics for ECG in the diagnosis of heart failure.

Given that in this model an abnormality on any test would mean referral for echocardiography, a key issue is the lowest estimate of specificity on any one test, as it is this that will determine the number of ‘unnecessary’ referrals for echocardiography. In fact the combined specificity will be lower than the lowest specificity of any one test given that not all tests will identify the same patients as having (or not having) the disease.

The baseline figure for sensitivity (0.97) is taken from the BNP study by Cowie et al.28 This sensitivity figure was found using a cut-off value for BNP level of 22.2 pmol/L chosen to give a negative predictive value of 98% in the study.

The baseline figure for specificity (0.61) is taken from a large study (n = 534) of patients sent by their GPs to an open access echocardiography service.306 The figure refers to the specificity of ECG in diagnosing heart failure, where a major abnormality on ECG was taken as a sign of left ventricular systolic dysfunction.**

In one-way sensitivity analysis increases in sensitivity and specificity of the three tests increase the cost per LYG (see below for more detail on key parameters).

Costs

Tests

The baseline cost for an echocardiogram (£53) is taken from the Finance Department at the Northern General Hospital, Sheffield. It is a full cost estimate including hospital overheads.

The baseline cost for ECG is taken from McMurray et al 285 and inflated to 2002 figures using the GDP market price index of inflation.

The costs for BNP testing were obtained from Martin Cowie (Clinical Adviser).

In one-way sensitivity analysis changes in the costs of BNP and ECG have no effect on the cost per LYG since these are included in both scenarios. As the cost of echocardiography increases the cost per LYG also increases and this relationship is linear. If the cost of an echocardiogram was only £25 the cost per life year gained would be £6,914. Doubling the cost of echocardiography to £106 results in a cost per life year gained of £32,989.

Drugs

In the baseline analysis it is assumed that ACE inhibitors form the only treatment available to patients with a confirmed diagnosis of heart failure that are not available to patients who do not have a confirmed diagnosis (see below).

The dosage and costs of treatment with ACE inhibitors are taken from the BNF (March 2002). The baseline cost estimate is £139 per year for the generic version of enalapril.

In one-way sensitivity analysis increases in the cost of treatment with ACE increase the cost per LYG. However, the cost per life year gained is not that responsive to changes in the cost of ACE inhibitors. Using the cheapest ACE (generic captopril at £52 per patient per year) gives a cost per life year gained of £14,318, whereas the most expensive ACE (fosinopril at £339) gives a cost per life year gained of £19,628.

Similarly if the additional cost of drug initiation and titration were included as well as the annual drug costs this would only result in small increases in the cost per life year gained.

Hospitalisation

In the baseline analysis the estimated hospitalisation rates for treated and untreated heart failure are taken from the SOLVD trial.121 This gives a hospitalisation rate for untreated heart failure of 0.65 admissions per patient per year with a reduction from ACE treatment of 18%. The average length of stay for a heart failure related admission (nine days) is taken from the NHS Reference Costs database, and the SOLVD trial suggests a reduction in LoS of 5% resulting from treatment with ACE. The average cost of an inpatient bed day (£198) is taken from the NHS Reference Costs database.

A simplifying assumption is made that hospitalisation rates and average length of stay are the same in each of the four years.

In one-way sensitivity analysis if the gains from ACE treatment in terms of hospitalisation are increased the cost per LYG is reduced, and these relationships are linear. Hence, the cost per life year gained is not very sensitive to changes in any of the hospitalisation parameters.

Life years gained

The assumption is that a confirmed diagnosis of heart failure means the patient has access to treatment that is not available to people who do not have a confirmed diagnosis despite presenting to the GP with possible symptoms of heart failure. In the baseline analysis it is assumed that ACE inhibitors are available to patients with a confirmed diagnosis of heart failure. A further assumption is that everyone who has a confirmed diagnosis is eligible for treatment with ACE inhibitors. Treatment with diuretics is assumed to be available to patients with clinical symptoms regardless of their diagnosis, therefore the cost of diuretic treatment is not included in the model. Estimates of the mortality gains from treatment with ACE inhibitors are taken from the treatment arm of the SOLVD trial.

The weight of evidence from the literature suggests that ACE inhibitors are a cost effective treatment for heart failure (see Appendix D). As explained above as the mortality differential between treated and untreated heart failure increases the cost per life year gained is reduced.

The assumption that ACE inhibitors are the only treatment available to patients with a confirmed diagnosis of heart failure that may have mortality gains and benefits in terms of reduced hospitalisation is not realistic. Clinical trials on the use of beta-blockers to treat heart failure have largely included people who are already being treated with ACE inhibitors. These trials have demonstrated an additional benefit from the use of beta-blockers over and above treatment with ACE inhibitors. A possible extension to the model would be the inclusion of the benefits of treatment with beta-blockers. This is likely to reduce the cost per LYG since, like ACE inhibitors, studies have shown beta-blockers to be a cost-effective treatment for heart failure. However, the results are not expected to be very sensitive to this modification.

Discounting

As the cost and benefits in this model accrue over four years they are discounted to current values using the rates recommended by NICE – 6% for costs and 1.5% for benefits. In one-way sensitivity analysis increases in either discount rate reduce the cost per LYG. As the time period is relatively short changes to the discount rate make little difference to the result.

Key parameters

There is a large amount of uncertainty around many of the parameters in this model but some are more important than others in terms of the sensitivity of the cost per life year gained to changes in these parameters.

The key parameters are discussed below.

Prevalence – Figure 2 shows the non-linear relationship between prevalence rate and cost per life year gained holding all the other parameters at their baseline values.

Figure 2. Prevalence.

Figure 2

Prevalence.

If the proportion of patients referred by the GP for echocardiography who actually obtain a diagnosis of heart failure falls below 20% the cost per life year gained increases very quickly. If the prevalence in this population was as low as 18% (as shown in the open access study by Davie et al),306 the estimate of the cost per LYG increases to just over £30,000.

Sensitivity and specificity – Figures 3 (below) and 4 (overleaf) show the relationship between the sensitivity and specificity of BNP and ECG and cost per LYG if all other parameters are held at their baseline values. It is important to note that the sensitivity plot assumes that specificity remains constant at 0.61, and the specificity plot assumes sensitivity remains constant at 0.97.

Figure 3. Sensitivity.

Figure 3

Sensitivity.

Figure 4. Specificity.

Figure 4

Specificity.

As the sensitivity of ECG and BNP increases to above 0.97 the cost per life year gained is increased rapidly.

The relationship is not as steep for specificity, but a specificity of larger than 0.75 results in a high cost per life year gained.

Other issues

The model assumes that the only value of echocardiography is in diagnosing heart failure and this is not realistic. Echocardiography can provide information on the underlying functional abnormality of the heart, and can therefore have a value in detecting other conditions that present with symptoms similar to heart failure. Echocardiography can also provide additional information that may affect the care pathway for heart failure.15 In addition, a more certain diagnosis can also provide an additional utility to the patient.

Where are the tests done and who interprets the results will both have implications for cost. This is particularly important in relation to echocardiography.*** Similarly the process of accessing echocardiography may also affect the cost and the appropriateness of doing the echo alongside the other tests or not – specialist clinic, cardiology outpatients, directly to echo from PC.

Footnotes

*

Following discussion at the CRG, chest X-ray was omitted from the list of initial tests.

**

The Davie et al study306 classifies ECG results as normal, minor abnormality or major abnormality. Minor abnormality is: atrial enlargement, bradycardia, tachycardia, broadening of QRS complex, poor R wave progression, right axis deviation, myocardial ischaemia, first degree atrioventricular block, non-specific ST-T wave changes. Major abnormality is: atrial fibrillation, previous MI, LV hypertrophy, bundle branch block, left axis deviation.

***

The structure of the model means that, in relation to BNP and ECG, this will not affect C/LYG, although the issues may still be important in terms of the appropriateness of doing the echo alongside the other tests.

Copyright © 2003, Royal College of Physicians of London.
Cover of Chronic Heart Failure
Chronic Heart Failure: National Clinical Guideline for Diagnosis and Management in Primary and Secondary Care.
NICE Clinical Guidelines, No. 5.
National Collaborating Centre for Chronic Conditions (UK).

Download

NICE (National Institute for Health and Care Excellence)

PubMed Health Blog...

read all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...