Home > Executive summary - A Systematic Review...

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.

Beswick AD, Brindle P, Fahey T, et al. A Systematic Review of Risk Scoring Methods and Clinical Decision Aids Used in the Primary Prevention of Coronary Heart Disease (Supplement) [Internet]. London: Royal College of General Practitioners (UK); 2008 May. (NICE Clinical Guidelines, No. 67S.)

  • This publication is provided for historical reference only and the information may be out of date.

This publication is provided for historical reference only and the information may be out of date.

Executive summary

Background

Guidelines recommend that cardiovascular disease risk assessment in the form of risk scoring is used to identify patients who are asymptomatic, but at high risk of suffering from cardiovascular disease in the future. Risk scoring methods enable clinicians to combine patient risk factor information and predict the risk of a cardiovascular event within a specified time period. This theoretically allows the targeting of health service resources to those most likely to benefit from preventive treatment and to avoid possible adverse effects, and cost, of unnecessary treatment in those at low risk.

Numerous complex prediction models and simplified risk scoring methods are available for clinicians to use in primary care. The accuracy of risk-scoring methods in different populations is thought to vary and no review of their effectiveness in improving health-related outcomes has been reported.

Objectives

Systematic review to ascertain:

  1. What risk scoring methods designed to identify asymptomatic people at high risk of cardiovascular disease are available for use in primary care?
  2. How does the accuracy of charts, tables and simplified methods derived from prediction models compare with the original complete models?
  3. How accurate are cardiovascular risk scoring methods in different populations?
  4. How effective are cardiovascular risk scoring methods in improving health-related outcomes?

Methods

A series of related systematic reviews was conducted using standard Cochrane Collaboration methods:

  1. Systematic review of risk scoring methods designed for use in primary care with the purpose of assessing an individual’s future risk of cardiovascular disease to aid the targeting of lifestyle or pharmacological intervention.
  2. Systematic review of the accuracy of simplified risk scoring methods such as charts and tables, compared with full prediction models (convergent validation – the comparison of a risk score or model in a single sample, with other similar models or with a “gold standard”).
  3. Systematic review of the accuracy of risk scoring methods in different populations (external validation). External validity is how well a risk score or model predicts events observed in a different population and its two main properties are calibration and discrimination. If the predicted risk equals the observed, the model is well calibrated, and if it successfully categorises individuals into those who will or will not get the disease, it is said to discriminate well.
  4. Systematic review of the effectiveness of using a cardiovascular risk scoring method by a medical practitioner or primary healthcare professional in targeting primary prevention on the basis of calculated risk.

Results

A comprehensive search strategy identified 3,439 articles of which 996 related to the use of risk scoring methods in the primary prevention of cardiovascular disease.

1. Risk scoring methods

  • One hundred and ten cardiovascular risk scoring methods were identified of which 70 were clearly intended for application in primary prevention.
  • A core set of variables was included in a large proportion of methods reflecting the contemporary established independent risk factors (i.e. age, sex, blood pressure, total blood cholesterol and cholesterol fractions).
  • Several widely recommended methods do not include family history of cardiovascular diseases.
  • Some methods incorporated an adjustment for use in geographic regions with different observed cardiovascular disease risk.

2. Convergent validation

  • In 16 studies reporting 40 comparisons, simplified methods derived from the Framingham-Anderson prediction model were compared with the full risk equation.
  • Risk scoring charts and tables that substitute HDL-cholesterol measurement for average values do not accurately reflect risk as estimated using the full prediction model.
  • Over-simplification of methods influenced sensitivity and specificity unfavourably.

3. External validation

  • External validation of Framingham-based methods was reported for 112 different population groups.
  • Framingham risk scoring methods tended to over-estimate coronary heart disease risk in groups with low observed risk. This was evident in a wide range of populations studied using longitudinal comparisons of predicted and observed risk, and in cross-sectional studies comparing estimated risk with appropriate population statistics.
  • Framingham risk scoring methods tend to under-predict risk in high-risk groups (diabetics, patients with a strong family history of premature cardiovascular disease, people from areas with a high incidence of disease and in socio-economically deprived groups).
  • There was a tendency for better discrimination, as measured by the area under the receiver operating characteristic curve, to be associated with poorer calibration as represented by the predicted: observed ratio.
  • There were no consistent sex differences in the calibration of the Framingham risk scores, however discrimination was better in women than in men.
  • There was a tendency for increased over-estimation in later compared with earlier populations.
  • Limited data suggest that other non-Framingham based risk scores such as PROCAM and the Dundee score show similar inaccuracies when applied in different populations.

4. Effectiveness

  • Four randomised controlled trials were identified that examined the effectiveness of risk scoring methods in improving health outcomes. One trial studied two interventions. Interventions were based on the Framingham-Anderson and Westlund risk scoring methods.
  • Groups included were diabetic or hypertensive patients. There were no trials involving intermediate risk or disease-free individuals for which the risk scoring methods were intended.
  • Interventions studied in trials were: incorporation of risk score in medical notes, informing the physician of risk score, use of risk chart by physician, and use of clinical decision support system incorporating risk score with or without a chart.
  • Outcomes were: cardiovascular risk, blood pressure, serum cholesterol or use of lipid-lowering or antihypertensive treatment. Studies were not large enough to assess benefit in relation to cardiovascular disease events.
  • Interventions showed no improvement in predicted absolute cardiovascular risk or in declared primary outcomes.
  • One study in hypertensive patients showed a small reduction in systolic blood pressure associated with use of a risk chart but not when used in conjunction with a computer based clinical decision support system. In a further study in diabetics, incorporation of a risk score in the medical notes was associated with increased prescription of lipid-lowering or antihypertensive treatment in high-risk patients.
  • Other interventions including those associated with use of clinical decision support systems provided no evidence of benefit.
  • One study noted very low uptake of risk-scoring methods by clinicians.

Conclusions

Implications for health care

  • Cardiovascular risk scoring methods are widely recommended but there is little evidence that they are effective in improving health outcomes.
  • In broadly representative populations, established Framingham-based methods offer reasonable discrimination between high and low risk individuals but tend to over-predict the absolute risk of cardiovascular disease.
  • Framingham-based methods under-estimate risk in diabetics and patients with a strong family history of premature cardiovascular disease.
  • Recent charts and tables are identified that compare well with the Framingham equation and may have value in explaining the consequences of increased risk and treatment options to the patient. Charts and tables are more accurate when they include all the risk factors present in the full prediction model.
  • Methods are available that permit adjustment for regional cardiovascular risk. However studies are required to assess external validity in their defined target populations.
  • The evidence supporting the general screening of asymptomatic individuals using current risk scoring methods, to identify high-risk individuals for preventive treatment is currently weak.

Recommendations for research and development

  • Risk assessment methods should be subject to quantitative assessment of effectiveness in relation to health outcomes and cost.
  • Qualitative evaluations of methodological characteristics that make a risk scoring method a practical clinical tool that benefits the doctor and the patient are required.
  • Validation studies of risk scoring methods that permit adjustment for between-population differences in cardiovascular risk are required to assess their validity in their defined target populations.
  • New prediction models should have multiple external validations in diverse populations with differing age ranges, ethnicity, sex and cardiovascular risk.
  • New prediction models should be developed and tested in populations other than established cohort studies. Use of the improving quality and quantity of routinely collected general practice data may facilitate the development of new risk scoring methods.
  • The advantages of inclusion of all major established risk factors and the balance between increased accuracy, against the additional workload and expense of including more complex risk factor information requires investigation.
  • A systematic review of the value of including additional and emerging risk factors in risk prediction models is needed, before they become included by default.
  • The impact of co-existing preventive treatment such as antihypertensive medication, on the accuracy of risk prediction should be explored. Analysis of more recent population studies to allow for widespread use of effective treatments is appropriate.
  • Consensus is required on the best outcome to use for a risk score (e.g. cardiovascular or coronary heart disease, “hard” or “soft” endpoints). The same outcomes that are used in trials of preventive treatment would be logical choices for consideration in primary care, as opposed to outcomes dictated by pragmatic choices from cohort studies.
Copyright © 2008, Royal College of General Practitioners.
Cover of A Systematic Review of Risk Scoring Methods and Clinical Decision Aids Used in the Primary Prevention of Coronary Heart Disease (Supplement)
A Systematic Review of Risk Scoring Methods and Clinical Decision Aids Used in the Primary Prevention of Coronary Heart Disease (Supplement) [Internet].
NICE Clinical Guidelines, No. 67S.
Beswick AD, Brindle P, Fahey T, et al.

Download

NICE (National Institute for Health and Care Excellence)

PubMed Health Blog...

read all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...