• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of bmjBMJ helping doctors make better decisionsSearchLatest content
BMJ. Nov 24, 2007; 335(7629): 1075–1076.
PMCID: PMC2094194
Measuring Quality through Performance

Measuring performance and missing the point?

Iona Heath, general practitioner,1 Julia Hippisley-Cox, professor of clinical epidemiology and general practice,2 and Liam Smeeth, professor of clinical epidemiology3

Targets do not necessarily translate into improvements for service users. Iona Heath, Julia Hippisley-Cox, and Liam Smeeth argue that performance measurement in the UK is shifting focus from what each patient needs and those who need it most

Since April 2004, the performance of general practitioners has been measured and remunerated against a limited, although increasing, number of easily measurable clinical activities. This initiative is unique worldwide and is attracting considerable international interest. It therefore behoves us to think critically about what we are doing.

General practitioners have responded by systematically recording the listed activities, apparently performing well above expectation, and the whole exercise has been hailed as a success for quality of care. Despite evidence that these sorts of incentives improve the quality of documentation while having a much more limited effect on underlying standards of care,1 there have undoubtedly been useful achievements. Of these, probably the most substantial are improvements in diabetic control and innovations in computer prompting systems. However, the system is in danger of missing the point of both quality and general practice.

Treatment by numbers

The clinical activities that are measured and rewarded by the quality and outcomes framework are largely evidence based. Nevertheless, almost all interventions cause some harm, and even when effective treatments are applied to a series of patients in clinical practice some will be harmed (although more will benefit). The risks of harm tend to increase with age, as does the potential for benefit. The stakes therefore become higher as the evidence becomes more tenuous because many trials focus on younger patients.

Evidence based care was never meant to be a substitute for clinical judgment but, combined with the inducements of the quality and outcomes framework, it becomes so. Mechanistic blanket management strategies, embedded into computer software, become fixed and static with the danger that innovation will be stifled. Interventions become routine, and practitioners are no longer required to grapple with the innate uncertainty of each different clinical situation. Most randomised trials systematically exclude patients' symptoms, functional status, comorbidity, severity of illness, ideas, and preferences. Yet these are the factors which should fundamentally affect decisions about appropriate treatment.2 Within large study populations, there will be smaller populations sharing different characteristics whose response to a given treatment will differ from that of the larger group. Such groups could be systematically harmed by the intervention, and there are currently no robust systems in place to measure or monitor this.3

The quality and outcomes framework diminishes the responsibility of doctors to think, to the potential detriment of patients, and encourages a focus on points scored, threshold met, and income generated. To give just one example, the failure to make any allowance for age means that doctors are encouraged to overtreat hypertension in old people4 with the danger of causing fainting, falls, and fractures.

The whole initiative is based on reductive linear reasoning that views the body as a machine and assumes that a standardised treatment will produce an equally standard unit of beneficial outcome. However, any practising clinician knows that the same treatment applied to two people with the same diagnosis can produce very different outcomes. Complexity theory suggests that the body is more usefully regarded as a complex adaptive system, characterised by rich interactions between multiple components that produce unpredictable outcomes. This analogy makes much more sense of clinical experience. Psychological states and social contexts exert measurable effects on the functioning of the body. Standardised treatments ignore all of this.

Best way to reach goals?

The quality and outcomes framework necessarily concentrates on clinical activities that are easily measured. Clearly, one of the easiest is the issuing of prescriptions. This leads to a situation where epidemiological research, with its tendency to extrapolate from short term studies over the whole of the remaining life span, combined with computer driven surveillance of whole populations, is driving hugely increased prescribing of some drugs. Almost all of this is designed to prevent future events rather than alleviate present suffering. Prevention has its merits, but was this an intention of the framework and what are the opportunity costs for other healthcare interventions?

One of the aims of the framework is to tackle health inequalities, but it has the potential to work in the opposite direction. Most fundamentally, it encourages the illusion that health inequalities can be solved by the health service and allows policy makers to ignore the extent to which health inequalities are a symptom of socioeconomic inequalities that continue to widen. The framework situates health inequalities at the level of the individual and ignores the political, social, and cultural context within which people find themselves. It is based on the astonishing assumption that everyone wants to live as long as possible whereas, in reality, some people seek to end their lives prematurely. Others adopt more chronically self harming behaviours in the full awareness that they are likely to shorten their lives, clearly prioritising coping in the immediate future above the uncertainties of the long term.

As there is a socioeconomic gradient for the prevalence of almost every disease, the poor are much more likely to experience comorbidity. The most marginalised people, who have a combination of physical and mental illnesses often compounded by drug or alcohol dependence, are difficult to engage in health care and even more difficult to coerce into the framework's unitary care pathways. Patients living in adverse social circumstances are also more likely to be taking maximal tolerated therapy without achieving the desired outcome. Not surprisingly, such people are much more likely to be reported as exceptions and, once given that status, are at risk of receiving proportionately less attention.5

Finally, as people living in deprived areas are sicker, more therapeutic effort will be needed for doctors to reach fixed targets. This means that doctors working in these areas have to work harder to achieve the same remuneration.6 The increased payments for higher disease prevalence do not take severity and complexity into account. Working in poorer areas becomes less desirable, further reducing quality of care or even making care difficult to find (figure(figure).). Furthermore, there is evidence that payment for performance systems reward already high achievers and penalise low achievers and so exacerbate inequalities.7

figure heai456210.f1
Policy of inverse care in quality and outcomes framework

What's not measured

Three quarters of the population do not have any of the diseases included in the quality and outcomes framework,8 and ever since its introduction, special interest groups have been lining up to have their particular priority included among the targets. However, worthy attempts to include depression have only made matters worse as the imposition of standardised questionnaires and scoring systems serves simply to reify and medicalise distress and unhappiness. Authentic dialogue between doctor and patient is disrupted and many doctors feel fundamentally compromised.9

None of the framework measures estimate clinically important outcomes. What they assess is treatment processes that are supposed to lead to improved outcomes. A marked discrepancy exists between the likely effect on health and the level of monetary reward, and there seems to have been no attempt to align the two.10 Until the undoubted and now well documented increase in process is translated into tangible outcomes such as diabetes complication rates, renal failure in hypertension, or incidence of myocardial infarction or smoking related deaths, the benefits and cost effectiveness of the exercise cannot be estimated. Outcomes are much more difficult to measure than processes, especially at the level of individual practices, but the heightened emphasis on process brought about by the framework should not be allowed to distract from the fundamental aims of medical care.

Summary points

  • Measurable differences do not necessarily translate into meaningful differences in patients' lives
  • Clinical care needs to be tailored to individual patients rather than using a mechanistic approach
  • The quality and outcomes framework needs to include clinically important outcome measures
  • It should also include mechanisms to measure and monitor potential harms

Notes

This is the fourth article in a series looking at use of performance indicators in the UK and elsewhere.

This series is edited by Azeem Majeed, professor of primary care, Imperial College London(a.majeed@imperial.ac.uk),and Helen Lester,professor of primary care, University of Manchester(helen.lester@manchester.ac.uk).

Notes

Competing interests: All the authors work as NHS general practitioners and are therefore subject to the quality and outcomes framework. JH-C has been commissioned by the Department of Health, National Audit Office, and Disability Rights Commission and the Information Centre to do analysis to inform the development of the quality and outcomes framework. This article reflects her views rather than the views of the commissioning bodies.

Contributors and sources: LS is supported by a Wellcome Trust senior fellowship in clinical science. The core ideas in this article arose from discussions between all three authors. IH wrote the first draft and all authors contributed to serial drafts and agreed the final submission. IH is guarantor.

Provenance and peer review: Commissioned; externally peer reviewed.

References

1. Fairbrother G, Hanson KL, Friedman S, Butts GC. The impact of physician bonuses, enhanced fees, and feedback on childhood immunization coverage rates. Am J Public Health 1999;89:171-5. [PMC free article] [PubMed]
2. Feinstein AR. The problem of cogent subgroups: a clinicostatistical tragedy. J Clin Epidemiol 1998;51:297-9. [PubMed]
3. Goodwin JS. Embracing complexity: a consideration of hypertension in the very old. J Gerontol A Biol Sci Med Sci 2003;58:653-8. [PubMed]
4. Rastas S, Pirttilä T, Viramo P, Verkkoniemi A, Halonen P, Juva K, et al. Association between blood pressure and survival over 9 years in a general population aged 85 and older. J Am Geriatr Soc 2006;54:912-8. [PubMed]
5. McLean G, Sutton M, Guthrie B. Deprivation and quality of primary care services: evidence for persistence of the inverse care law from the UK quality and outcomes framework. J Epidemiol Community Health 2006;60:917-22. [PMC free article] [PubMed]
6. Hippisley-Cox J, O'Hanlon S, Coupland C. Association of deprivation, ethnicity, and sex with quality indicators for diabetes: population based survey of 53 000 patients in primary care. BMJ 2004;329:1267-9. [PMC free article] [PubMed]
7. Rosenthal MB, Frank RG, Li Z, Epstein AM. Early experience with pay-for-performance: from concept to practice. JAMA 2005;294:1788-93. [PubMed]
8. Qresearch. Comorbidity of diseases in the new general medical services contract for general practitioners www.qresearch.org/Public_Documents/DataValidation/Co-morbidity%20of%20diseases%20in%20the%20new%20GMS%20contract%20for%20GPs.pdf
9. Jeffries D. Ever been HAD? Br J Gen Pract 2006;56:392
10. Fleetcroft R, Cookson R. Do the incentive payments in the new NHS contract for primary care reflect likely population health gains? J Health Serv Res Policy 2006;11:27-31. [PubMed]

Articles from BMJ : British Medical Journal are provided here courtesy of BMJ Group
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...