• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of bmjBMJ helping doctors make better decisionsSearch bmj.comLatest content
BMJ. May 18, 2002; 324(7347): 1210–1213.
PMCID: PMC1123166
Education and debate

Consent, confidentiality, and the threat to public health surveillance

Chris Verity, past chairman, British Paediatric Surveillance Unit Executive Committeea and Angus Nicoll, directorb

Effective protection of public health requires direction from the information provided by disease surveillance1—for example, in the cases of AIDS and variant Creutzfeldt-Jakob disease surveillance data led to action that protected health.2,3 Health surveillance relies entirely on prompt and accurate reporting of the occurrence of disease by doctors and other health professionals.1 Recently there has been increased concern in the United Kingdom about the need to maintain the confidentiality of information arising from consultations between doctor and patient. Documents have been issued regulating or advising on transfer of patient data. Some have argued that, unless needed for direct patient care, data should not be transferred to third parties without patients' explicit consent, or, alternatively, that all identifying information must first be removed.4 The difficulty in countering these arguments arises partly from the fact that health surveillance, including that for communicable diseases, has been neglected in official guidance on confidentiality.5 We and our colleagues are concerned that if some interpretations of this guidance were enforced they would impair or stop important surveillance activities and thus seriously prejudice public health.

Summary points

  • In the light of recent guidelines some doctors have been advised not to share health data that could potentially identify patients without either obtaining the patients'explicit consent or totally anonymising the data
  • These interpretations, if widely held and enforced, would compromise many surveillance activities essential for protection of the health of individuals and the public overall
  • The public should be made aware of the important ways in which information about individual patients is used to protect health
  • Those responsible for framing guidelines on the handling of clinical data and for advising doctors should consider issues related to health surveillance so that public health is not put at risk

Disease surveillance

Data derived from patient consultations, investigations, and care are passed on to third parties for a number of legitimate purposes, such as clinical audit, confidential enquiries, health service administration, and disease surveillance.6 A detailed description of the United Kingdom's many mechanisms for disease surveillance (see summary on bmj.com) is beyond the scope of this paper, but the following are five important points:

  • Health surveillance in the United Kingdom is currently working reasonably well and providing data that are invaluable for protecting individuals and the community
  • Surveillance differs from intrusive research in that it is primarily concerned with the collection and analysis of pre-existing data and usually does not involve contact with patients1,7
  • Surveillance data come from most types of patient consultation and therefore from many NHS staff; numerically public health staff play a minor though essential role (see box A on bmj.com)8
  • When personal information is transmitted for surveillance it should be held confidentially, under close medical supervision9
  • Reporting of infectious diseases by clinicians and laboratories is not usually supported financially nor is it at present specified in job descriptions. With one exception (notifications of infections) reporting is not protected by law. The reporting of laboratory results, HIV diagnoses, general practitioner activity, adverse drug and vaccine reactions, etc, is voluntary and relies on the goodwill of doctors, other health professionals, and patients.

The informality of UK infectious disease surveillance is a strength and a weakness. It provides a flexible and often powerful system for protecting health that is acknowledged internationally. Conversely it is vulnerable to changes in professional confidence and increasing work pressures on those who contribute. If doctors had serious doubts about the legitimacy of surveillance the easiest course of action would be for them to stop reporting, which would have harmful effects on the health of children and adults.

Recent guidance on handling of confidential patient data

Recent regulatory documents relating to confidentiality do not take health surveillance mechanisms into account and present a confusing picture for those contributing to surveillance. Guidance in 1996 from the Department of Health recommended that the NHS make patients aware by patient information leaflets and posters that, while confidentiality is protected, specific patient information is transferred around the NHS for important purposes.6,10

The 1998 Data Protection Act emphasised the need for consent by those from whom data originate. In Schedule 3, however, the Act allows that this does not apply where public health is being protected.10 In 1999 the NHS Executive circulated Protecting and Using Patient Information: A National Framework. This included information on implementation of the 1997 Caldicott Report. It gave guidance as to how the NHS should regulate individual data flows through the Caldicott guardian system, generally strengthen data security and confidentiality, and reduce transfers of personal data to the minimum necessary for specific purposes.11 Around the same time it was suggested that the common law duty of confidence rules against transfer of patient data to third parties without consent.12

In 2000 the passage of the Human Rights Act introduced another dimension.13 The General Medical Council followed earlier statements on confidentiality with guidance that transfer of data to third parties should either be with patient consent or with identifiers removed, and, specifically, that patient data could only be entered on to “registers” with consent.14 The Council's recommendations were open to different interpretations. In 2001 the government introduced into the Health and Social Care Act a facilitating clause that prescribes a mechanism for identifying specific data that may be transferred without consent, but this has been criticised both for giving too much power to central government and for being unworkably bureaucratic.15

How health is protected and promoted through surveillance

The table indicates categories and examples of protection and promotion of child health illustrating how surveillance data support health protection. To be effective these studies require relatively complete and minimally biased reporting. Information about individual patients is often needed for linkage to other data to enable validation and the elimination of duplicates, or to undertake essential analyses.4 Total anonymisation of reporting would make this impossible.

Why is surveillance important?

Since the 1980s the public has become increasingly concerned about health protection against real and perceived hazards, including HIV, bovine spongiform encephalopathy/variant Creutzfeldt-Jakob disease, food poisoning, possible adverse effects of medicines and vaccines, etc.16 People expect that health surveillance will be undertaken efficiently and effectively. When outbreaks of infectious disease occur, local public health doctors, regional epidemiologists, or, centrally, the Communicable Disease Surveillance Centre, mount rapid investigations to enable them to provide protective measures, identify hazards, and reduce the risk of further infections and disease.

Is surveillance acceptable?

When the rationale for surveillance is explained to colleagues in primary care and in hospitals, they are almost always cooperative, as are affected patients and healthy “controls,” sometimes providing personal information over the phone. Similarly, feedback from parents of children with rare but important disorders indicates that they support surveillance and would not welcome changes that threaten its completeness or accuracy. No major effort, however, has been made to explain surveillance mechanisms or their importance to the public.

Threats to health surveillance

The present arrangements for health surveillance are threatened by the proposal that either explicit consent should be sought from patients for use of their personal data, or data must be completely anonymised.

Obtaining explicit consent before sharing identifiable patient data

Simple but unrealistic suggestions have been made to solve the complex problems surrounding surveillance, consent, and confidentiality. One is that consent for reporting can readily be obtained from patients or parents. Almost all reporting and referral of clinical specimens relies on the cooperation of busy people such as clinicians and microbiologists who are providing patient care with growing workloads in an increasingly bureaucratic environment. For example, over 90% of paediatricians return the British Paediatric Surveillance Unit's monthly surveillance card, but the data the card requests are purposely kept to a minimum in order to sustain good response rates.17

It is our considered opinion, and that of our colleagues, that if explicit consent for sharing data had to be obtained the completeness and timeliness of reporting would be dangerously disrupted. Obtaining explicit consent would be most difficult for single consultations—for example for an acute infection—when the need for a report is often only appreciated some time after collection of the specimen or after initial diagnosis. It is usually impossible to determine at the time of a consultation which specimen will reveal a significant pathogen. For laboratory reporting, clinicians would have to ask for consent for sharing of data or specimens for every proposed investigation, or else pathologists would later be ringing up clinicians to ask them to trace and contact patients for consent. Neither system could be expected to work well.

When approached, families almost always wish to cooperate,4 but reporting doctors do not readily want to add the task of obtaining consent to their other work commitments,4,18 as a sequential surveillance and research investigation undertaken by the Royal Colleges of Ophthalmologists and Paediatrics and Child Health on retinopathy of prematurity illustrates. In this study, 235 cases were initially reported through conventional surveillance (without seeking explicit consent from parents). Later, reporting doctors were asked to obtain consent because an additional research study involved seeking parental views. In some cases repeated reminders to clinicians were needed although there were only three parental refusals. Eventually consent was obtained for 188 of 221 eligible cases (85%), and each consent took on average 3 months to obtain (L Haines, personal communication, 2001). The percentage would have been far lower had not the investigators been able, from prior knowledge of existing cases, to remind clinicians that consent was outstanding.

Other specialties have had similar experiences. Introduction of the requirement for consent for cancer registration resulted in a 70% drop in notifications to the long established Hamburg cancer register, destroying its comparative value—it is no longer referred to in European publications (M Parkin, International Agency for Research on Cancer, personal communication, 2001). An American study on consent found that the requirement for consent led to selective exclusion of some patients and hence introduced bias.19 In the United Kingdom, a belief that patient consent was needed before inclusion in a general practitioner diabetes register contributed to ascertainment of only 60% of eligible diabetic patients,20 mainly because some doctors never got round to obtaining consent (S Burnett, UCL London, personal communication, 1999). A disturbing recent development is that, notwithstanding official reassurances,9,10,12 some NHS trusts have instructed doctors not to transfer data about patients unless they do have consent, and this has inhibited some doctors who were keen to contribute to, for example, cancer registries. This has already impaired the work of the cancer registries, and the reporting of infectious diseases might be similarly affected.

Anonymising data before transfer

It has been suggested that removal of the identifiers from patient data will obviate the need for consent, but in a number of health surveillance studies the identifiers are essential links to other sources of health information about individuals that provide validation and eliminate duplication (table). An alternative suggestion is to use NHS numbers instead of such identifiers as names and dates of birth, but at present NHS numbers are rarely included in routine data sets, so this solution would also interfere with surveillance.

Conclusions

Health surveillance is essential to protect public health, and existing surveillance mechanisms work reasonably well. Surveillance could be seriously threatened if it was thought that there was an over-riding need to maintain patient confidentiality or always to have to seek explicit consent to sharing of data. We are very concerned that restrictive interpretations of some of the recent guidance on patient consent would so damage surveillance mechanisms that they would cease to protect the health of the public, thus resulting in preventable ill health and deaths.

Many documents that discuss patient confidentiality and consent take no account of the particular issues related to surveillance. Organisations that determine or influence health policy should consider these issues and support doctors and other health professionals in their important role as providers of patient data for surveillance. These organisations include the Royal Colleges, the General Medical Council, and the Department of Health.

Surveillance procedures should maintain confidentiality in line with arrangements elsewhere in the NHS.11 Our organisations have implemented Caldicott procedures and found them helpful in reducing patient identifiers and strengthening data security.15

The importance of surveillance is not recognised by the general public, primarily because it has not been explained to them. Since surveillance is fundamental to protecting the health of the public there should be formal public information and consultation, and the public should be made more aware that surveillance is in place, and that it is essential to the provision of health protection.

Table
How surveillance data support health protection: categories and examples derived from child health (further information appears on bmj.com)

Supplementary Material

[extra: Definitions and list]

Acknowledgments

We are grateful for comments on earlier drafts of this paper from Mike Catchpole, Michel Coleman, Barry Evans, and Phillip Mortimer.

Footnotes

Competing interests: None declared.

References

1. Langmuir AD. The surveillance of communicable diseases of national importance. N Engl J Med. 1963;268:182–192. [PubMed]
2. Berridge V. AIDS in the UK, the making of policy. 1981-94. Oxford: Oxford University Press; 1996.
3. Pattison J. BSE and the principles of public health. London: Nuffield Trust; 1999. . (Faculty of Public Health Medicine; Queen Elizabeth the Queen Mother Lecture.)
4. Al-Shahi R, Warlow C. Using patient-identifiable data for observational research and audit. BMJ. 2000;321:1031–1032. [PMC free article] [PubMed]
5. Singleton P, Hunnable J, Mason R. Gaining patient consent to disclosure—a consultancy project for the NHS Executive. London: Department of Health; 2001. www.doh.gov.uk/ipu/confiden/gpcd/ (accessed 14 March 2002).
6. Department of Health. The protection and use of patient information. Guidance from the Department of Health. London: Department of Health; 1996. www.doh.gov.uk/ipu/confiden/protect/pguide.htm . (HSG(96)5.) www.doh.gov.uk/ipu/confiden/protect/pguide.htm (accessed 14 Mar 2002). (accessed 14 Mar 2002).
7. On Department of Health. Ethical review of multicentre research in the NHS where there is no local researcher (Report of a Working Group—chair Dr Hugh Davies). www.doh.gov.uk/research/documents/ethicalreviewletter.pdf (accessed 16 April 2002).
8. Donaldson L. The report of the chief medical officer's project to strengthen the public health function. Leeds: Department of Health; 2001.
9. The Data Protection (processing of sensitive personal data) Order. London: Stationery Office; 2000. www.hmso.gov.uk/si/si2000/20000417.htm (accessed 14 March 2002).
10. France E. Research and privacy [letter]. Times 2001 May 22.
11. NHS Executive. Caldicott guardians. Health Service Circular 1999 January. (HSC 1999/012.)
12. Falconer C. Privacy law and medical research [letter]. Times 2001 May 17.
13. Hewson B. Why the Human Rights Act matters to doctors. BMJ. 2000;321:780–781. [PMC free article] [PubMed]
14. General Medical Council. Confidentiality—protecting and providing information. London: GMC; 2000.
15. Anderson R. Undermining data privacy in health information. BMJ. 2001;322:442–443. [PMC free article] [PubMed]
16. Regan M. Health protection in the next millennium from tactics to strategy. J Epidemiol Community Health. 1999;53:517–518. [PMC free article] [PubMed]
17. Nicoll A, Lynn R, Rahi J, Verity J, Haines L. Public health outputs from the British Paediatric Surveillance Unit and similar clinician-based systems. J R Soc Med. 2000;93:580–585. [PMC free article] [PubMed]
18. Paterson ICM. Consent to cancer registration—an unnecessary burden. BMJ. 2001;322:1130. [PMC free article] [PubMed]
19. Jacobsen SJ, Xia Z, Campion ME, Darby CH, Plevak MF, Seltman KD, et al. Potential effect of authorization bias on medical record research. Mayo Clin Proc. 1999;74:330–338. [PubMed]
20. Burnett SD, Woof CM, Yudkin J. Developing a district diabetic register. BMJ. 1992;305:627–630. [PMC free article] [PubMed]
May 18, 2002; 324(7347): 1210–1213.

Commentary: Don't waive consent lightly—involve the public

Donal Manning, consultant paediatrician

Verity and Nicoll argue that the benefits of public health surveillance using personal medical data outweigh harm to patients from disclosure without consent. Proponents of a care ethic based on rights or duty would reject this consequentialist argument and claim that bypassing consent and breaching confidentiality are in themselves harmful, regardless of the calculations of potential risk and benefit. Tension between these viewpoints has been heightened by the perception that doctors, albeit with good intent, flouted consent in the organ retention affair, and by concern that advances in information technology further threaten patient confidentiality. Although currently emphasis is on promoting patients' autonomy, there are both legal and ethical limits to the scope of this protection. Confidentiality may be waived if preserving it would pose a serious public health risk, such as in notification of infectious diseases. The onus is on doctors to persuade legislators and society that public health surveillance is morally equivalent to the other circumstances in which consent and confidentiality can be waived. Given current public suspicion of centrally derived guidance, for example on the measles, mumps, and rubella immunisation, this will be no easy task.

The response of the health service to this challenge has so far been disappointing. As the authors point out, the Department of Health's advice that the NHS should provide written, accessible information to patients outlining the importance of research and audit using medical records has been implemented erratically and inconsistently. Yet public response might be more positive than anticipated. Melton reported that the Mayo Clinic, faced with similar challenges to medical records research from new Minnesota State legislation, asked outpatients for broad authorisation for such research in advance,1-1 and 96% of respondents agreed, a proportion that would be amply representative for most epidemiological studies.

Waiving consent should be contemplated only if it cannot reasonably be sought, as for very large or retrospective studies, or (rarely) when seeking consent might be distressing or harmful. Mere inconvenience is insufficient justification, and the public will not accept waiving consent because doctors cannot “get round” to seeking it. A waiver, ideally, should be linked with broad authorisation (not, strictly speaking, consent) for controlled access to records. This, if sanctioned by legislation and the public, would be appropriate for large scale public health surveys. Engaging the public in this way might even be held to constitute implied consent to disclosure. Seeking consent might still be appropriate for smaller studies, such as of rare childhood diseases. Verity and Nicoll's experience in this area is that the vast majority of parents would support access to records, and seeking consent should not be too onerous for individual doctors.

Doctors must give the consent process reasonable priority among their other commitments before making a convincing case to the public that for some important studies it may be impossible to attain. Since most of the recommendations of the Kennedy report of the inquiry into children's heart surgery at the Bristol Royal Infirmary have been accepted by the government, doctors can anticipate that the standard of consent sought for clinical activities will be raised to match the current standard for participation in research, not vice versa.

Footnotes

 Competing interests: None declared.

Definitions and a list of UK surveillance systems appear on bmj.com

References

1-1. Melton LJ. The threat to medical-records research. N Engl J Med. 1997;337:1466–1469. [PubMed]

Articles from BMJ : British Medical Journal are provided here courtesy of BMJ Group

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...