A Study of Adverse Occurrences and Major Functional Impairment Following Surgery

Introduction

Studies have shown that 40–70 percent of adverse events are associated with surgery,^1–3 and that the risk of major disability (moderate or permanent impairment) resulting from such events varies from 10–25 percent.^4–7 These studies, however, did not include a comparison group, leaving open the question of whether the primary risk factor for patients undergoing similar procedures was their own health status at the time of surgery, the procedure itself, or the adverse event. The few studies that have used comparison groups have focused on death as the outcome and found that adverse incidents increased the risk of dying. Rosen and colleagues⁸ found, in a random sample of Medicare discharges from seven States, that individuals who had adverse incidents related to coronary artery bypass grafting had a mortality rate roughly five times greater than similar patients who did not have an adverse event. Garcia-Martin and coauthors⁹ discerned, from their study of 524 consecutive deaths at a tertiary care hospital among surgical patients, that the adjusted, attributable risk of death was 56 percent for those patients who had an adverse occurrence.

In this report, we examined whether an adverse occurrence places patients at increased risk for functional status impairment and increases their hospital lengths of stay. We based our study upon the New York Patient Occurrence and Reporting System (NYPORTS) definition of adverse occurrence: an “unintended adverse and undesirable development in an individual patient's condition, such as a patient death or impairments of bodily functions, in circumstances other than those related to the natural course of illness, disease, or proper treatment in accordance with generally accepted medical standards.”¹⁰

Methods

Figure 1. Study flow design

   Figure 1. Study flow design

Figure 1

Sampling strategy

We identified adverse occurrences using the New York Patient Occurrence and Reporting System (NYPORTS),¹⁰ a statewide mandatory reporting system that started in 1998. In 2001, a total of 28,689 adverse occurrence reports were submitted to NYPORTS electronically, a rate of 1,159 reports per 100,000 discharges. Not all incidents were the result of medical treatment. For example, NYPORTS includes all deaths within 2 days of a surgical procedure, even if the death was anticipated and a direct consequence of the disease process. We selected a smaller subset (25 out of 54) of NYPORTS codes representing adverse surgical occurrences for an elderly population, such as pulmonary embolism or wrong site surgery.

We included only those surgical procedures with a prevalence of 1 percent or more in the New York State (NYS) Medicare population and at least 1 percent prevalence in the NYPORTS database for the same time period, and that were consistent with surgical procedures listed in the literature as being associated with adverse events.⁵ We grouped these procedures using the Clinical Classification Software (CCS) developed by the Agency for Healthcare Research and Quality (AHRQ).¹³ The resulting procedure groups were aortic resection, appendectomy, arthroplasty of the knee, coronary artery bypass graft (CABG), carotid endarterectomy, cholecystectomy, colorectal resection, hip replacement, hysterectomy, laminectomy, open prostatectomy, peripheral vessel bypass, hip fracture repair, spinal fusion, and transurethral resection of the prostate (TURP).

Table 1

Sample characteristics (N = 1545)

Table 1

Sample characteristics (N = 1545)

Characteristics	No. or value	% or SD*
Male	731	47.3
Age mean (SD)	75	(10)
ASA class mean (SD)	2.8	(.65)
No adverse occurrence	973	63.0
Impaired on admission	334	21.6
Impaired at discharge	511	33.1
Surgical procedures
Aortic resection	50	3.2
Appendectomy	36	2.3
Arthroplasty of the knee	75	4.9
CABG	47	3.0
Cholecystectomy	156	10.1
Colorectal resection	296	19.2
Carotid endarterectomy	103	6.7
Hip fracture repair	155	10.0
Hip replacement	172	11.1
Hysterectomy	87	5.6
Laminectomy	76	4.9
Open prostatectomy	48	3.1
Peripheral vessel bypass	123	8.0
Spinal fusion	42	2.7
TURP	79	5.1

*

SD = standard deviation

Table 2

Distribution of adverse occurrences in full (more...)

Table 2

Distribution of adverse occurrences in full sample (N = 1545)

Adverse occurrences	No.	%
Unplanned operation or reoperation related to the primary procedure	270	17.5
Hemorrhage or hematoma requiring drainage, evacuation, or other procedural intervention	154	10.0
Procedure-related injury requiring repair, removal of organ, or other procedural intervention	101	6.5
Acute MI unrelated to cardiac procedure	78	5.1
Loss or Impairment of bodily function	54	3.5
Post-op wound infection following clean or clean/contaminated care requiring drainage	49	3.2
Displacement, migration, or breakage of an implant, device, graft, or drain, whether repaired intentionally, left in place, or removed	47	3.0
Any new central neurological deficit	46	3.0
New documented deep vein thrombosis	44	2.9
New, acute pulmonary embolism	31	2.0
Volume overload leading to pulmonary edema	27	1.8
Wound dehiscence requiring repair	24	1.6
Thrombosis distal bypass graft requiring repair	21	1.4
Unplanned laparoscopic conversions to an open procedure because of an injury and/or bleeding during the procedure	20	1.3
Cardiac and/or respiratory arrest requiring basic life support/ advanced cardiac life support intervention	18	1.2
Impairment of limb	15	1.0
Anastomotic leakage requiring repair	14	0.9
Cardiac arrest with successful resuscitation	12	0.8
Any new peripheral neurological deficit	11	0.7
Unintentionally retained foreign body due to inaccurate surgical count or break in procedural technique	9	0.6
Loss of limb or organ	8	0.5
Aspiration pneumonia related to conscious sedation	3	0.2
Malfunction of equipment during treatment or diagnosis or a defective product that resulted in death or serious injury	2	0.1
Incorrect procedure or invasive treatment	1	0.1
Errors of omission resulting in death or serious injury relating to the patient's underlying condition	1	0.1

Note: The sum of adverse occurrences does not equal the overall adverse occurrence variable, which is binary (any versus none), as some patients had more than one occurrence.

Most investigations on adverse occurrences have employed a two-stage selection process of initial nurse screening followed by physician classification of adverse events, with analysis of only the adverse event cases. Instead, for the original project we chose a case-control design because the incidence of adverse events is low and would require a large and lengthy study to obtain adequate statistical power. By selecting cases and matched controls, we were able to develop and test the screening tool while comparing the exposure to an adverse occurrence. Cases were randomly selected from the eligible pool of Medicare beneficiaries who had 1 of the 15 surgical procedures described previously and had been reported by hospitals to NYPORTS (see Table 1, in “Results”) with 1 of the 25 adverse occurrence codes (see Table 2 in “Results”). The control group consisted of patients not reported to NYPORTS. These Medicare patients were matched to each NYPORTS case based on surgical procedure group, date of discharge (within 30 days), and hospital. Where more than one control existed we selected as the control the patient closest to the discharge date for the NYPORTS case.

Data collection

We constructed the abstraction tool to collect the NYPORTS categories, demographic data, LOS, admission and discharge Modified Rankin scores,⁴ and the American Society of Anesthesiologists Physical Status measure (ASA) recorded by the case anesthesiologist.¹⁴ We initially tested the abstraction instrument using 75 charts with known adverse occurrences reported in the NYPORTS database. Each case was abstracted by two of the authors, who were unaware of the initial NYPORTS case status, and the two reviewers discussed discrepancies. The abstraction instrument was modified on the basis of lessons learned in the pilot abstraction.

Thirteen nurses then were trained using the modified abstraction tool, in part using materials developed by the NYS Department of Health for NYPORTS coding. Nurses were trained to identify whether incidents occurred—not whether the occurrences were due to medical error. Using the modified instrument, nurse-abstractors could assign anywhere from 1 to 28 NYPORTS codes for each patient record (26 for nonfatal and 2 for fatal occurrences). Prior to abstraction, the nurses first had to pass “gold standard” and interrater reliability testing. Gold standard testing on major analytic variables required agreement of at least 95 percent in five test cases. Interrater reliability testing was done on 15 additional charts. A total of 49 data elements were tested. Proportion of agreement ranged from 95 to 100 percent with kappa coefficients ranging from 0.70 to 1.0.¹⁵ Some elements could not be evaluated, due to the lack of variability in the selected review cases (i.e., the events did not occur in patient records that were not randomly selected for interrater reliability testing). These elements were wound dehiscence requiring repair, displacement/migration/ breakage of implant, thrombosis requiring repair, postoperative wound infection, wrong patient/site, incorrect procedure or invasive treatment, errors of omission, and malfunction of equipment.

We requested 2,810 Medicare medical records from 180 NYS hospitals for the time period of 1998-2001. We received 2,727 records (97 percent). Twenty-six cases were voided because they did not meet inclusion criteria (22); key-question data were not available (1); wrong admission (2); or missing documentation (1). A total of 2,701 records were fully abstracted.

Results

Figure 2. Relationship of adverse occurrences and (more...)

   Figure 2. Relationship of adverse occurrences and change in impairment

Table 3

Comparisons of patients impaired at discharge with (more...)

Table 3

Comparisons of patients impaired at discharge with an adverse occurrence and patients who were NOT impaired on admission

	Total not impaired on admit		No adverse occurrence			Adverse occurrence			Odds of impairment at discharge with adverse occurrence
Surgical procedure	No.		No.	%		No.	%		OR	95% CI
Overall	1211		92	12%		115	26%		2.6	(1.9, 3.5)
Aortic resection	47		2	8%		8	36%		6.6	(1.2, 35.5)
Appendectomy	34		0	0%		1	7%		-	- -
Arthroplasty of the knee	63		17	38%		6	33%		0.8	(0.3, 2.6)
CABG	45		4	13%		6	38%		8.1	(1.4, 46.9)
Cholecystectomy	142		4	5%		7	12%		2.8	(0.8, 10.2)
Colorectal resection	276		16	9%		25	25%		3.2	(1.6, 6.4)
Carotid endarterectomy	101		0	0%		9	20%		-	- -
Hip fracture repair	31		16	57%		1	33%		0.4	(0.0, 4.6)
Hip replacement	75		13	32%		17	50%		2.1	(0.8, 5.5)
Hysterectomy	83		1	2%		2	10%		6.4	(0.6, 74.8)
Laminectomy	60		7	15%		4	29%		2.2	(0.5, 9.1)
Open prostatectomy	48		0	0%		1	5%		-	- -
Peripheral vessel bypass	99		8	13%		19	53%		7.7	(2.9, 20.7)
Spinal fusion	34		4	19%		6	46%		3.6	(0.78, 17.0)
TURP	73		2	4%		3	12%		2.9	(0.5, 18.8)

OR = odds ratio

CI = confidence interval

- denotes that risk estimates could not be computed

Figure 2

Using multivariate logistic regression, we modeled impairment at discharge—adjusting for admitting impairment, age, gender, ASA class—and found a two-fold increase in the log odds of impairment for those having an adverse occurrence (OR = 2.3, 95% CI = 1.7–3.1). We further examined the adjusted risk for the 5 surgical procedures for which there were 100 or more cases with complete data, and found that colorectal resection (OR = 2.9, 95% CI = 1.5–5.7), peripheral vessel bypass (OR = 6.3, 95% CI = 2.4–16.2) resulted in impairment following an adverse occurrence. The other surgical procedures examined (cholecystectomy, hip fracture repair, and hip replacement) did not show an increase in risk after adjustment.

Table 4

Median lengths of stay, in days (N = 1,545)

Table 4

Median lengths of stay, in days (N = 1,545)

Surgical procedure	No adverse occurrence	Adverse occurrence	Chi-square	P value
Overall	6 (5)	11 (14)	173.2	<.001
Aortic resection	7 (3)	19 (16)	18.00	<.001
Appendectomy	5 (5)	9 (6)	7.0	.008
Arthroplasty of the knee	4 (2)	10 (8)	12.5	<.000
CABG	7 (3)	13 (21)	8.1	.004
Cholecystectomy	5.5 (7)	9 (14)	10.2	<.001
Colorectal resection	9 (6)	17 (13)	39.3	<.001
Carotid endarterectomy	2 (2)	4 (5)	15.4	<.001
Hip fracture repair	7 (8)	12 (13)	14.5	<.001
Hip replacement	6 (5)	11 (12)	22.9	<.001
Hysterectomy	4 (3)	9 (14)	18.2	<.001
Laminectomy	4 (6)	11 (18)	9.7	.002
Open prostatectomy	4 (3)	7 (5)	14.1	<.001
Peripheral vessel bypass	7 (6)	15.5 (20)	23.5	<.001
Spinal fusion	5 (6)	10.5 (8)	8.6	.003
TURP	3 (4)	7 (8)	15.5	<.001

In parentheses is the interquartile range (3rd quartile-1st quartile).

Discussion

Overall, patients admitted without major impairment and who experienced an adverse occurrence during hospitalization were roughly 2.5 times more likely to have major impairment at discharge than patients who did not have an adverse occurrence. We found this pattern prevailed for almost all of the surgical groups, though this finding failed to reach statistical significance for some surgical categories due to small sample sizes. Patients having an adverse occurrence during their hospitalizations stayed about twice as long as those without an adverse occurrence. Again, the pattern was consistent across procedures.

Our study is among the few that have examined changes in disability and adverse occurrences using a comparison group. Our findings are consistent with those of Garcia-Martin and colleagues⁹ and Zhan and Miller.¹⁷ Adverse incidents are associated with adverse outcomes. In our study, roughly two-thirds of patients who had a major diminution in functional impairment during the hospital stay had an adverse occurrence. For patients who already had major impairment on admission, the likelihood of having an adverse occurrence was about the same as among those not having major impairment on admission (39 percent versus 37 percent). Patients with major disability on admission were unlikely to have a change in functional status, regardless of hospital events.

We examined adverse occurrences, not adverse events. Adverse events have been defined as “an injury that was caused by medical management (rather than the underlying disease) and that prolonged the hospitalization, produced a disability at the time of discharge, or both.”¹⁸ Determination of adverse events typically involves both the finding of an adverse incident and a structured implicit assessment by a reviewer as to the causal relationship of medical management to the occurrence. Although the literature supports this process,^{18, 19} others suggest it is very much open to interpretation.²⁰ Weingart and Iezzonni²¹ stated, “You can't measure what you can't define.” By adopting the approach of examining the presence of incidents rather than attempting to discern errors, we believe we have increased the chart review's accuracy in making an association between adverse occurrences and functional status change.

We used a validated measure of functional status assessment. Most adverse event literature uses the reviewer's opinion of the likelihood of disability (e.g., functional or cosmetic) lasting for a period of time or interfering with activities.^{6, 18} Although these assessments have reliability,²² they are not directly related to explicit assessments of activities of daily living or another important marker of functional status: the ability of a patient to walk and take care of bodily needs without aid. We believe the Modified Rankin assessment better reflects the measurement capability of hospital staff and is therefore reproducible. It also is more likely to be associated with the need for either home care or long-term care followup at time of discharge. What the status is of patients at 6–12 months followup for a variety of adverse occurrences is not known.

Like other studies, we found that adverse incidents increased LOS.^23–25 Zhan & Miller, ¹⁷ looking at a much larger and broader sample of patients, were able to demonstrate increased LOS for various AHRQ patient safety categories. In our study, we looked at a variety of adverse occurrences within a delimited set of surgical groups common among Medicare beneficiaries. The overall LOS was nearly doubled, and there were statistically significant increases in LOS among every examined surgical category.

There are limitations to the study design, results, and conclusions. This project was not designed to examine the relationship of adverse occurrences and functional impairment, but was designed to develop and test a brief tool that hospitals could use to retrospectively review medical charts for missed adverse occurrences. As such, we did not randomly sample across all surgical procedures, nor did we select sample sizes based on power calculations. To determine the sensitivity and specificity of the abstraction tool, we sampled half from surgeries already associated with adverse occurrence and half from matched surgeries without associated occurrences. As a result, findings cannot be generalized to all surgical procedures. The true real-world distribution of adverse occurrences in the study's surgical procedures is likely to be significantly smaller.

The sample included Medicare fee-for-service beneficiaries, who represent approximately 85 percent of the Medicare population in NYS. Findings may not be applicable to Medicare managed-care beneficiaries or all patients undergoing the studied surgical procedures. Although we utilized both the ASA and Modified Rankin scales to adjust for acuity of illness, we could not adjust for all the differences due to unmeasured aspects of clinical variation among patients. Not all adverse occurrences are due to medical errors, but we believe that using the NYPORTS taxonomy maximizes the likelihood of accurately determining untoward incidents. Further focused review of these incidents by hospitals will determine whether process changes can improve outcomes.

Adverse occurrences are not all the same. We have demonstrated that the pattern of functional status change is consistent across many of the NYPORTS occurrences; however, our sample size for each individual type of occurrence (e.g., hematoma) was too small for true statistical comparison. We also did not group adverse occurrences according to seriousness; instead we analyzed adverse occurrences as one group, giving each an equal weight. Although not ideal, it is a first step.

We sought to examine whether a major change in functional status was more likely to occur among surgical patients who experienced an adverse occurrence than among patients who had a similar surgery but did not have an adverse occurrence. We found there was harm, as measured by impairment at discharge. How long these impairments persist before they resolve remains a question.

Finally, the use of retrospective medical chart review by nurses to identify adverse occurrences may not provide adequate information about an adverse event. When hospitals in NYS report an adverse occurrence into NYPORTS, it is usually not due to a medical chart review. Occurrences are identified through other means and, if appropriate, root-cause analysis is done; many data sources are reviewed; and the providers are interviewed. Initial screening is costly and time consuming. We sought to explore the use of a brief nurse medical chart review as a first stage screening. Of course, the final determination of an adverse occurrence requires more careful examination by a clinician.

Despite the study's shortcomings, the information collected has the potential to highlight areas for quality improvement efforts and future study. It may be valuable to target peripheral vascular bypass and colorectal resection in patient safety efforts. Not all surgical procedures result in an increased risk of impairment following an adverse occurrence.

Conclusion

Much attention has been paid to defining and classifying adverse occurrences and reporting systems. Little work has focused on whether adverse occurrences are associated with disability. To our knowledge, there are no studies that directly assess disability using a clinically accepted functional status measure among patients who experienced an adverse occurrence, compared to similar patients who did not experienced an adverse occurrence.

We examined the consequences of an adverse occurrence among surgical patients. Overall, there was a relationship between adverse occurrence and impairment, even after adjusting for baseline functional status. Twenty-six percent of patients who had no-to-mild functional impairment at admission and who experienced an adverse occurrence left the hospital with significant impairment—more than twice the incidence among similar patients who did not have an adverse occurrence. Adverse occurrences are associated with an increase in the risk of functional impairment and a near-doubling of length of stay.

Acknowledgments

We would like to thank the New York State Department of Health and the NYPORTS subcommittee for their support design of the project and review of preliminary results, Thomas Huang for his programming and sampling, and Dee Nelson for her editorial comments.

Funding for this project came from the Centers for Medicare and Medicaid Services awarded to IPRO, Contract No. 500-99-NY01.

Author affiliations

Mt. Sinai School of Medicine, New York (MR). IPRO, Lake Success, NY (AS, CL). North Shore-Long Island Jewish Health System, Great Neck, NY (AS, LR).

Address correspondence to: Alan Silver, M.D.; IPRO, 1979 Marcus Avenue, Lake Success, NY 11042-1002. Phone: 516-326-7767, x509; fax: 516-326-7462; e-mail: asilver@nyqio.sdps.org.

References

1.

Thomas E, Studdert D, Burstin H. et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care. 2000; 38(3): 261–71. [PubMed]

2.

Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ. 2001; 322(7285): 517–9. [PubMed] [Free Full text in PMC]

3.

Leape L, Brennan T, Laird N. et al. The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II. NEJM. 1991; 324: 377–84. [PubMed]

4.

de Haan R, Limburg M, Bossuyt P. et al. The clinical meaning of Rankin ‘handicap’ grades after stroke. Stroke. 1995; 26(1): 2027–30. [PubMed]

5.

Gawande A, Thomas E, Zinner M. et al. The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery. 1999; 126(1): 66–75. [PubMed]

6.

Kable A, Gibberd R, Spiegelman A. Adverse events in surgical patients in Australia. Int J Qual Health Care. 2002 August;14(4): 269–76. [PubMed]

7.

Davis P, Lay-Yee R, Briant R. et al. Adverse events in New Zealand public hospitals I: occurrence and impact. New Zealand Med J. 2002; 115(1167): U271. [PubMed]

8.

Rosen A, Geraci J, Ash A. et al. Postoperative adverse events of common surgical procedures in the Medicare population. Med Care. 1992; 30(9): 753–65. [PubMed]

9.

Garcia-Martin M, Lardelli-Claret P, Bueno-Cavanillas A. et al. Proportion of hospital deaths associated with adverse events. J Clin Epidem. 1997; 50(12): 1319–26.

10.

New York State Department of Health. NYPORTS: the New York State Occurrence and Tracking System Annual Report. Albany, NY: 2001.

11.

IPRO. Collaboration to optimize New York Patient Occurrence and Reporting System (NYPORTS). Lake Success (NY): Centers of Medicare and Medicaid Services; 2002.

12.

Mausner J, Kramer S. Epidemiology: an introductory text. 2nd ed. Philadelphia: W.B. Saunders; 1985.

13.

Agency for Health Care Policy and Research. Clinical Classifications Software (ICD-9-CM) summary and download. Rockville, MD. Available at http://www.ahrq.gov/data/hcup/ccs.htm. Last accessed April 2003.

14.

American Society of Anesthesiologists Physical Status measure (ASA). Available at http://www.asahq.org/clinical/physicalstatus.htm. Last accessed March 15, 2004.

15.

Landis J, Koch G. The measurement of observer agreement for categorical data. Biometrics. 1977; 33: 159–74. [PubMed]

16.

van Swieten J, Koudstaal P, Visser M. et al. Interobserver agreement for assessment of handicap in stroke patients. Stroke. 1988; 19: 604–7. [PubMed]

17.

Zhan C, Miller M. Excess length of stay, charges and mortality attributable to medical injuries during hospitalization. JAMA. 2003; 290(14): 1868–74. [PubMed]

18.

Brennan T, Leape L, Laird N. et al. Incidence of adverse events and negligence in hospitalized patients: results from the Harvard Medical Practice Study I. NEJM. 1991; 324: 370–6. [PubMed]

19.

Localio A, Weaver S, Landis J. et al. Identifying adverse events caused by medical care: degrees of physician agreement in a retrospective chart review. Ann Intern Med. 1996; 125(6): 457–64. [PubMed]

20.

Thomas E, Studdert D, Brennan T. Reliability of medical record review for estimating adverse event rates. Ann Intern Med. 2002; 136: 812–6. [PubMed]

21.

Weingart S, Iezzonni L. Looking for medical injuries where the light is bright. JAMA. 2003; 290(14): 1917–9. [PubMed]

22.

Brennan T, Russell J, Laird N. Reliability and validity of judgments concerning adverse events suffered by hospitalized patients. Med Care. 1989; 27: 1148–58. [PubMed]

23.

Classen D, Pestonik S, Evans R. et al. Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality. JAMA. 1997; 227(4): 301–6. [PubMed]

24.

Bates D, Spell N, Cullen D. et al. The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group. JAMA. 1997; 277(4): 307–11. [PubMed]

25.

Kalish R, Daley J, Duncan C. et al. Costs of potential complications of care for major surgery patients. Amer J Med Qual. 1995; 10(1): 48–54. [PubMed]