Re-engineering the Hospital Discharge: An Example of a Multifaceted Process Evaluation

Anthony D, Chetty VK, Kartha A, et al.

Publication Details

Abstract

Introduction: The transfer of patient care from the hospital team to primary care and other providers in the community at the time of discharge is a high-risk process characterized by fragmented, nonstandardized, and haphazard care that leads to errors and adverse events. The development of interventions to improve the discharge process requires a detailed evaluation of the process by a multidisciplinary team. Methods: Using the resources of the Boston University-Morehouse College of Medicine AHRQ Developmental Center for Patient Safety Research (funded by the Agency for Healthcare Research and Quality), multidisciplinary teams have been assembled to identify and address the sources of error at discharge. To better understand the current hospital discharge process, the researchers have applied a battery of epidemiologic and quality control methods taken from industry. These include probabilistic risk assessment, process mapping, qualitative analyses, failure mode and effects analysis, and root cause analysis. The researchers describe each of these methods and discuss their experience with them, displaying concrete tools that have arisen from their application. Conclusions: A detailed, multifaceted process analysis has provided us with powerful insight into the many patient safety issues surrounding the discharge process. The generalizable methods described here have produced the re-engineering of the discharge process, allowing for the planning of a clinical trial and significant improvements in patient care.

Introduction

According to the Institute of Medicine's (IOM's) 1999 report, To Err Is Human: Building a Safer Health System, 1 the number of deaths due to iatrogenic errors of omission and commission in hospitals is estimated to be between 44,000 and 98,000 per year. More people die each year from medical errors than from car accidents (43,458), breast cancer (42,297) or AIDS (16,515). 2 In 2001, a second IOM report, Crossing the Quality Chasm: A New Health System for the 21st Century, 3 noted: “Effective methods of communication, both among caregivers and between caregivers and patients, are critical to providing high-quality care. Personal health information must accompany patients as they transition from home to clinical office setting to nursing home and back.”

Since 2001, our team has used the resources of the Boston University-Morehouse College of Medicine AHRQ Developmental Center for Patient Safety Research (DCERPS) to identify and address sources of error at discharge that lead to subsequent hospital utilization.

In an effort to improve the care of our patients, we are studying the transition from the inpatient service at Boston Medical Center (BMC) to community care. In our inner-city, safety-net environment, we have identified the high rate of unnecessary rehospitalization as a major problem that has existed (at least anecdotally) for the institution's 150-year history.

Investigating the sources of the high rate of rehospitalization has made it clear that the hospital discharge process is characterized by fragmented, nonstandardized, and haphazard care. The problems inherent in such care are compounded because the 15-minute post-hospital follow-up visit scheduled by primary care physicians (PCPs) does not allow the physician adequate time to become familiar with the details of the hospitalization. Most such visits must be added to already overbooked schedules at the time of discharge. Increasingly, as hospitalists provide more inpatient care, it is difficult for PCPs to be aware of all the complexities of a hospitalization. Thus, the transition from hospital care to primary care is a hand-off that provides an opportunity for a high rate of medical errors. Forster, et al. 4 provide data demonstrating that many adverse events occur at the time of hospital discharge. Many of these events were deemed preventable and thus could be viewed as errors, while others were simply previously undetected adverse events. It is likely, therefore, that carefully designed interventions that target the discharge process could successfully reduce medical errors, adverse events, and rates of subsequent hospital utilization. However, before such interventions can be devised, it is essential to diagnose the problem systematically.

The Structure-Process-Outcome model 5 can be applied to hospital discharge since there are risks and hazards embedded within the structure and process of care that can potentially cause injury or harm to patients. There is also potential for active failure among the “sharp end” (i.e., direct patient care) providers responsible for the discharge. In order to produce an effective intervention, we have used a multifaceted approach to perform a comprehensive process evaluation. With this work, we are developing tools that can now be tested, allowing us to measure their impact; ultimately, these tools will be used as part of an outcome evaluation in an interventional trial. Our conceptual model of the components related to high rates of rehospitalization is shown in Figure 1.

FIgure 1. Conceptual model of the components affecting rates of re-hospitalization.

Figure

FIgure 1. Conceptual model of the components affecting rates of re-hospitalization.

Hospital discharge as a dangerous situation

The IOM report To Err Is Human argued that the majority of medical errors are a result of systemic problems rather than poor performance by individual providers. 1 Hospital discharge is an example of a systemic problem that can be characterized as a dangerous situation in which latent conditions exist such that sharp end individuals are set up to fail. While knowledge-based, rule-based, and skill-based behaviors are needed for optimal care, there are many opportunities for slips, lapses, mistakes, and adverse events.

Re-engineering clinical systems requires an understanding of the causes of errors and the use of safety design concepts to prevent or minimize errors by detecting them before harm occurs. 6 Our analysis considers both active and latent errors occurring at the time of hospital discharge. Active errors include, for example, those occurring at the time of hospital discharge during knowledge-based decisionmaking performed at the point of care. 7 Needing a conceptual model from which to start our analysis, we developed a taxonomy of the types of errors that can occur at the time of hospital discharge to guide our work (Figure 2). This taxonomy demonstrates how latent and active errors interrelate, and highlights the importance of rule-based decisionmaking. One such rule to be followed at the time of discharge is scheduling an appointment with a primary care provider at a time convenient to the patient. Errors occur when latent conditions or system failures occur that are the consequences of failures in technical design or organization. For instance, in many hospitals, nurses and first-year residents are responsible for the discharge process. The harried nature of their work, as well as competing interests (e.g., new admissions requiring attention), results in the discharge of a patient not being considered a high priority, and can lead to an incomplete discharge process. Patient discharge is also fragmented among various caregivers, including first-year residents, nurses, trainees in both fields, and support staff. A clear delineation of discharge responsibilities often does not exist and lack of communication results in repetition and gaps. Clearly, the hospital discharge is a time when accidents and adverse events happen because latent conditions or system failures are combined with active failures.

Figure 2. Taxonomy of errors at time of hospital discharge.

Figure 2

Taxonomy of errors at time of hospital discharge.

It is therefore not surprising that medical errors are an important concern at hospital discharge. In their study, Forster et al. 4 identified four areas of systems requiring improvement: (1) assessment and communication of problems that remain unresolved at the time of discharge; (2) patient education regarding medications and other therapies; (3) monitoring of drug therapies after discharge; and (4) monitoring of the overall condition after discharge. Many adverse effects occur during the peridischarge period, and many could be prevented with relatively simple strategies. As Moray 8 and Van Cott 9 point out, the prevention of active or sharp end failures requires re-engineering systems so that they are designed for safety.

In our pursuit of an effective intervention, we have employed a user-centered framework that recognizes human factors and cognitive engineering principles. 10, 11 In so doing, we have assembled a multidisciplinary team and used a combination of epidemiologic methods and quality control techniques taken from industry. These methods include: probabilistic risk assessment, process mapping, failure mode and effects analysis, qualitative interviews, and root cause analysis (Figure 3). We propose that this multifaceted approach, which has significantly advanced our understanding of the discharge process, may be generalized to many or most patient safety problems.

Figure 3. Schematic of the methods used in the process assessment of the hospital discharge.

Figure 3

Schematic of the methods used in the process assessment of the hospital discharge.

Methods

In describing the process we used to accomplish our work, we hope to demonstrate a useful model for the investigation of health care quality and patient safety issues. Overall, our process has included the development of multidisciplinary teams who then employed the tools we chose for our multifaceted investigation.

Project team

To widen our perspective, we first assembled two multidisciplinary teams to conduct the various aspects of the project. An advisory group—made up of the chief medical officer; the directors of our DCERPS center; senior researchers and statisticians; and BMC's directors of nursing, inpatient service, case management, and quality improvement—meets monthly to provide oversight, make recommendations to the project team, and assure sustainability of the outcomes of our work. The working group meets weekly and consists of a health services researcher, statistician, substance abuse counselor, nurse manager from the medical wards, clinical pharmacist, directors of the inpatient teams, a research assistant, and a member of the hospital administrative staff. Working both individually and as a group, the working group has conducted the separate aspects of our approach. This dual structure ensures continuity of progress and diversity of input, coupled with valuable oversight and hospital-wide support for our efforts.

Probabilistic risk assessment

Keeping in mind our ultimate goal of developing an intervention to improve the quality of the discharge process and reduce subsequent hospital utilization, we began by identifying those patients at highest risk of rehospitalization. We conducted this portion of the project in two phases. First, we used hospital administrative data to formulate a statistical model to predict the probability of rehospitalization when a patient is discharged, with the presence of a rehospitalization within 90 days as the dependent variable. For each admission, we calculated the number of admissions that patient experienced in the preceding 6 months. We evaluated the contribution of each of the 10 most frequent admitting diagnoses and used the Deyo adaptation of the Charlson Comorbidity Index 12 to generate comorbidity scores from the ICD-9 codes of the three additional diagnoses from each admission. Using logistic regression, we found that age, length of stay, number of prior admissions, and the Charlson Comorbidity Index all significantly predict rehospitalization.

To improve the power of the predictive model described above for a high-risk population of patients, we evaluated the utility of adding psychosocial information to the model. After institutional review board approval, we identified currently admitted patients who had a prior admission in the last 6 months and collected from consenting patients' medical records (1) demographics (age, gender, insurance, race, ethnicity); (2) admission diagnosis and other diagnoses; (3) medications; and (4) length of stay. Additionally, we conducted structured interviews with these patients, during which we administered the following eight survey instruments: the Mini-Mental Status Examination, 13, 14 the Short Form 12-item survey of health status (SF-12), 15, 16 the depression and anxiety components of the Patient Health Questionnaire, 17, 18 the Nutrition Screening Initiative checklist, 19, 20 the Norbeck Social Support Questionnaire, 21 23 the Alcohol Use Disorder Identification Test (AUDIT), 24 the Drug Abuse Screening Test (DAST), 25, 26 and a patient satisfaction survey. 27 We are now following this cohort and recording their hospital activity during the 3 months following discharge, identifying hospital utilization via the hospital database and phone calls to all subjects. Analysis of the first 59 patients shows the mean age is 54 years, 56 percent are women, 71 percent speak English, 45 percent are black and 33 percent are white. Two-thirds of patients categorized as high-risk by the single criterion of being admitted previously (within the past 6 months) were hospitalized in the next 90 days.

In interim analyses, we have found that the Physical Component Summary of the SF-12 (SF-12 PCS) score was associated (P = 0.002) with rehospitalization. We have entered the three most significant predictors of readmission (SF-12 PCS, Emotional Support scale of the NSSQ and the Major Depressive Disorder scale of the Patient Health Questionnaire) into a logistic regression model. Physical functioning and mental health have been associated with hospital utilization and death among urban elders using the emergency room, an observation that supports our finding in this pilot. 28 In recent months, we have added two instruments, both well validated to assess health literacy: the Rapid Estimate of Adult Literacy in Medicine (REALM) 29 and the Test of Functional Health Literacy in Adults (TOFHLA). 30 These instruments will be included in subsequent analyses. We have found this process invaluable in focusing our efforts on the patients most at risk for bad outcomes. As resources are always limited, pairing the process measures below with a probabilistic risk assessment helps to insure a targeted and cost-effective intervention.

Process mapping

Process maps are one of the most effective ways of gaining an understanding of existing processes. Such maps are intended to represent a process in such a way that it is easy to read and understand. A process map is considered to be a visual aid for picturing work processes that show how inputs, outputs, and tasks are linked. 31 It has been described as being one of the “most important and fundamental elements of business process re-engineering.” 32 Process maps have several benefits: 33 (1) they give a clearer explanation of a process; (2) the action of working to develop process maps imparts understanding of the tasks and problems faced within the organization; (3) these maps rapidly allow participants in individual tasks to see the entire process and help clarify their interactions with others involved; and, (4) the maps prompt new thinking. It is important to be sure process maps are clearly understood. 34 Given the large number of potential users from varied backgrounds, it is difficult to establish a universally understood representational format. There are a number of different methods for process mapping. The American Society of Mechanical Engineers (ASME) has produced a mapping standard that is widely used in manufacturing and increasingly popular in office and service environments. 35 It is suited for detailed level mapping and has the distinct advantage that inherent in its use is an evaluation of whether a step is value-adding. The ASME approach uses 11 different symbols in process diagrams. We have tailored this approach to the mapping of the discharge process.

To map the hospital discharge process, we employed the dual structure of our weekly working group and monthly advisory group. Using an iterative group process over a 3-month period, we explored all elements of the hospital discharge. Using ASME process mapping standards, each step in the process was reviewed and modeled to document how that process is currently performed. We then printed the map on poster-sized paper and brought it to meetings of residents, nurses, and ancillary staff. In each case the map was reviewed and revised based upon feedback and observations of the groups (Figure 4). This work provided structure to our thinking about the discharge process and will serve to display the standard care received by a control group in future clinical trials.

Figure 4. A portion of the discharge process map.

Figure 4

A portion of the discharge process map.

Qualitative analysis

Qualitative research embodies a family of approaches with the overall goal of understanding the life experience of people. 36 Qualitative approaches are especially useful for addressing questions of process (e.g. organizational change, decisionmaking), perceptions and understandings, experience, and descriptions of an event or situation. Theoretical approaches to qualitative research vary and are derived from many different intellectual disciplines including anthropology, sociology, psychology, and linguistics. In contrast to quantitative approaches that may seek to control for or predict phenomena, qualitative studies tend to focus on the natural history of events or relationships, highlighting themes of the phenomenon of interest as seen from an individual's perspective. This approach can give a broad view and provide rich information when paired with quantitative studies.

Interviews with key informants comprise one domain of qualitative research and inquiry. One-on-one interviews can vary in terms of structure and the latitude the interviewee has in responding to questions. 37 Interviewing is a useful way to get large amounts of information quickly. Information about complex or ambiguous topics may be immediately addressed with followup and clarification. Therefore, using this technique to study the processes of hospital discharge offers many advantages.

As a pilot test of interviewing, we conducted focus groups with physicians who admit their patients to the inpatient service, as well as semi-structured, open-ended interviews with 12 patients on the inpatient service who have been rehospitalized within 90 days of a prior admission. The following themes have emerged as contributing to rehospitalization:

1.

lack of adequate support (social, familial, financial);

2.

premature discharge from hospital;

3.

nonadherence with medication;

4.

nonadherence with followup procedures or instructions;

5.

substance abuse;

6.

homelessness;

7.

events external to the patient that were out of their control;

8.

incarcerated patients, who thus have limited control over dietary restrictions and activity level; and

9.

delay in seeking medical treatment at the first sign of recurring symptoms. These data suggest that there are generic issues related to rehospitalization that go beyond the diagnosis or the clinical and demographic variables.

We are now commencing a more detailed qualitative investigation of rehospitalized patients, in which we will design an interview template to explore the key conceptual domains identified in the pilot study. The template will be revised on the basis of our initial results to maximize reliability and technical and conceptual validity. Indepth interviews with subjects and with one family member or support person will be conducted. From these interviews, a richness of detail about internal experiences and beliefs will be extracted. This exploration of attitudes and beliefs will inform the development of more quantifiable indices regarding hospital discharge processes, through which we will explore belief systems and self-reported behaviors regarding a person's understanding of his or her illness, help-seeking behavior, and the rationale for and justification of behaviors (e.g., medication taking, follow-through with self-care, appointments, and self-monitoring) that might lead to or prevent rehospitalization. We view these efforts as indispensable in guiding our investigation of the discharge process, as they allow a perspective not achieved through our research teams.

Failure mode and effects analysis (FMEA)

Failure mode and effects analysis (FMEA) is an ongoing quality improvement process that stems from the work of Reason and may be carried out in health care organizations. 38 FMEA is a proactive process that acknowledges that errors are inevitable and predictable, anticipates their occurrence, and designs systems that will minimize their impact. If employed before new processes are implemented, FMEA can identify potential failure modes. As a result, the process might reveal that specific steps must be put in place to address and avoid potential errors with significant impact—errors that are intolerable. 39, 40 FMEA is used extensively in the aerospace, nuclear, manufacturing, and chemical industries as a proactive, systematic way of examining processes for possible ways in which failure can occur. With patient safety now a priority, the technique recently has seen application in health care. FMEA, which is widely used at NASA, is now a requirement under the Joint Commission on the Accreditation of Healthcare Organizations' Patient Safety Standards. 41, 42 The NASA model includes the identification, selection, and screening of potential initiation events (IEs) that would lead to errors, and the modeling of scenarios linking each IE, by way of pivotal events (PEs).

We analyzed the expected and unexpected errors occurring at the hospital discharge using the process map. The project team scheduled two 4-hour sessions in an area away from the activities of daily patient care to brainstorm the potential errors highlighted by the process map, giving particular focus on IEs and PEs. The team identified and categorized all potential sources of medical error associated with the hospital discharge processes, using event and fault trees. Once the failure modes were identified, the staff determined the likelihood of making a mistake and potential consequences of the error. It then identified any pre-existing processes that could help detect the initiating and pivotal events that could lead to error, and suggested an action plan for each significant failure mode (Table 1). Such results are invaluable in guiding the development of interventions to reduce errors and improve quality. In combination with the targeting allowed by the probabilistic risk assessment, the structure of the process map, and the detail allowed by the qualitative work, our FMEA has led to important, focused steps that are improving patient care.

Table 1. A portion of the failure mode and effect analysis (FMEA)*.

Table 1

A portion of the failure mode and effect analysis (FMEA)*.

Root cause analysis

Root cause analysis is a technique that we are using to complement the FMEA approach described above. While FMEA is a technique that explores potential errors and means of reducing them, root cause analysis provides indepth insight into errors that have actually occurred—in our case, preventable rehospitalizations. Accurate identification of the root causes of rehospitalization should precede identification and implementation of appropriate interventions. 43 Root cause analysis is a process for identifying the basic or causal factors that underlie a variation in performance, such as the occurrence of a sentinel event. 44 A root cause analysis focuses primarily on systems and processes, not individual performance. It progresses from special causes in clinical processes to common causes in organizational processes, and identifies potential improvements in processes or systems that would tend to decrease the likelihood of such events in the future. 45

Root cause analysis can be both reactive and proactive. Most commonly, the technique is used reactively, to probe the reason for a bad outcome or for problems that have already occurred. Root cause analysis may also be use to probe a near-miss mistake, an error is caught before it causes harm. The product of the root cause analysis is an action plan that identifies the strategies to reduce the risk of similar events occurring in the future.

Boston Medical Center has conducted root cause analyses of both major adverse outcomes and near misses for the past 3 years. The root cause analyses group is coordinated by the director of quality improvement and involves senior leadership of the medical center. Root cause analysis meetings are deemed a very high priority by the medical center leadership, ensuring that all those involved in an event convene within 24 hours of the occurrence to initiate the analysis.

We are using the structure of our existing teams and the institutional familiarity with such procedures to apply root cause analyses to the discharge process. The team has begun to explore the details of patients who have been rehospitalized to the inpatient service, and identify and categorize all potential sources of medical error associated with the hospital discharge processes. The teams will then dig deeper by asking “Why?” and, when answered, ask “Why?” again, in an iterative process. Such iteration will help identify enhancements of systems and processes that would reduce the risk of rehospitalization in the future, and lead to an action plan that takes into account human and other factors most directly associated with the sentinel event, and the process and systems related to its occurrence.

Conclusion

By bringing together a multidisciplinary team to apply several epidemiologic and quality control methods, we are identifying high risk patients, detailing the essential aspects of the discharge process, and identifying sources of error that can impact on outcomes. The diversity of methods applied to this problem has given us valuable insight into the complex array of issues surrounding the high-risk transition of patients from the hospital to community care providers. Such insight would not come from any one method alone. The successes of our project have been greatly aided by the resources available to us through the Boston University—Morehouse College of Medicine AHRQ DCERPS Center, and from the support of senior administrators at Boston Medical Center.

As a concrete example of the value of our multifaceted approach, we have begun to take what we have learned to improve care at Boston Medical Center and the surrounding community health centers. With the above analyses fresh in mind, we have brought the advisory group together for three 2-hour sessions to create a newly re-engineered discharge process. We began by printing the process map on cardboard and cutting out each individual component of the map. The pieces were then placed in an envelope. Eight similar envelopes were produced for use by the advisory group, which was divided into groups of two or three. Each small group was instructed to use its creativity and the knowledge gained by the previous work to develop a new process map that solves the problems identified in the FMEA. Each of the eight groups then described their new map and the new themes or principles that they thought important to any new process. These “principles of the newly re-engineered hospital discharge” were captured and discussed at subsequent meetings and are listed below:

  • There must be explicit delineation of roles and responsibilities.
  • Patient education must occur throughout the hospitalization, not only at the time of discharge.
  • Information must flow easily from the PCP to the hospital team, among the hospital team, and back to the PCP.
  • Information should be captured throughout the hospital stay, not only at the time of (or after) discharge.
  • Every discharge must have a written discharge plan that is comprehensive in scope and that addresses medications, therapies, dietary and other lifestyle modifications, followup care, patient education, and instructions about what to do if the condition worsens.
  • This comprehensive discharge plan should be completed before the patient leaves the hospital.
  • Patients at high risk of rehospitalization should have the discharge plan reinforced by contact from the hospital team after discharge.
  • All information about the admission must be organized and delivered to the PCP within 24 hours.
  • Waiting until the discharge order is written before beginning the discharge process is likely to increase the risk of errors.
  • Efficient and safe hospital discharge is significantly more difficult to achieve if the case management staff works only the 7 a.m.-3 p.m. shift (i.e., the “first” shift).
  • All patients should have access to their discharge information in their language and at their educational level.

We anticipate that the success of our work will culminate in the development of a new, re-engineered discharge process and its evaluation in a clinical trial. The strength of the methods described here come from the diversity of their input and output. We encourage other groups investigating patient safety issues to employ the combination of probabilistic risk assessment, process mapping, qualitative analysis, failure mode and effects analysis, and root cause analysis to fully understand the complexity of issues that affect quality and the care of patients.

Acknowledgments

The authors would like to acknowledge the work of all of the members of the advisory and working groups, especially: Gail Burnoske, Phar.D.; Kevin Casey, M.P.H.; John Chessare, M.D.; Larry Culpepper, M.D., MPH; Jeffrey Greenberg, M.D.; Christopher Manasseh, M.D.; Amy Rosen, Ph.D.; Nancy Torres-Finnerty, M.D.; and Cornelia Walsh, R.N..

References

1.
Kohn LT, Corrigan JM, Donaldson MS, editors. Err is human: building a safer health system. A report from the Committee on Quality of Health Care in America. Institute of Medicine. Washington, DC: National Academy Press; 2000.
2.
Centers for Disease Control and Prevention (National Center for Health Statistics). Births and Deaths: Preliminary Data for 1998. National vital statistics reports 47(25):1999. [PubMed: 10641521]
3.
Crossing the quality chasm: a new health system for the 21st century. A report of the Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC: National Academy Press; 2001.
4.
Forster AJ, Murff HJ, Peterson JF. et al. The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med. 2003;138:161–7. [PubMed: 12558354]
5.
Eisenberg JM. Medical errors as an epidemic. A presentation at the national Summit on medical errors and patient safety research. Washington, DC: QuIC; 2000 Sept.
6.
Joint Commission on Accreditation of Healthcare Organizations. 2005 National patient safety goals. http://www​.jcaho.org​/accredited+organizations​/patient+safety/npsg.htm.
7.
Leape LL, Brennan TA, Laird N. et al. The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice study II. N Engl J Med. 1991;32:377–84. [PubMed: 1824793]
8.
Moray N. Error reduction as a systems problem. In: Bogner MS, editor. Human error in medicine. Hillsdale, NJ: Lawrence Erlbaum, 1994.
9.
Van Cott H. Human error: their causes and reduction. In: Bogner MS, editor. Human error in medicine. Hillsdale, NJ: Lawrence Erlbaum, 1994.
10.
Norman DA. The design of everyday things. New York: Doubleday; 1988.
11.
Donabedian A. Explorations in quality assessment and monitoring: the definition of quality and approaches to its assessment, Vol I. Ann Arbor, MI; Health Administration Press; 1980.
12.
Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. Journal of Clinical Epidemiology. 1992;45(6):613–9. [PubMed: 1607900]
13.
Gallo JJ, Fulmer T, Paveza GJ, et al. Handbook of geriatric assessment, 3rd ed. Gaithersburg, MD: Aspen Publications; 2000.
14.
Folstein MF, Folstein SE, McHugh PR. “Mini-mental state.” A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–98. [PubMed: 1202204]
15.
Ware JE, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: Construction of scales and preliminary tests of reliability and validity. Medical Care. 1996;34(3):220–33. [PubMed: 8628042]
16.
Gandek B, Ware JE. Cross-validation of item selection and scoring for the SF-12 health survey in nine countries: results from the IQOLA Project. J Clin Epidemiol. 1998;51(11):1171–8. [PubMed: 9817135]
17.
Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary care evaluation of mental disorders. Patient health questionnaire. JAMA. 1999;282(18):1737–44. [PubMed: 10568646]
18.
Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–13. [PMC free article: PMC1495268] [PubMed: 11556941]
19.
Posner BM, Jette AM, Smith KW. et al. Nutrition and health risks in the elderly: the Nutrition Screening Initiative. Amer J Public Health. 1993;83(7):972–8. [PMC free article: PMC1694757] [PubMed: 8328619]
20.
Azad N, Murphy J, Amos SS. et al. Nutrition survey in an elderly population following admission to a tertiary care hospital. CMAJ. 1999;161(5):511–5. [PMC free article: PMC1230579] [PubMed: 10497606]
21.
Norbeck JS, Lindsey AM, Carrieri VL. The development of an instrument to measure social support. Nurs Res. 1981;30:264–9. [PubMed: 7027185]
22.
Norbeck JS. Scoring instructions for the Norbeck Social Support Questionnaire (NSSQ), revised 1995. http://www​.nurseweb.ucsf.edu/.
23.
Norbeck JS, Lindsey AM, Carrieri VL. Further development of the Norbeck Social Support Questionnaire: normative data and validity testing. Nurs Res. 1983;32:4–9. [PubMed: 6549842]
24.
Allen JP, Litten RZ, Fertig JB. et al. A review of research on the Alcohol Use Disorders Identification Test (AUDIT) Alcoholism: Clinical & Experimental Research. 1997;21(4):613–9. [PubMed: 9194913]
25.
Skinner HA. The Drug Abuse Screening Test. Addictive Behaviors. 1982;7:363–71. [PubMed: 7183189]
26.
Gavin DR, Ross HE, Skinner HA. Diagnostic validity of the Drug Abuse Screening Test in the assessment of DSM-III drug disorders. Brit J Addict. 1989;84(3):301–7. [PubMed: 2650770]
27.
Department of Health Research and Services Administration. Bureau of Primary Health Care Patient Satisfaction Survey. http://www​.bphc.hrsa​.gov/quality/PatientTool.htm.
28.
Friedmann PD, Jin L, Karrison TG. et al. Early revisit, hospitalization, or death among older persons discharged from the ED. Am J Emerg Med. 2001 Mar;19(2):125–9. [PubMed: 11239256]
29.
David TC, Crouch MA, Long SW. et al. Rapid assessment of literacy levels of adult primary care patients. Fam Med. 1991;23(6):433–5. [PubMed: 1936717]
30.
Parker RM, Baker DW, Williams MV. et al. The test of functional health literacy in adults: a new instrument for measuring patients' literacy skills. J Gen Intern Med. 1995;10(10):537–41. [PubMed: 8576769]
31.
Anjard RP. Process mapping: a valuable tool for construction management and other professionals. Int J Operat Product Mgt. 1998;16(3/4):79–81.
32.
Soliman F. Optimum level of process mapping and least cost business process re-engineering. Int J Operat Product Mgt. 1998;18(9/10):810–6.
33.
Peppard J, Rowland P. The essence of business process re-engineering. Hemel Hempstead, UK: Prentice Hall, Europe; 1995.
34.
Curtis B., Kellner M, Over J. Process modeling. Commun ACM 1992 Sept;35(9).
35.
Oakland JS. Total Quality Management: the route to improving performance, 2nd ed. Oxford, UK: Butterworth-Heineman, Ltd.; 1994.
36.
Frankel RM, Devers K. Qualitative research: a consumer's guide. Educ Health (Abingdon) 2000;13(1):113–23. [PubMed: 14741803]
37.
Frankel RM, Devers K. Study design in qualitative research-1: developing questions and assessing resource needs. Educ Health (Abingdon) 2000;13(2):251–61. [PubMed: 14742087]
38.
Reason J. Human error. New York: Cambridge University Press; 1991.
39.
Cohen MR, Davis NM, Sendersj JW. Failure mode and effects analysis: a novel approach to avoiding dangerous medications errors and accidents. Hosp Pharm. 1994;29:319–24. [PubMed: 10133461]
40.
Williams E, Talley R. The use of failure mode effect and criticality analysis in a medication error subcommittee. Hosp Pharm. 1994;29:331–7. [PubMed: 10133462]
41.
Greenfield MA. The inherent values of probabilistic risk assessment. http://www​.hq.nasa.gov​/office/codeq/risk/uva.pdf. 2001, June 19.
42.
Feldman SE, Roblin DW. Medical accidents in hospital care. Applications of Failure Analysis to hospital quality appraisal. JCAHO J Qual Improv. 1997;23:567–80. [PubMed: 9407261]
43.
Van Der Schaaf TW. Medical applications of industrial safety science. Qual Saf Health Care. 2002;11(3):205–6. [PMC free article: PMC1743634] [PubMed: 12486976]
44.
Brennan TA, Leape LL, Laird NM. et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324(6):370–6. [PubMed: 1987460]
45.
Bates DW, Cullen DJ, Laird N. et al. Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group. JAMA. 1995;274(1):29–34. [PubMed: 7791255]