• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of jgimedspringer.comThis journalToc AlertsSubmit OnlineOpen Choice
J Gen Intern Med. Apr 2008; 23(4): 418–422.
Published online Mar 29, 2008. doi:  10.1007/s11606-007-0404-0
PMCID: PMC2359505

Use of a Handheld Computer Application for Voluntary Medication Event Reporting by Inpatient Nurses and Physicians

Abstract

OBJECTIVE

To determine the feasibility of capturing self-reported medication events using a handheld computer-based Medication Event Reporting Tool (MERT).

DESIGN AND PARTICIPANTS

Handheld computers operating the MERT software application were deployed among volunteer physician (n = 185) and nurse (n = 119) participants on the medical wards of four university-affiliated teaching hospitals. Participants were encouraged to complete confidential reports on the handheld computers for medication events observed during the study period.

MEASUREMENTS AND MAIN RESULTS

Demographic variables including age, gender, education level, and clinical experience were recorded for all participants. Each MERT report included details on the provider, location, timing and type of medication event recorded. Over the course of 2,311 days of clinician participation, 76 events were reported; the median time for report completion was 231 seconds. The average event reporting rate for all participants was 0.033 reports per clinician shift. Nurses had a significantly higher reporting rate compared to physicians (0.045 vs 0.026 reports/shift, p = .02). Subgroup analysis revealed that attending physicians reported events more frequently than resident physicians (0.042 vs 0.021 reports/shift, p = .03), and at a rate similar to that of nurses (p = .80). Only 5% of MERT medication events were reported to require increased monitoring or treatment.

CONCLUSIONS

A handheld-based event reporting tool is a feasible method to record medication events in inpatient hospital care units. Handheld reporting tools may hold promise to augment existing hospital reporting systems.

KEY WORDS: incident reporting, medication errors, computers, handheld

INTRODUCTION

Since the publication of the Institute of Medicine’s (IOM) To Err Is Human, national attention has been focused on medical errors and especially hospital safety.1 However, progress toward developing safer health care systems has been slow, impeded in large part by the difficulty in detecting hazards using conventional monitoring or reporting mechanisms.2 In its review of the scope and impact of medication error in hospitals, the IOM estimates that 400,000 preventable drug-related injuries still occur each year, the majority of which are undetected.3 New methods of identifying safety hazards are needed to generate the data to inform system redesign.4

Methods currently available for monitoring patient safety in hospitalized patients include clinician event reporting, direct observation, patient surveys, and extraction of practice data from medical records or computerized databases. Direct observation has been shown to have the highest rate of event detection, but may be prohibitively expensive for routine monitoring.2,5 Patient surveys can identify certain types of events such as adverse drug reactions, but are insensitive to potential events or medication events that are unrecognizable to patients.6 Chart review relies on the retrospective analysis of medical record documents, which may limit the rate of event detection because of deficiencies in documentation.7 In addition, although automated database trigger tools may increase the yield of event detection, they are often limited to lab-related measures and depend on computerized medical record databases not yet available in many clinical settings.816

Medication event reporting remains a central element of many hospital safety monitoring systems. Event reports (sometimes called “incident reports”) offer providers at the sharp end of patient care a means to describe and document safety events that result from system failures.17,18 Clinician-reported events are rarely false-positive, the influence of recall bias is limited,2 and they offer the greatest amount of clinically relevant data gathered at the time of the actual event. However, consistent underreporting by practicing clinicians limits the effectiveness of this component of safety monitoring.1921 Analysis of currently available self-report methods demonstrates that such reports may capture as little as 0.04% of recognizable hospital events.2,22

Emerging handheld technologies could serve to augment clinician self-reporting of medication events in the hospital setting. Handheld computers offer a number of potential advantages over stationary computers including convenience, portability, confidentiality, and point-of-care reporting. Yet, despite an increasing number of clinical applications,23 handheld technology has been used on a very limited basis to facilitate medical event reporting.24,25 To determine the feasibility of a handheld computer-assisted reporting system, we report on the development and deployment of a Medication Event Reporting Tool (MERT) among physicians and nurses on inpatient medical units.

METHODS

Study Participants

A total of 119 nurses and 185 physicians were recruited from four university-affiliated hospitals. Physician participants were enrolled from both general medicine and pediatric service wards and included attending and resident physicians. Physicians were recruited in rotating units of ward teams that were typically comprised of attendings, residents, and interns. Although teams were recruited to participate as a unit, individual clinician participation was strictly voluntary. Nurse participants were enrolled from medicine, surgery, and intensive care units from three of these hospitals. All participant reports were voluntary and confidential, and did not replace the standard event reporting systems currently in place at participating hospital sites. All participants provided written informed consent, and Institutional Review Board (IRB) approval was obtained for data collection at each hospital.

Application

The Medication Event Reporting Tool (MERT), a customized software application, was developed to allow self-reporting of medication events on a Palm Zire™ 21 handheld computer (Palm, Inc., Sunnyvale, CA, USA). The software application was developed on an AppForg platform, with code written in Visual Basic. The MERT interface was designed to be consistent with the standard reporting systems used by participating institutions to facilitate interpretation and classification. Data collected in the report included information about the provider involved, event date and time, location, type, severity, stage of the medication process, and contributing factors (Table 1). The application used a forced-choice, menu-driven response format that directed respondents to provide detailed descriptions of a medication event on the handheld computer. A software algorithm timed and recorded the number of seconds required to complete each report. Individualized medication formularies were constructed on the reporting tool specific to each participating hospital. Collected data were transferred each week via a custom hotsync conduit to a centralized server database from the participating hospitals during the study.

Table 1
Survey Structure for MERT Tool

Protocol

Nurses and physicians were recruited to carry a handheld computer for 1-week work intervals. Participants were eligible to participate for more than 1 study interval based on hospital staffing assignments. Participants received a nominal financial incentive of 4 dollars per day for their participation in the study, but there was no direct incentive tied to the completion of MERT reports. Baseline demographic and professional experience information including descriptive variables of participants’ sex, age, professional experience, and educational level were recorded for all participants by the handheld application through an initial sign-on process. Participants were prompted through an audible alarm feature of the software program, which then subsequently elicited responses from participants and collected survey data. Daily workload information was captured at the beginning of each shift. Random repeating daily surveys collected dynamic characteristics such as work activity, stress, and perceived work demands. These ecologic momentary assessments were elicited from study participants randomly over 90-minute intervals throughout their work shift (data reported separately). At the end of each computer alarm and survey, an opportunity to launch the event reporting application was provided. Participants were also given instructions on how to initiate the MERT application at any time during the study period they wished to report immediately an observed medication event. Event reporting rates were determined as the number of reports generated per shift of clinician participation.

Statistical Analysis

For continuous variables, Student’s t tests were used to determine statistically significant differences between participant groups. A chi-square test of proportions was performed to examine the association between education level and MERT use by nurse participants, and also to compare the response rate differences between participant groups. All statistical calculations were performed using SPSS version 12.0. Statistical comparisons were completed as 2-tailed tests, using an alpha level of 0.05 for determining significance.

RESULTS

Physician participants were evenly distributed between male (46%) and female (54%), while nurse participants were predominantly female (89%). Fifty participants (28 physicians and 22 nurses; 16% of all participants) completed at least 1 electronic MERT report over the 10-month study period (Table 2). Over 2,311 days of clinician participation, reporters generated 76 complete event reports (Table 3). The median time required to complete an event report was 231 seconds (interquartile range of 179.3–281.8). The overall MERT reporting rate for the study period was 0.033 reports/shift.

Table 2
MERT “Reporter” Versus “Non-reporter” Provider Characteristics
Table 3
MERT Event Reports by All Providers

Physician study participants were overall younger and less experienced on their units compared to nurse participants. Physician reporters had significantly more days of study participation than physicians who did not complete an event report (12.9 vs 7.3 days; p < .001). There was no difference in age or unit experience for physician reporters versus nonreporters. Nurse reporters compared to nonreporters were similar in age, experience, and days of participation. There was a trend toward nurse reporters having a higher educational level compared with nurse nonreporters (p = .06). Physician reporters overall had more average days of participation with the handheld tool compared to nurse reporters (12.9 vs 7.5 days; p = .016).

Nurses had significantly higher reporting rates compared to all physicians (0.045 vs 0.026 reports/shift, p = .02). This difference was primarily due to the relatively lower reporting rate for resident physicians. Both attending physicians and nurses reported events significantly more often than resident physicians (0.042 vs 0.021 reports/shift, p = .03 and .045 vs 0.021 reports/shift, p = .003, respectively). In contrast, reporting rates for nurses and attending physicians were similar (0.045 vs 0.042 reports/shift, p = .80). MERT reports were generated in an even distribution across the days of participation. An assessment of the distribution of MERT events suggested a nonsignificant trend toward increased reporting rates during later days of participation (p = .07).

Event reports were fairly evenly distributed between those observed as involving others (53%) and those involving self (47%) (Table 4). House staff physicians reported themselves personally responsible in a majority of medication events (71%), whereas the level of personal responsibility recorded by nurses (44%) and physician (19%) participants was lower. Reported events were predominantly potential adverse events, with medication either not administered to the patient (69%) or administered but resulting in no discernible patient harm (27%). An event resulting in an increased need for monitoring, temporary patient harm, or the need for treatment, was noted in only 5% of the MERT reports.

Table 4
Events Reported by Physician and Nurse Subjects (N = 76)

DISCUSSION

The MERT software application was developed to enable convenient, real-time reporting of medication events on a handheld computer in the hospital setting. Study participants using the handheld tool required less than 4 minutes on average to complete an event report, indicating a response burden, which compares favorably to that described for other computerized event reporting systems.24 Age, clinical experience, and education level of participants did not discriminate reporters versus nonreporters, suggesting acceptance of this tool across a range of hospital clinicians.

Nurses used the handheld application to report medication events at a greater rate than the physicians in this study. This was not a surprising finding as hospital event reporting has historically suffered from significant participation bias, with nursing staff submitting up to 89% of all reports received in some settings,26 and typically very few reports gathered from senior physicians. Subgroup analysis of our data, however, demonstrated that attending physicians and nurses actually displayed very similar rates of event reporting. The lower rate of resident reporting largely accounted for the observed reporting rate difference between nurse and physician groups. These data would suggest that handheld systems may represent an opportunity to mitigate the impact of participation bias and gather event reports more broadly across provider disciplines, particularly from attending physicians.

Resident physicians filed event reports at a rate significantly lower than both attending physicians and nurses in our study. Extended-duration work shifts have put resident physicians at particular risk for medical error leading to adverse patient events.27 Although some studies demonstrate that house staff are willing to report events when appropriately queried,28,29 actively engaging resident physicians in event reporting has been a substantial challenge. Reporting systems are sometimes difficult for house staff to navigate, and lack of clinical experience may interfere with successful reporting.30 In addition, resident physicians experience substantially higher perceived work stress levels in the hospital environment,31,32 and fear of professional or legal reprisal may limit willingness to file event reports.33 Yet, resident physicians are perhaps more likely to utilize handheld computer technology,23 and adapting reporting tools to their needs could enhance their reporting. In addition, house staff were the group most likely to report events for which they were personally responsible (71%). These data underscore the importance of developing tools to facilitate event reporting from resident physicians, and reducing the barriers inherent to conventional reporting systems.

Event reports collected by our handheld computer application reflected a broad spectrum of both actual adverse drug events (ADEs) and near misses (i.e., potential ADEs). Sixty nine percent of events captured by MERT were recorded before medication was administered, while 27% of events resulted in no discernible patient injury after the administration of medication. This observed reporting of near-miss medication events with MERT was greater than that reported by other monitoring systems, which have captured near misses in approximately 35% of events.34 Near misses are an important component of successful reporting systems, as they typically occur at a greater frequency than ADEs and allow for quantitative analysis of system failures that ultimately lead to adverse outcomes.18 Emphasizing the reporting of near-miss events tends to reduce the culture of blame when harmful events do occur, and promotes the study of recovery strategies to build more resilient safety systems.35,36

Finally, the results of this study suggest that this novel handheld tool may hold promise to improve critical event reporting in hospitals as part of an integrated health information technology. There is substantial evidence that inpatient medication events are grossly underreported,2,19,33 and traditional reporting methods are often not well integrated into hospital data systems. Handheld computers offer the potential to bundle clinical care applications on a single portable device, and to prompt users for critical event data reporting. As hospitals increasingly integrate electronic health information technology, bundling an event reporting mechanism into clinical applications offers a unique opportunity to simplify the process and potentially increase the yield of event reports gathered in hospital settings. Future study should critically evaluate the quantitative impact of a handheld event reporting tool in comparison to traditional reporting systems, as hospitals increasingly adopt novel health information technology.

There are limitations to interpreting the results and generalizability of this study, as this was an unblinded evaluation of a newly developed software application. The study participants were physicians and nurses on inpatient units at academic centers, and use in community hospitals will need to be evaluated. A handheld computer-based system would likely only be feasible at hospitals with IT resources to support data download, software modification and maintenance. Considerable attention would need to be given to encrypting and maintaining sensitive clinical information while more broadly integrating this novel technology, and the associated security and information technology (IT) costs of a larger application of MERT were not specifically evaluated in this study. The format of event reports captured on handheld computers also remains limited in its ability to collect narrative data, although advances in scripting and voice recognition software hold promise to allow expanded narrative event reporting on handheld devices in the near future. In addition, the overall percentage of events captured by this handheld event reporting application likely remains quite small, still representing only the “tip of the iceberg” of hospital events. An ideal event reporting system would also include incentives for voluntary reporting, place emphasis on a systems perspective for event analysis, and promote culture change to value event reporting as a nonpunitive means to provide critical feedback and improve patient safety.32,34

A statistical limitation of this study is that our sample size made it impractical to adjust for increases in family-wise error rates introduced by completing multiple tests. The failure to correct statistically for multiple comparisons makes it possible that some of our statistically significant findings occurred by chance. However, the importance of this limitation is mitigated by the consistent pattern of results we observed across multiple groups and the congruence of our results with that found in the literature. Observed reporting rates could have been also artificially inflated by the novelty of the handheld tool, by a volunteer bias introduced by the recruitment of clinician volunteers for the study, and by the nature of the study itself. Data was not collected for clinicians choosing not to participate in this study. Comparison of MERT-generated reports with events identified by concurrent direct observation would help to establish the effectiveness of this event reporting modality in detecting medication events in the hospital setting. Future studies may determine whether event reporting rates would be sustained in routine practice with the use of the handheld device.

Notwithstanding these limitations, several features of the handheld MERT including ready accessibility and ease of use suggest that this approach may prove an effective complement to existing hospital event reporting systems. In our study, both physician and nursing participants were able to effectively utilize the handheld tool to generate medication event reports on inpatient hospital units. Event reports were collected with a reasonable response burden, and reflected a spectrum of both potential and actual medication events. Future work will need to examine if deploying MERT-enabled handheld devices could translate into a sustainable increase in overall medication event detection in hospital settings, and provide substantive opportunities to improve patient safety.

Acknowledgments

The authors wish to thank Tod Kuykendall and Jason Slagle for their invaluable technical assistance in the development of the handheld computer application. We also express our gratitude to project site leaders Erin Stucky, MD, Greg Maynard, MD, and Martha Shively, RN. This project was supported by grant number 1 UC1 HS014283-01 from the Agency for Healthcare Research and Quality (AHRQ).

Conflict of Interest None disclosed.

Footnotes

This project was supported by grant number 1 UC1 HS014283-01 from the Agency for Healthcare Research and Quality (AHRQ).

References

1. Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999.
2. Flynn EA, Barker KN, Pepper GA, Bates DW, Mikeal RL. Comparison of methods for detecting medication errors in 36 hospitals and skilled-nursing facilities. Am J Health-Syst Pharm. 2002;59:436–46. [PubMed]
3. Aspden P, Wolcott J, Bootman JL, Cronenwett LR, eds. Preventing Medication Errors. Committee on Identifying and Preventing Medication Errors. Washington, DC: The National Academies Press; 2006.
4. Leape LL, Berwick DM. Five years after To Err Is Human: what have we learned? JAMA. 2005;293:2384–90. [PubMed]
5. Kopp BJ, Erstad BL, Allen ME, Theodorou AA, Priestley G. Medication errors and adverse drug events in an intensive care unit: direct observation approach for detection. Crit Care Med. 2006;34:415–25. [PubMed]
6. Weingart SN, Toth M, Eneman J, et al. Lessons from a patient partnership intervention to prevent adverse drug events. Int J Qual Health Care. 2004;16:499–507. [PubMed]
7. Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug events and potential adverse drug events. JAMA. 1995;274:29–34. [PubMed]
8. Field TS, Gurwitz JH, Harrold LR, et al. Strategies for detecting adverse drug events among older persons in the ambulatory setting. J Am Med Inform Assoc. 2004;11:492–8. [PMC free article] [PubMed]
9. Resar RK, Rozich JD, Classen D. Methodology and rationale for the measurement of harm with trigger tools. Quality and Safety in Health Care. 2003;12;ii39–45. [PMC free article] [PubMed]
10. Classen DC, Metzger J. Improving medication safety: the measurement conundrum and where to start. Int J Qual Health Care. 2003;15:i41–7. [PubMed]
11. Hardmeier B, Braunschweig S, Cavallaro M, et al. Adverse drug events caused by medication errors in medical inpatients. Swiss Med Wkly. 2004;134:664–70. [PubMed]
12. Payne TH, Savarino J, Marshall R, Hoey CT. Use of a clinical event monitor to prevent and detect medication errors. Proc AMIA Symp. 2000;640–4. [PMC free article] [PubMed]
13. Murff HJ, Patel VL, Hripcsak G, Bates DW. Detecting adverse events for patient safety research: a review of current methodologies. J Biomed Inform. 2003;36:131–43. [PubMed]
14. Murff HJ, Forster AJ, Peterson JF, Fiskio JM, Heiman HL, Bates DW. Electronically screening discharge summaries for adverse medical events. J Am Med Inform Assoc. 2003;10:339–50. [PMC free article] [PubMed]
15. Centers for Disease Control. Assessing the national electronic injury surveillance system—cooperative adverse drug event surveillance project—six sites, United States, January 1–June 15, 2004. MMWR Morb Mort Wkly Rep. 2005;54:380–3. [PubMed]
16. Bates DW, Evans RS, Murff H, Stetson PD, Pizziferri L, Hripcsak G. Detecting adverse events using information technology. J Am Med Inform Assoc. 2003;10:115–28. [PMC free article] [PubMed]
17. Nebeker JR, Barach P, Samore MH. Clarifying adverse drug events: a clinician’s guide to terminology, documentation, and reporting. Ann Intern Med. 2004;140:795–801. [PubMed]
18. Morimoto T, Gandhi TK, Seger AC, Hsieh TC, Bates DW. Adverse drug events and medication errors: detection and classification methods. Quality and Safety in Health Care. 2004;13:306–14. [PMC free article] [PubMed]
19. Taylor JA, Brownstein D, Christakis DA, et al. Use of incident reports by physicians and nurses to document medical errors in pediatric patients. Pediatrics. 2004;114:729–35. [PubMed]
20. Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies. JAMA. 1998;279:1200–5. [PubMed]
21. Capuzzo M, Nawfal I, Campi M, Valpondi V, Verri M, Alvisi R. Reporting of unintended events in an intensive care unit: comparison between staff and observer. BMC Emerg Med. 2005;5:1471–8. [PMC free article] [PubMed]
22. Tamuz M, Thomas EJ, Franchois KE. Defining and classifying medical error: lessons for patient safety reporting systems. Quality and Safety in Health Care. 2004;13:13–20. [PMC free article] [PubMed]
23. McLeod TG, Ebbert JO, Lymp JF. Survey assessment of personal digital assistant use among trainees and attending physicians. J Am Med Inform Assoc. 2003;10:605–7. [PMC free article] [PubMed]
24. Kobus DA, Amundson D, Moses JD, Rascona D, Gubler KD. A computerized medical incident reporting system for errors in the intensive care unit: initial evaluation of interrater agreement. Mil Med. 2001;166:350–3. [PubMed]
25. Bent PD, Bolsin SN, Creati BJ, Patrick AJ, Colson ME. Professional monitoring and critical incident reporting using personal digital assistants. Med J Aust. 2002;177:496–9. [PubMed]
26. Busse DK, Wright DJ. Classification and analysis of incidents in complex medical environments. Top Health Inf Manag. 2000;20:1–11. [PubMed]
27. Barger LK, Ayas NT, Cade BE, et al. Impact of extended-duration shifts on medical errors, adverse events, and attentional failures. PLoS Med. 2006;3:2440–8. [PMC free article] [PubMed]
28. Weingart SN, Ship AN, Aronson MD. Confidential clinician-reported surveillance of adverse events among medical inpatients. J Gen Intern Med. 2000;15:470–7. [PMC free article] [PubMed]
29. Weinger MB, Slagle J, Jain S, Ordonez N. Retrospective data collection and analytical techniques for patient safety studies. J Biomed Inform. 2003;36:106–19. [PubMed]
30. Weingart SN, Callanan LD, Ship AN, Aronson MD. A physician-based voluntary reporting system for adverse events and medical errors. J Gen Intern Med. 2001;16:809–14. [PMC free article] [PubMed]
31. Thomas NK. Resident burnout. JAMA. 2004;292:2880–9. [PubMed]
32. Weinger MB, Reddy SB, Slagle JM. Multiple measures of anesthesia workload during teaching and nonteaching cases. Anesth Analg. 2004;98:1419–25. [PubMed]
33. Lawton R, Parker D. Barriers to incident reporting in a healthcare system. Quality and Safety in Health Care. 2002;11:15–8. [PMC free article] [PubMed]
34. Hicks RW, Cousins DD, Williams RL. Selected medication-error data from USP's MEDMARX program for 2002. Am J Health-syst Pharm. 2004;61:993–1000. [PubMed]
35. Firth-Cozens J. Barriers to incident reporting. Quality and Safety in Health Care. 2002;11;7–8. [PMC free article] [PubMed]
36. Barach P, Small SD. Reporting and preventing medical mishaps: lessons from non-medical near miss reporting systems. BMJ. 2000;320:759–63. [PMC free article] [PubMed]

Articles from Journal of General Internal Medicine are provided here courtesy of Society of General Internal Medicine
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • Cited in Books
    Cited in Books
    PubMed Central articles cited in books
  • MedGen
    MedGen
    Related information in MedGen
  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...