• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of jamiaJAMIA - The Journal of the American Medical Informatics AssociationVisit this articleSubmit a manuscriptReceive email alertsContact usBMJ
J Am Med Inform Assoc. 2002 May-Jun; 9(3): 223–229.
PMCID: PMC344582

Clinician Use of a Palmtop Drug Reference Guide

Jeffrey M. Rothschild, MD, MPH, Thomas H. Lee, MD, MBA, Taran Bae, MD, and David W. Bates, MD, MSc

Abstract

Objective: Problems involving drug knowledge are one of the most common causes of serious medication errors. Although the information that clinicians need is often available somewhere, retrieving it expeditiously has been problematic. At the same time, clinicians are faced with an ever-expanding pharmacology knowledge base. Recently, point-of-care technology has become more widely available and more practical with the advent of handheld, or palmtop, computing. Therefore, the authors evaluated the clinical contribution of a drug database developed for the handheld computer. ePocrates Rx (formerly known as qRx; ePocrates, San Carlos, California) is a comprehensive drug information guide that is downloadable free from the Internet and designed for the Palm OS platform align="right".

Design: A seven-day online survey of 3,000 randomly selected ePocrates Rx users was conducted during March 2000.

Measurements: User technology experience, product evaluation and usage patterns, and the effects of the drug reference database on information-seeking behavior, practice efficiency, decision making, and patient care.

Results: The survey response rate was 32 percent (n=946). Physicians reported that ePocrates Rx saves time during information retrieval, is easily incorporated into their usual workflow, and improves drug-related decision making. They also felt that it reduced the rate of preventable adverse drug events.

Conclusions: Self-reported perceptions by responding clinicians endorse improved access to drug information and improved practice efficiency associated with the use of handheld devices. The clinical and practical value of using these devices in clinical settings will clearly grow further as wireless communication becomes more ubiquitous and as more applications become available.

Medical knowledge is growing exponentially,1 especially in the field of therapeutics. Pharmacologic advances have resulted in an ever-increasing variety of medications for clinicians to employ in patient care. Medications frequently have complex dosing regimens, changing indications, unique contraindications, and multiple drug–drug interactions and can cause serious adverse reactions. Because of the limitations of human memory,2,3 it has become increasingly difficult for clinicians to keep up to date on the prescribing requirements of drugs, especially recently approved drugs.4

Inquiries about drugs are one of the most frequent questions that clinicians have.5 This makes point-of -care technology—which can provide prescribing information at the bedside, on the hospital ward, and in the office examining room—especially attractive.6 Such technology has the potential to reduce the information search times of physicians, and such access may reduce the likelihood of incorrect and harmful drug-prescribing behavior.7

Adverse drug events (ADEs), or injuries due to drugs, are frequent in a variety of patient care settings. Many are preventable and may be associated with significant patient injury.8–11 Data suggest that drug knowledge deficiencies are the single most common systems cause of serious medication errors in inpatients.12 Recent publicity has encouraged public and private support for patient safety improvements, including building safer medication practices.10,11,13

Several drug reference guides designed for use on handheld, or palmtop, computers have recently become commercially available, including ePocrates Rx (formerly known as qRx; ePocrates, San Carlos, California), a comprehensive drug information application. We sought to survey physician users to assess their experience incorporating handheld electronic drug reference guides into routine patient care.

Methods

The Rx database contains pharmacologic and prescribing information for more than 1,600 of the most commonly prescribed drugs. Rx is available only for handheld computers that use the Palm OS platform. The database is downloadable from the Internet and requires 1,100 KB of memory. The download is free but requires registration. The database is updated automatically through the AutoUpdate feature that connects to the company Web site during synchronization (HotSync). Data are exchanged and updated by a centralized application server using Mobile Application Link, a communication standard for handheld computers.

The Rx database allows for drug searches by several different methods. Drugs may be sought by either generic or trade name and by therapeutic class (Figure 1[triangle]). The database provides the following drug information—dosing by indication, drug–drug interactions, adverse drug reactions (listed by severity and in descending order of frequency), pediatric dosing, contraindications, Drug Enforcement Agency schedule of controlled substances, cost and packaging information, mechanism of action, mechanism of metabolism, routes of excretion, and prescribing for special conditions, such as pregnancy and lactation. The user can create individual drug categories (e.g,. most commonly used drugs) and personal notes for any drug.

Figure 1
ePocrates Rx drug information database screens.

The survey was conducted during the first week of March 2000. A user table was created from a list of verified physicians and medical students who registered with ePocrates and downloaded the Rx database (qRx v.3.0). An e-mail solicitation was sent to a randomly selected sample of users. The solicitation requested responses within a seven-day window. Respondents were provided a link to the Web page containing the survey questionnaire. Responses were submitted online to a secure Web page.

At the time of our survey, approximately 75,000 physicians and medical students had downloaded the database. To assess relationships between practice experience of clinicians and survey responses, experience was divided into “15 years or more” and “less than 15 years” of practice.

The survey addressed five broad areas—user demographics and technology experience, product evaluation and patterns of product use, the effects of the drug reference database on information-seeking behavior, the effects on practice efficiency, and the effects on decision making and patient care. Survey question format was multiple choice or yes/no/not applicable.

Data for respondent physicians were compared with national data by specialty distribution using the one-way chi-square test. Differences in the amount of time needed for information lookup between Rx and traditional search methods were compared using the median two-sample test.

Results

The survey response rate was 32 percent (946 of 3,000), including 703 physicians (74.3 percent) and 243 medical students (25.7 percent) (Table 1[triangle]). Among the respondents, physician practice settings were approximately one fifth inpatient only, one fifth outpatient only, and three fifths providing both inpatient and outpatient care. Physician practice experience averaged 7.8 years, with 50 percent of respondents having 4 years of practice experience or less. When comparing responses of the 156 physicians (22 percent) who had been in practice for 15 or more years with those of the 547 physicians (78 percent) who had been in practice less than 15 years, we found no relationship between physician experience and any of the survey responses.

Table 1
Demographics of Survey Respondents

Among physicians in practice, the most common specialties were general internal medicine, family practice, and medical and surgical subspecialists (Table 2[triangle]). The distribution of respondent physician specialties was not significantly different from the distribution of recent national physician specialties data (p=0.09).14

Table 2
Comparison of Specialties for Physician Respondents and Active U.S. Physicians*

The users of Rx were surveyed about their experience with handheld computers and palmtop technology as well as Internet access. The overwhelming majority of respondents had Internet access at both home and work (Table 1[triangle]). At the time of the survey, 25.7 percent of physicians were communicating with patients via e-mail. Almost a quarter of respondents had high-speed Internet connections (232 of 946, or 24.5 percent) including corporate local-area network (112 respondents), cable modem (57 respondents), digital subscriber line (24 respondents), and integrated service digital network (23 respondents). Duration of ownership of a handheld computer was more than 6 months for two thirds of respondents. Duration of experience using the Rx drug information database was 3 months or less for 57.2 percent of respondents.

The mean frequency of daily Rx use among survey respondents was 0 or 1 time a day, 18.1 percent; 1 or 2 times a day, 20.8 percent; 2 to 5 times a day, 35.9 percent; and more than 5 times a day, 25.2 percent. The perceived time needed to looking up or find drug information using the handheld drug reference guide was significantly less than with previously used or traditional methods (p=0.0001) (Figure 2[triangle]). Traditional methods used to find drug information include Physicians' Desk Reference, medical textbooks, and informal consultations with colleagues.

Figure 2
Comparison of time it took clinicians to look up drug information using ePocrates Rx (hatched columns) and traditional means (e.g., Physicians' Desk Reference, medical textbooks, and informal consultations with colleagues). Times to look up information ...

The speed with which Rx found drug information was rated, on average, as 10 sec or less by 63.3 percent of users and as 1 min or more by only 1.5 percent of users. Traditional methods of information retrieval were judged to require 10 sec or less for 8.1 percent of users and 1 min or more for 53.6 percent of users.

Regarding how often drug information queries were satisfactorily answered with use of Rx, 25.3 percent of users were satisfied in 95 percent or more of their queries, 31.1 percent in 90 to 94 percent of queries, 30.7 percent in 75 to 89 percent of queries, and 13 percent in less than 75 percent of queries. Drug information features were ranked as most to least useful on a 1 through 5 scale. The most useful feature was adult dosing (mean, 1.8), and the least useful was pediatric dosing (3.3). Contraindications, drug–drug interactions and adverse reactions received intermediate usefulness ratings (2.5 to 2.6).

Practice efficiency was rated as improved with use of the handheld drug information database. These improvements were present in both outpatient (86.3 percent) and inpatient (87.1 percent) settings (Table 3[triangle]). The perceived average time saved per encounter with use of Rx was more than 1 min for 29.6 percent of respondents.

Table 3
Effects of Rx on Clinician Behavior, Practice Efficiency, and Patient Care

The majority of respondents (78.9 percent) found that using Rx increased self-reported drug knowledge and contributed to improved drug-related decision making (Table 3[triangle]). The majority of users (72.3 percent) also indicated that at least one clinical decision per week was favorably affected by use of Rx.

Most surveyed physicians (83 percent) found that patient care improved, because Rx enabled them to better inform patients about their medications. Slightly more than half the respondents (54 percent) were of the opinion that their patients were more satisfied with their care as a result of Rx use. Nearly two thirds of clinicians (63.1 percent) supported a statement that Rx contributes to a reduction in potential adverse drug events (ADEs). With regard to the frequency of ADE reduction, half (49.8 percent) of the respondents estimated that using Rx has prevented, on average, at least one ADE a week (Table 3[triangle]).

Discussion

While physicians have generally been considered relatively slow to embrace new technologies,15,16 they may be willing to accept a new technology if it represents a clear breakthrough. In this study, we identified a group of “early adopters.”17 Rx was publicly released for general use in November 1999. Our online survey took place less than 5 months later. The rapid popularity of handheld computers, and this software in particular, is evidence that physicians will rapidly accept technology that they perceive that it improves the quality or efficiency of their practice.

A recent survey of internists, by the American College of Physicians–American Society of Internal Medicine, found that nearly half their respondents use handheld computers, including 80 percent of users who access drug information on handheld computers.18 As of July 2001, 300,000 individuals, including 150,000 physicians and 33,000 medical students, had registered and downloaded the Rx drug reference guide. This represents approximately one fifth of physicians and half of the medical students in the United States.

In this survey, we found that the handheld drug information database Rx could satisfy many of the drug information needs of clinicians. This database provides point-of-care information that can be accessed fast, from any location, and can fit into existing clinician practice workflow. Thus, users judged that Rx improved practice efficiency by reducing the time needed to find drug information. In addition, users considered that both their drug knowledge base and drug-related decisions improved with Rx. Although we did not assess this directly, users perceived that they made fewer medication errors when using this technology.

The prescribing of medication is an important part of medical care in both inpatient and outpatient settings and is the most common type of therapeutic intervention in patient treatment.19 An estimated 3 billion prescriptions are written annually in the United States.20 Physicians and other health care providers are continually challenged by the increasing information demands required to safely prescribe and monitor medication effects.9 In prior studies of hospitalized adult medical-surgical patients, the medication delivery system stage most frequently responsible for preventable ADEs is the clinician ordering stage.8 Ordering errors are also the most common cause of medication errors and ADEs in intensive care units,10 pediatric inpatients units,21 nursing homes,22 and outpatient settings.23 In one study in which serious medication errors in inpatients were assessed, dissemination of drug knowledge was the leading category of system failures underlying the errors, accounting for 29 percent of errors.12

The informational needs of clinicians, including information pertaining to medication prescribing, are often unmet during patient encounters.6,24,25 In an early outpatient study,26 access to information sources was the most important factor determining which data source was chosen when information needs arose. In an observational study of both inpatient and outpatient encounters in an academic setting, an average of five clinical questions were raised for each patient discussed, with 16 percent of questions relating to drug therapy.27 A more recent observational study found that the most common category of questions (19 percent) among primary care physicians concerned drug prescribing.5 This study also found that the mean time spent pursuing answers to prescribing information was 74 sec.

Clinicians in our survey most frequently needed 1 to 5 min to find drug information when using traditional search methods, but they reported needing less than 20 sec when using Rx. Unlike the observational study, our study was based on self-reported survey data rather than observational data. Our study was limited to the respondents' retrospective perceptions of their mean times to look up information with and without the handheld drug information database.

Several features available on the version of Rx current at the time of this study were not included in the earlier version (v.3.0) used at the time of the survey. The most important new feature, MultiCheck, permits up to 30 medications to be simultaneously cross-checked for drug–drug interactions. DocAlerts is an application that permits notices to be sent through AutoUpdate and viewed at the user's discretion. Notices include information about new drug releases, drug updates, and other relevant public health topics (e.g., diagnostic and treatment guidelines for anthrax). In addition, ePocrates has developed an antimicrobial therapeutics database (qID) that requires a separate download. This database provides infectious disease drug recommendations organized by organ involvement or the suspected organism(s). qID links with Rx if more specific drug prescribing information is needed.

Handheld computers have been reported to improve several processes associated with patient care and will no doubt have additional uses.28 Examples include improved documentation and reporting of resident physicians' procedural experience,29 increased physician adherence to outpatient asthma guideline recommendations,30 assistance for nurses in eliciting patient preferences at the bedside,31 identification of eligible patients for cancer trials,32 and wireless decision support and alerts for patient test results for critically ill patients.33,34 Wireless electronic prescribing can also be done on handheld computers.35

This study has several limitations. The users and respondents represented a group of early adopters who may have been biased toward favorably reviewing newer technologies. With half of the physicians in practice for 4 years or less, our respondents are less experienced and probably younger than average physicians. On the other hand, our sample of physicians (and medical students) may better foresee the future degree of acceptability for similar computer-based technologies.

Conclusions

Drug information databases that are available on handheld devices, such as palmtop computers, improve perceptions about practice efficiency, improve drug knowledge, and may decrease medication errors. Wireless communication will undoubtedly further enhance point-of-care capabilities. Randomized controlled studies are needed to determine whether portable, point-of-care decision support tools are effective in reducing the rate of serious medication errors or ADEs. In addition, studies are needed to compare computerized order entry and decision support using wireless handheld computers with current desktop computer technology; it is likely that these technologies will be complementary.

Acknowledgments

The authors thank Rena Yamamoto, Jan Horsky, and Elisabeth Burdick for valuable research assistance.

Notes

This work was presented as a poster at the AMIA Annual Symposium, Nov 4–8, 2000, in Los Angeles, California.

References

1. Wyatt J. Use and sources of medical knowledge. Lancet. 1991;338:1368–73. [PubMed]
2. McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med. 1976;295:1351–5. [PubMed]
3. Weed LL. New connections between medical knowledge and patient care. BMJ. 1997;315:231–5. [PMC free article] [PubMed]
4. Cohen JS. Dose discrepancies between the Physicians' Desk Reference and the medical literature, and their possible role in the high incidence of dose-related adverse drug events. Arch Intern Med. 2001;161:957–64. [PubMed]
5. Ely JW, Osheroff JA, Ebell MH, et al. Analysis of questions asked by family doctors regarding patient care. BMJ. 1999;319: 358–61. [PMC free article] [PubMed]
6. Weed LL. Clinical judgment revisited. Methods Inf Med. 1999;38:279–86. [PubMed]
7. Bates DW. Using information technology to reduce rates of medication errors in hospitals. BMJ. 2000;320:788–91. [PMC free article] [PubMed]
8. Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug events and potential adverse drug events: implications for prevention. ADE Prevention Study Group. JAMA. 1995;274:29–34. [PubMed]
9. Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies. JAMA. 1998;279:1200–5. [PubMed]
10. Cullen DJ, Sweitzer BJ, Bates DW, Burdick E, Edmondson A, Leape LL. Preventable adverse drug events in hospitalized patients: a comparative study of intensive care units and general care units. Crit Care Med. 1997;25:1289–97. [PubMed]
11. Classen DC, Pestotnik SL, Evans RS, Lloyd JF, Burke JP. Adverse drug events in hospitalized patients: excess length of stay, extra costs, and attributable mortality. JAMA. 1997;277: 301–6. [PubMed]
12. Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA. 1995;274:35–43. [PubMed]
13. Kohn LT, Corrigan JM, Donaldson MS (eds). To Err Is Human: Building a Safer Health System. Institute of Medicine, Committee on Quality of Health Care in America. Washington, DC: National Academy Press, 2000.
14. Pasko T, Seidman B, Birkhead S. Physician Characteristics and Distribution in the United States: 2001–2002 Chicago, Ill.: American Medical Association, 2001.
15. Massaro TA. Introducing physician order entry at a major academic medical center, part 2: Impact on medical education. Acad Med. 1993;68:25–30. [PubMed]
16. Sittig DF, Stead WW. Computer-based physician order entry: the state of the art. J Am Med Inform Assoc. 1994;1:108–23. [PMC free article] [PubMed]
17. Rogers EM. Diffusion of Innovations. 3rd ed. New York: The Free Press, 1983.
18. ACP-ASIM survey finds nearly half of U.S. members use handheld computers. ACP-ASIM Online. Oct 25, 2001. Available at: http://www.acponline.org/college/pressroom/handheld_survey.htm. Accessed Nov 13, 2001.
19. Lesar TS, Lomaestro BM, Pohl H. Medication-prescribing errors in a teaching hospital: a 9-year experience. Arch Intern Med. 1997;157:1569–76. [PubMed]
20. Schiff GD, Rucker TD. Computerized prescribing: building the electronic infrastructure for better medication usage. JAMA. 1998;279:1024–9. [PubMed]
21. Kaushal R, Bates DW, Landrigan C, et al. Medication errors and adverse drug events in pediatric inpatients. JAMA. 2001;285:2114–20. [PubMed]
22. Gurwitz JH, Field TS, Avorn J, et al. Incidence and preventability of adverse drug events in nursing homes. Am J Med. 2000;109:87–94. [PubMed]
23. Gandhi TK, Weingart SN, Seger A, et al. Impact of basic computerized prescribing on outpatient medication errors and adverse drug events. J Gen Intern Med. 2001;16(suppl 1):908.
24. Smith R. What clinical information do doctors need? BMJ. 1996;313:1062–8. [PMC free article] [PubMed]
25. Ebell M. Information at the point of care: answering clinical questions. J Am Board Fam Pract. 1999;12:225–35. [PubMed]
26. Covell DG, Uman GC, Manning PR. Information needs in office practice: Are they being met? Ann Intern Med. 1985;103:596–9. [PubMed]
27. Osheroff JA, Forsythe DE, Buchanan BG, Bankowitz RA, Blumenfeld BH, Miller RA. Physicians' information needs: analysis of questions posed during clinical teaching. Ann Intern Med. 1991;114:576–81. [PubMed]
28. Sittig DF, Jimison HB, Hazlehurst BL, et al. Techniques for identifying the applicability of new information management technologies in the clinical setting: an example focusing on handheld computers. Proc AMIA Annu Symp. 2000:804–8. [PMC free article] [PubMed]
29. Garvin R, Otto F, McRae D. Using handheld computers to document family practice resident procedure experience. Fam Med. 2000;32:115–8. [PubMed]
30. Shiffman RN, Freudigman M, Brandt CA, Liaw Y, Navedo DD. A guideline implementation system using handheld computers for office management of asthma: effects on adherence and patient outcomes. Pediatrics. 2000;105:767–73. [PubMed]
31. Ruland CM. Clinicians' use of a palm-top based system to elicit patient preferences at the bedside: a feasible technique to improve patient outcomes. Proc AMIA Annu Symp. 2000:739–43. [PMC free article] [PubMed]
32. Breitfeld PP, Weisburd M, Overhage JM, Sledge GJ, Tierney WM. Pilot study of a point-of-use decision support tool for cancer clinical trials eligibility. J Am Med Inform Assoc. 1999;6:466–77. [PMC free article] [PubMed]
33. Shabot M, LoBue M. Real-time wireless decision support alerts on a palmtop PDA. Proc Annu Symp Comput Appl Med Care. 1995:174–7. [PMC free article] [PubMed]
34. Shabot M, LoBue M, Chen J. Wireless clinical alerts for critical medication, laboratory and physiologic data. Proceedings of the 33rd Hawaii International Conference on System Sciences, Jan 4–7, 2000 [CD-ROM]. Piscataway, NJ: Institute of Electrical and Electronics Engineers, 2000.
35. Walsh PJ. E-pharmacy systems: prescription and medication fulfillment come of age. MD Comput. 2000;17:45–8. [PubMed]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of American Medical Informatics Association
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • Cited in Books
    Cited in Books
    PubMed Central articles cited in books
  • PubMed
    PubMed
    PubMed citations for these articles
  • Substance
    Substance
    PubChem Substance links

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...