Abstract

The purpose of this study was to develop and pilot-test an instrument that determines the relationship between perceptions, intended use, and actual use of the personal digital assistant (PDA) by primary care physicians for drug information access and prescribing. The instrument studied addressed human factor limitations, attitudes toward learning to use the PDA, and fear of losing either confidential information or the PDA itself. The study instrument was intended to differentiate between applications a physician will or will not incorporate into daily practice and to predict which physicians are more likely to successfully adopt use of a PDA. A 63-item survey was administered at baseline to 78 primary care physicians, randomized into control and intervention groups. The physicians in the intervention group were provided a PDA with electronic prescribing software and drug information software, training in the use of the device and the software, and an infrared printer to write prescriptions. They were then asked to prescribe using the PDA or by hand, as they preferred. The control group physicians continued traditional handwritten prescribing and accessed their usual drug information sources. The survey was readministered to both groups after they had completed 500 prescriptions. Postintervention data did not support ready acceptance of the PDA as a prescribing device; however, the physicians did find the PDA useful for accessing drug information at the point of care. Potential factors that predict physician use of the PDA were learning-related attitudes, human factor limitations, and concerns about loss of confidential and critical information.

Introduction

Personal digital assistants (PDAs) have substantial potential for improving the health-related safety, quality, and efficiency of care by physicians in office-based practices. These devices have already demonstrated improved accuracy of patient identification in the hospital setting and access to patient-specific medical information for use in diagnosis and care decisions. ^{1, 2} Yet, there are few references relating to adoption of information technology by family and general practitioners in small office-based practice settings. ³

Early experiences with this technology application suggest that the PDA's convenience, ease of use, and portability within the physician's office solves several logistical barriers associated with fixed, desktop computer systems used for similar purposes. ⁴ Many respected, credible, electronic information sources are now available for these devices, and trends suggest that all published expert information resources will be accessible electronically in the future. ⁵ The PDA also provides a convenient means of generating a legible printed prescription at the point of care using local wireless transmission to printers.

Accessing evidence-based information to improve medical decisionmaking at the point of care and improving prescription legibility are critical steps to improving medication-related safety. However, it is known that new technologies create new demands on users. Even a device like the PDA, which features low-level technology, will meet barriers to acceptance and use. Human factor principles must be addressed to account for the human-machine interface. ^{6, 7} Introduction of PDA technology may also cause a shift in workload to the physician, placing further strain in the work environment. This shift occurs at a time when office efficiency remains a medical office business goal. Therefore, technology acceptance is not a foregone conclusion.

Over the past 30 years, several theories have been offered to explain the acceptance of technology or, more specifically, computer usage behavior. The Technology Acceptance Model (TAM) is a recent theory developed by Davis et al. ⁸ The TAM is an adaptation of the widely studied model known as the Theory of Reasoned Action (TRA), ^{9, 10} a general theory that postulates that actual performance of a task or function (behavior) is based on an individual's behavioral intention (BI). Behavioral intention is determined by both the individual's attitude (A) and the subjective norm (SN) concerning the behavior in question (Figure 1). Attitude is a person's positive or negative feelings about performing the target behavior. Subjective norm is the person's perception of what other people think. ⁹ Behavioral intention measures the strength of one's intention to perform a specified behavior. According to TRA, attitude toward a behavior is determined by beliefs about the consequences of performing the behavior multiplied by the evaluation of those consequences. Subjective norm is determined by normative beliefs multiplied by a motivation to comply with perceived expectations.

Figure 1

Schematic representation of the Theory of Reasoned Action and the Technology Acceptance Model.

The Technology Acceptance Model is based on TRA and was developed to specifically explain computer usage. ⁸ One similarity between TAM and TRA is that both models postulate that behavioral intention (BI) is determined by attitude (A). However, TAM adds a perceived usefulness (PU) construct and a perceived ease of use (PEOU) construct. Perceived usefulness is the prospective user's subjective probability that using a specific application system will increase job performance within an organizational context. Perceived ease of use refers to the degree to which the prospective user expects the target technology to be free of effort. TAM postulates that BI is determined by both A and PU (Figure 1). Unlike the TRA, TAM does not include SN as a determinant of BI. Social norm was not included in TAM because of uncertain theoretical and psychometric status. According to TAM, A is determined by PU and PEOU. Perceived usefulness and perceived ease of use have both been theorized to be determined by external variables. In the case of the PDA, relevant external variables that affect ease of use include system features such as drop-down menus, use of graffiti abbreviations, and the touch screen stylus. An example of perceived usefulness beliefs would be the ability to compare two drug information sources that are equally easy to operate on a PDA. This comparison may indicate that one electronic drug information source is used more frequently than the other simply because it gives more reliable and accurate information. Perceived usefulness beliefs are likely influenced by learning based on feedback.

Although these two theoretical models have made significant contributions toward behavior theory, TAM has not yet been tested in the application setting to determine how behavioral intention relates to actual technology use. This project studies how both the constructs that determine behavioral intentions and behavioral intentions themselves relate to actual use of PDAs by physicians for drug information and prescribing in office-based practices. This project also recognizes the contribution of learning-related attitudes and of specific external variables such as psychomotor skills that may influence behavior intentions and actual usage of hand-held technology.

The learning of new technology (the PDA) and its applications (prescription writing and drug information) by physicians in office-based practices is the specific focus of the current study. The investigators viewed primary care physicians as adult learners who were making decisions about the acceptance of PDA technology into every day practice.

Adult learners have been shown to invest new learning selectively into efforts that they perceive to be of substantive value. There is little evidence to help us understand the variables that influence adult learners regarding technology use. Several variables have been studied in students, however, as possible predictors of application, learning, and achievement with computers in higher education. Variables studied in relationship to student attitudes toward computers in education have included locus of control, gender, computer aptitude, previous computer experience, prior attitudes toward computers, beliefs, and perceived usefulness of computers. ^{11 – 14} No published studies have been shown to predict computer technology adoption in relationship to these or other variables.

Purpose

The purpose of this study was to develop and pilot-test a survey instrument that determines how physicians' perceived usefulness of the PDA, perceived ease of use of the device, attitudes toward using PDAs, and fear of losing either confidential information or the PDA itself relate to intended and actual use of the PDA for accessing drug information and prescribing in daily medical practice. Ease of use in the context of human factor limitations, attitudes toward PDA use as they relate to learning technology, and fear of loss of confidential information or the PDA itself have not been previously studied within the general theory of TAM. Our aims were to develop an instrument for use with physicians (both new and experienced PDA users) that would (1) predict which applications a physician will or will not voluntarily choose to incorporate into daily practice, and (2) determine those physicians who would be more likely to successfully adopt a PDA for drug information and prescribing.

Methods

Results

Instrument

Scale reliability was established for seven constructs (alphas ranged from 0.84 to 0.94), five of which held constant pre- to postsurvey administration (Table 1). The number of items in each subscale ranged from 2 to 5, with a total of 21 items for the entire instrument. The five subscales that held constant pre and post are in the perceived usefulness construct (beliefs that PDA use will generate more accurate and complete prescriptions and beliefs that PDAs increase efficiency), the attitudes toward use construct (emotional reaction to computers and learning-related attitudes toward computers), and the external factor (less fear of loss of patient information with PDA use). These results indicate that physicians' perceptions about the PDA's usefulness in preparing accurate and complete prescriptions and in increasing efficiency held true after gaining PDA-use experience. Similarly, perceived attitudes toward PDA use and fear of loss of patient information held true after gaining PDA-use experience.

Table 1

Final pre- and postsurvey constructs representing all cases.

Two constructs formed that did not hold constant pre- to postintervention. One construct formed preintervention related to usefulness of PDAs. It is labeled “beliefs that PDAs save time.” This unique construct formed around the expectations that using hand-held devices would save time and not slow down the physicians in their work. This construct did not hold true after PDA-use experience, indicating that actual PDA use did not result in meeting the time savings expected by physicians. Three items grouped to form the construct, with a Cronbach's coefficient alpha of 0.84. The other construct formed postintervention related to ease of use. It is labeled “no human factors limitations with computing devices.” Four items grouped to form the construct, with a Cronbach's coefficient alpha of 0.87. This construct formed after PDA-use experience, indicating that physicians did not anticipate experiencing limitations using the PDA device, but actually did. It is noteworthy that a construct did not form around the notion that they expected to use the PDA as a drug information source in daily practice.

Behavioral intentions and actual use outcomes

A single construct formed as a behavioral intention from the baseline survey (Figure 2). This is outcome 1—expectation that the PDA is useful for writing and printing prescriptions (Table 2). Two postintervention survey outcomes formed, one as a behavioral intention and one as actual use. Outcome 2, expectation and attitude about PDA use in every day medical practice, indicated that respondents like the idea of working with the PDA and that the PDA is useful to their work. The construct about expecting to use the PDA daily formed as a result of actual PDA use. However, the additional outcome 3 that formed provided further explanation. This construct indicated that physicians who are PDA users are confident in their use of the PDA, find it easy to use, and believe it is useful as a drug information resource. The preintervention survey construct about writing and printing prescriptions with the PDA as a daily expectation did not remain intact. Postanalysis indicated that physicians expect to use a PDA in everyday practice only as a drug information source, not as a prescription writing and printing device.

Figure 2

Outcome 1—path analysis for preintervention data, all physicians.

Table 2

Outcome constructs representing all cases: behavioral intention (presurvey) and actual use (postsurvey).

Figure 3 displays the relationships observed in the post-survey construct model. The physicians' expectations/attitudes about PDA use in everyday practice were positively impacted by a positive emotional reaction toward computers, less fear of patient information loss with a PDA, and a positive belief about the impact of PDAs on the accuracy and completeness of their prescription writing. Learning-related attitudes and emotional reaction toward computers were highly correlated compared to preintervention data. Human factors limitations also emerged as important after experiencing use of the PDA. Physicians' confidence in PDAs' ability to increase work efficiency positively impacted the beliefs that physicians have about the impact of PDA use on the accuracy and completeness of prescriptions and their positive learning-related attitudes toward computers. Clinicians who did not believe that the PDA brought value did not expect to adopt the PDA for daily use.

Figure 3

Path analysis for postintervention data, all physicians.

While Figure 2 displays the relationships that formed between the baseline survey constructs and behavioral intention of physicians prior to PDA use, Figure 3 shows the relationships that formed between the postintervention survey constructs, behavioral intention outcome, and actual use outcome.

Discussion

A universal expectation of physicians suggesting that hand-held devices would save time was revealed, as evidenced by the formation of this preintervention construct. This construct did not remain intact after users gained experience with the PDA, suggesting that these expectations of saving time were not met. It is likely that when introducing PDA technologies to new users, there will be an expectation for efficiency that may not be retained after having the experience of using the technology.

Learning-related attitudes and emotional reactions toward computers emerged as important to actual PDA users. Adult learners expect their educational experiences to be applicable to some purpose or function they are responsible for. The factors that emerged in this study are consistent with this notion. However, respondent general emotional reaction to computers emerged as an overarching factor—a factor that suggests that education and training for users must respond to this affective domain. Learners need to be taught in ways that introduce fun and create joyful experiences while learning this technology. Specific approaches for overcoming negative emotional reactions to computers are needed.

Three components emerged related to PDAs: beliefs that PDAs will generate more accurate and complete prescriptions, beliefs that the use of PDAs will improve patient care efficiency, and reduced fears of losing patient information as well as the device itself. Beliefs about the potential usefulness of the PDA for improving prescribing and performance were generally positive. Physicians believed that improved accuracy and completeness of prescription information are related to use of a PDA. However, physicians' experiences with using the PDA had a negative influence on their voluntary use of the PDA to prescribe and print prescriptions.

Despite this, beliefs about the PDA's impact on efficiency were positive. Physicians believed there was a relationship between using a PDA and their ability to see more patients each day or to deliver more care per visit. In general, they thought the PDA was more likely to improve the quality of general patient care. Training may need to focus on emphasizing reasonable expectations about the PDA's impact.

Physicians who expected to use the PDA in daily practice were consistently less concerned about the loss of critical and confidential patient information resulting from losing or misplacing their PDA when compared to those who did not expect to use it daily. Teaching good practice in device storage and management may be a simple but important strategy to facilitate adoption of PDA use.

The postintervention formation of the construct labeled “no human factors limitations with computing devices” suggests that some individuals will not use a PDA in daily practice if there are physical and sensory-related barriers to use. Only a small percentage of the general physician population will likely be hindered by physical barriers/ability-related barriers. Preliminary results indicated that a relationship between these human factors and lack of adoption in daily use does exist. Practical methods to compensate for these barriers need to be identified at the outset in order to address the needs of these few individuals during training.

Finally, there is a shift in perceptions about usefulness, ease of use, and attitudes toward PDAs as a result of the experience of actual use. Expectations about the usefulness and ease of use of technology can be inaccurate prior to use. Prior to the intervention, physicians expected prescribing efficiency to be the primary usefulness of the PDA. After experiencing PDA use, access to drug information at the point of care became the primary usefulness of the PDA. Education and training should focus on correcting inaccurate perceptions to improve the rate and extent of technology acceptance.

Limitations

The limited sample size precludes the investigators from establishing construct validity. A minimum sample size required pooling data in order to conduct the path analyses. A larger sample to test this instrument is needed to better define the relationships. However, the subject selection and study design minimized sources of internal invalidity related to history, maturation, testing, and instrumentation.

Future research

The instrument has the potential for predictive validity. It may be useful as a tool to discern who is more likely to adapt to hand-held technology prior to implementing a technology-based program. Research should be conducted on the effectiveness of technology teaching programs that incorporate the constructs formed as a result of this study. In addition, the retention of the human factor-based items, learning attitudes, and fear of loss of confidential patient information should occur in larger samples in future research to determine if there is a predictive association.

Conclusion

Our aim was to describe the initial development and pilot-testing of an instrument that would improve our ability to differentiate between applications physicians will or will not voluntarily choose to incorporate into daily practice. Both new and experienced hand-held device users comprised the sample of physicians. The instrument distinguished between the utility of the PDA for prescribing and the utility of the PDA for acquiring drug information. Three constructs of importance as potential factors in physician adaptation to use of the PDA emerged from this work: learning-related attitudes, human factor limitations, and concerns about confidential and critical information loss. These findings suggest specific content and skill areas to include when teaching physicians how to use PDAs and to gain their acceptance of this technology.

Acknowledgments

This study was supported in part by Grant No. R18HS11808-1 from the Agency for Healthcare Research and Quality.

References

1.

Overdyk FJ, Haynes GR, Arvanitis PJ. Patient-borne memory device facilitates “point of care” data access. MD Comput. 1999 May–Jun;16(3):60–3. [PubMed: 10439604]

2.

Pettis KS, Savona MR, Leibrandt PN. et al. Evaluation of the efficacy of hand held computer screens for cardiologists' interpretations of 12-lead electrocardiograms. Am Heart J. 1999 Oct;138(4 Pt 1):765–70. [PubMed: 10502225]

3.

Dixon DR, Stewart M. Exploring information technology adoption by family physicians: survey instrument valuation. Proceedings of AMIA Symposium; 2000 Nov 4–8; Los Angeles, CA. pp. 185–9. [PMC free article: PMC2243739] [PubMed: 11079870]

4.

Miller RH, Hillman JM, Given RS. Physician use of IT: results from the Deloitte research survey. J Healthcare Inform Management. 2004;18(1):72–9. [PubMed: 14971083]

5.

Arnone G, Bianchi A, Della-Pietra B. et al. Easy Medic: an Internet application for the general practitioner. J Telemed Telecare. 1998;4(Suppl 1):93–4. [PubMed: 9640752]

6.

Cook R, Woods D. Operating at the sharp end: the complexity of human errors. In: Bogner MS, editor. Human error in medicine. Hillsdale, NJ: Lawrence Erlbaum Associates; 1994.

7.

Van Cott H. Human errors: their causes and reductions. In: Bogner MS, editor. Human error in medicine. Hillsdale, NJ: Lawrence Erlbaum Associates; 1994.

8.

Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models. Management Science. 1989;35(8):982–1003.

9.

Fishbein M, Ajzen I. Belief, attitude, intention and behavior: an introduction to theory and research. Reading, MA: Addison-Wesley; 1975.

10.

Ajzen I, Fishbein M. Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice-Hall; 1980.

11.

Lorenzi NM, Riley RT, editors. Organizational aspects of health informatics: managing technological change. New York: Springer-Verlag; 1995.

12.

Sittig DF, Jimison HB, Hazlehurst BL, et al. Techniques for identifying the applicability of new information management technologies in the clinical setting: an example focusing on handheld computers. Proceedings of AMIA Symposium; 2000 Nov 4–8; Los Angeles, CA. pp. 804–8. [PMC free article: PMC2243837] [PubMed: 11079995]

13.

Francis LJ. Measuring attitude toward computers among undergraduate college students: the affective domain. Computers Educ. 1993;20(3):251–5.

14.

Griswold PA. Some determinants of computer awareness among education majors. Ass Educ Data Systems J. 1983;16:92–103.

15.

Galt KA. Medication errors in primary care. Top Health Inform Manage. 2002;23(2):34–46.

16.

Galt KA, Rich EC, Young WW. et al. Impact of hand-held technologies on medication errors in primary care. Top Health Inform Manage. 2002;23(2):71–81.

17.

U.S. Census Bureau. Glossary. Available at: http://www​.census.gov/main/www/glossary​.html. (Accessed Oct 11, 2002.)

18.

Bergner M, Bobbitt RA, Carter WB, et al. The sickness impact profile: development and final revision of a health status measure. Med Care 1981 Aug 19;(18):787–805. [PubMed: 7278416]

19.

Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 1989 Sept:319–40.

20.

Brown SH. Changes in physician's computer anxiety and attitudes related to clinical information system use. J Am Med Informatics Assoc. 1994;1(5):381–94. [PMC free article: PMC116220] [PubMed: 7850562]

21.

Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models. Management Science. 1989;35(8):982–1003.

22.

Communication from Veterans Affairs Patient Safety Center, 2002.

23.

Center for Devices and Radiologic Health. http://www​.fda.gov/cdrh​/useerror/you_choose_checklist.html. (Accessed Oct 2002.)

24.

http://www​.fda.gov/cdrh​/humanfac/hfbrochure.html. (Accessed Nov 19, 2002.)

25.

Dambrot FJ, Watkins-Malek MA, Silling SM. et al. Correlates of sex differences in attitudes toward and involvement with computers. J Vocatl Behav. 1985;27:71–86.

26.

Koohang AA. A study of the attitudes of pre-service teachers toward the use of computers. Educl Communs Technol J. 1987;35:145–9.

27.

Byrd DM, Koohang AA. A professional development question: is computer experience associated with subjects' attitudes toward the perceived usefulness of computers? J Res Computing Educ. 1989;21:401–10.

28.

Todman J, File P. A scale for children's attitudes to computers. School Psychology International. 1990;11:71–5.

29.

SPSS for Windows, version 11.0; SPSS Inc., 233 S. Wacker Drive, 11th floor, Chicago, IL.