Exploring clinician adoption of a novel evidence request feature in an electronic medical record system

Objective: The research evaluated strategies for facilitating physician adoption of an evidence-based medicine literature request feature recently integrated into an existing electronic medical record (EMR) system. Methods: This prospective study explored use of the service by 137 primary care physicians by using service usage statistics and focus group and survey components. The frequency of physicians’ requests for literature via the EMR during a 10-month period was examined to explore the impact of several enhanced communication strategies launched mid-way through the observation period. A focus group and a 25-item survey explored physicians’ experiences with the service. Results: There was no detectable difference in the proportion of physicians utilizing the service after the communication (11% each time period, P (cid:2) 1.0, McNemar’s test). Forty-eight physicians (35%) the Respondents used the service indicated that information provided the service was highly relevant to clinical practice (mean rating 4.6, scale 1 ‘‘not relevant’’–5 ‘‘highly relevant’’), and most (n (cid:2) 15) reported sharing the information with colleagues. Conclusion: use librarian-summarized the the potential librarian

• The communication strategies implemented during the study did not induce a noticeable increase in questions from physicians through the electronic medical record system. • Surveyed physicians were generally satisfied with the evidence-based medicine literature request service and noted several resulting changes in clinical practice associated with librarian-provided evidence. • Survey respondents viewed the monthly ''evidence alert'' newsletter, one of the communication strategies, as particularly effective, and it has become an integrated facet of the service.

Implications
• Librarians may contribute significantly to effective patient care by providing evidence to support the clinical decision-making process. • The study's findings emphasize the complexity of facilitating the adoption of services providing evidence for clinical practice.

INTRODUCTION
Practicing clinicians have an obvious imperative to manage and apply information effectively. However, physicians attempting to identify research relevant to their clinical practice are faced with a number of potential roadblocks. These issues may be contributing to the large proportion of questions encountered during clinical practice that go unpursued [1][2][3][4][5][6][7][8]. In addition, lack of time is a clear concern for the busy practicing clinician and is one of the most often cited barriers to the effective practice of evidence-based medicine (EBM) [1,[9][10][11][12][13][14][15][16][17][18][19][20]. Given the time constraints of clinical practice, it is not surprising that clinicians are not generally able to consider the multiple sources of evidence that an answer to a complex question may necessitate. A growing number of information resources provide ready access to evidence-based syntheses of the clinical research literature (e.g., UpToDate, MDConsult, Cochrane Database of Systematic Reviews). These resources attempt to assist clinicians with quick access to readily available answers. However, these resources are less useful in addressing more complex queries, such as the conflicting reports in the literature regarding the relative merit of multiple diagnostic or therapeutic strategies [18,[21][22][23][24]. Numerous investigations continue to demonstrate significant gaps between research evidence and the clinical care actually provided in a broad range of clinical specialties [25][26][27][28][29][30][31][32], underscoring the need for additional, systematic ways to integrate evidence into clinical practice.
Given these challenges, considerable attention is be-ing devoted to developing technical and human systems that incorporate the best evidence into clinical practice. Recent interest has focused on the potential role of expert information professionals who can work with clinicians to address unanswered information needs. These papers describe efforts to train ''clinical informationists,'' professional members of the health care team who inhabit the intersection between information and clinical care and act as expert information providers to identify and fulfill the complex information needs of the team [33][34][35][36][37]. This emerging professional is skilled in identifying, assessing, and synthesizing research relevant to specific clinical contexts. At the Vanderbilt University Medical Center (VUMC), the Eskind Biomedical Library (EBL) developed a service providing clinical information, called the Clinical Informatics Consult Service (CICS), as a local strategy to address clinician needs for evidence to support patient care [38,39]. The main goal of the library's evidence-based services has been to provide clinicians with the highest-quality and most relevant research evidence to support effective clinical decision making and evidence-based practice.
While the service initially focused primarily on librarian consultations with clinicians during in-patient morning rounds, library staff also recognized a need to adapt this model to provide service to VUMC's outpatient clinics, where attending physicians and housestaff care for patients in individual examination rooms rather than through the group ''bedside rounds'' of the inpatient setting. In late 2003, the library began using VUMC's electronic medical record (EMR) system, StarPanel [40,41], as an entry point into the outpatient arena [42]. The StarPanel EMR system includes a secure messaging feature that has been successfully integrated throughout the outpatient clinics at VUMC and is heavily used in the clinic workflow. The library's EBM Literature Request service permits clinicians to use this messaging feature to consult with EBL librarians to request evidence summaries that address complex patient care questions. As with the consultation services provided to clinicians during in-patient rounds, each out-patient clinical area has one librarian designated as its primary liaison, with additional members of the library team substituting as needed.
In response to a clinical question received through the StarPanel system message basket, the librarian searches the literature, selects the ''best'' evidence based on methodological rigor and relevance to the question at hand, and summarizes the evidence in a written synthesis that is delivered to the clinician through the StarPanel system. (An example of a question and librarian response is provided in Appendix A online). The librarian is also able to access the record of the patient prompting the question, when appropriate, to facilitate tailoring the literature to the clinical context, which is also a key characteristic of the service provided to in-patient clinicians. These clinical questions have required from three to more than ten hours of librarian time to complete, depending on the complexity and volume of the literature published on each topic.
Because this service is integrated into the EMR, it directly links evidence expertise into the clinical workflow using the StarPanel message system to facilitate easy and direct communication between clinicians and the librarian. The current study combines service usage statistics, a focus group, and a clinician survey to evaluate the strategies used to market the service to clinicians and to assess clinicians' perceptions about the utility of the EBM Literature Request service. The specific research questions addressed by the current work are: 1. Did newly implemented marketing and communication strategies increase physicians' use of the EBM Literature Request function in StarPanel? 2. How did clinicians rate the relevance of the information provided through the EBM Literature Request Service? 3. How was the information provided to the clinicians used and shared?

METHODS
The study employed a prospective observational design to examine physician use of the EBM Literature Request service over a 10-month period (March-December 2005), complemented by a focus group in July and a survey conducted in January 2006, each described further below. The study population included 137 attending and resident physicians in the Adult Primary Care Center (APCC) at VUMC, the first clinic to receive access to the service. The Vanderbilt Institutional Review Board reviewed and approved the procedures employed in each of the study components.

Standard marketing and communication strategies
Starting with the launch of the EBM Literature Request service in the APCC in late 2003, the librarian assigned to this unit used several strategies for advertising the service to the clinic's physicians. These techniques were designed to remind clinicians about the availability of the service and to help the librarian develop collegial relations with the clinic physicians. From March through July 2005 in the current study, the librarian used standard publicity methods, including a periodic email sent to all APCC physicians describing the service, flyers and posters distributed and posted in the APCC clinic areas, and individual in-person demonstrations of the service for approximately fifteen physicians during clinic hours.

Focus group
With the goal of designing improved strategies for marketing the service's utility in APCC clinic workflow, the investigators invited clinicians to share their feedback by participating in a focus group discussing the EBM Literature Request service in July 2005, the mid-point of the study. This 1-hour focus group included a convenience sample of 8 APCC clinicians (6 attending physicians, 2 resident physicians). Attendees received lunch and a $25 gift card to a local bookstore for their participation. The participants were asked their opinions about their past experiences with the EBM Literature Request service, the types and frequency of clinical questions requiring additional assistance in locating relevant literature, and their preferred methods for receiving communications about the service. The investigators reviewed a transcript of the focus group discussion, noting communication strategies that the participating clinicians suggested would increase service visibility and use in the APCC clinic.

Customized communication strategies
Based on the communication techniques recommended for librarians by the focus group participants, the investigators developed a set of new customized strategies for communicating with the physicians about the availability and utility of the EBM Literature Request service. These ''booster'' communications to foster ongoing use of the service began in August and continued through December 2005. The specific new communications included (1) an electronic monthly ''current awareness'' newsletter, the APCC Evidence Alert, that shared recent clinical questions answered by the service and key developments in the primary care literature; (2) more frequent visits by the librarian to the clinic during resident clinic hours to strengthen the potential educational impact of the service; and (3) collaborations between the librarian and internal medicine residents to prepare for morning report presentations.

Clinician survey
In January 2006, immediately after the 10-month observation period, a web-based survey was distributed to 137 APCC and affiliate physicians with access to the service. The 25-item survey explored their perceptions of the EBM Literature Request service and the electronic APCC Evidence Alert newsletter. (The survey instrument is included in Appendix B online). Respondents received a $5 coffee gift card for their participation. Developed from comments and suggestions shared by the focus group participants, this instrument was further refined through iterative review and feedback by additional clinicians, faculty from the VUMC Department of Biomedical Informatics, and librarians who worked with the APCC clinicians. The survey consisted primarily of brief multiple-choice and Likert-scale questions, with several open-ended questions to gather participants' general impressions and suggestions regarding the service. The survey focused in particular on the APCC Evidence Alert newsletter, the most novel and time-consuming of the new communication strategies.

Statistical analysis
Because physician use of the service was measured both before and after the launch of the customized communication strategies in August 2005, McNemar's test was used to compare the before versus after proportion of use of the service. Descriptive analyses included the number of questions asked by individual physicians. Each question was also classified by its primary purpose into one of ten categories previously developed for describing clinical queries [43].
With 137 clinicians, the study had 80% power to detect a 10% increase in the proportion of APCC physicians using the service after versus before the intervention, based on a type I error rate of 0.05 and correlation coefficient of 0.2, as recommended by Dupont for matched pairs analysis [44]. Differences in the categorical responses to the survey questions were assessed using Fisher's exact test, and the Mann-Whitney U-test was used to assess the Likert scale items.

Evidence-base Medicine Literature Request service usage by Adult Primary Care Center (APCC) clinicians
From March through December 2005, librarians received 45 questions from 23 unique requestors through the EBM Literature Request service, including 9 questions from residents (20%) and 36 from attending physicians (80%). Individual physicians asked a median of 1 question during the study period, with a range of 1-12 questions. The monthly distribution of service use over the study period is illustrated in Figure 1. There was no detectable difference in the proportion of physicians using the EBM Literature Request service between the 2 time periods (11% utilization in each time period, Pϭ1.0). Figure 2 shows the distribution of purpose categories among the 45 questions received during the study period. Treatment-related questions (treatment, treatment adverse effects, treatment efficacy) accounted for 58% (nϭ26) of the questions, while diagnosis-related queries (diagnostic procedure, differential diagnosis) composed 22% (nϭ10) of the questions. There was no detectable difference in the types of questions asked by attending physicians compared to residents (Pϭ0.11).
Overall results. All physicians in the APCC and its affiliate clinics were invited to participate in the Jan-     Table 1 summarizes clinician responses to the 3 general survey questions. The majority of respondents (nϭ45, 94%) indicated that they were aware of the EBM Literature Request service; however, only 40% (nϭ19) reported that they had actually used the service. Although the response rate was much lower among residents, those residents who responded were significantly more likely than attending physicians to report past use of the service (PϽ0.001). Over half of the participants (nϭ30, 63%) had read the monthly current awareness email newsletter, the APCC Evidence Alert.
Utility of librarian-provided information. The 19 physicians (6 attending physicians, 13 housestaff) who indicated that they had used the service in the past responded to an additional set of 4 questions about this experience (Table 2). Most physicians had used the service 1-2 times, but residents reported significantly more frequent use of the service than attending physicians (PϽ0.001). When asked to respond to the statement, ''Information provided by the librarian was relevant to my clinical practice'' (scale: 1ϭstrongly disagree-5ϭstrongly agree), participants assigned a mean ranking of 4.6 (range: [4][5]. In response to the statement, ''Information provided by the librarian sometimes leads to a change in my clinical practice'' (scale: 1ϭstrongly disagree-5ϭstrongly agree), respondents indicated a mean agreement of 3.9 (range: 3-5).
The majority of the physicians (84%) responded that the length of the librarian's written synopsis of the literature was appropriate, although 2 physicians felt that it was too long (11%). When queried regarding turnaround time for the service, most physicians (nϭ15, 79%) felt it was acceptable, while 3 physicians (16%) indicated that the turnaround time was somewhat too long.
Seventy-nine percent of the respondents (nϭ15) indicated that they had shared the information provided by the librarian with others. Residents were significantly more likely than attending physicians to report such information sharing (Pϭ0.03). Participants reported that the predominant recipients of this sharing included other attending physicians (nϭ10) or residents (nϭ11). This information was also sometimes shared with patients (nϭ4), fellows (nϭ3), and nurses (nϭ2). The main mode of information sharing was verbal (nϭ13). Other strategies included distributing a printout of the information (nϭ9), forwarding by email (nϭ5), and forwarding within the EMR message system (nϭ2).
Seventy-nine percent (nϭ15) responded that they Clinician-reported impact of librarian-provided information on clinical practice (nϭ19) had recommended the service to others, primarily to other residents (nϭ12, 63%) and attending physicians (nϭ8, 42%). There was no significant difference in the proportion of attending physicians compared to residents noting they had recommended the service to others (Pϭ1.0). Participants were then asked to describe how the information provided by the librarian affected their clinical practice (Figure 3). General self-education or instruction of trainees (nϭ18, 95%) and confirmation of a current plan (nϭ11, 58%) were the most frequently selected items. However, other more explicit ways the information changed clinical practice (e.g., implementing a new or different treatment, adding a diagnostic test) were also selected by more than 25% of respondents to this portion of the questionnaire (respondents were able to choose more than 1 item).
The APCC Evidence Alert newsletter. The 30 respondents who indicated that they had read the APCC Evidence Alert newsletter answered a set of additional questions about their experience with this new marketing and communication tool (Table 3). Most of these physicians (nϭ25, 83%) were aware that the library created the newsletter; however, several respondents did not know, or were unsure of, the alert's authorship. Most participants responded that the length of the APCC Evidence Alert was appropriate (nϭ27, 90%). When asked to indicate their level of agreement (1ϭstrongly disagree-5ϭstrongly agree) with the statement, ''The content of the Alert is relevant to my clinical practice,'' participants indicated a mean agreement of 3.9 (range: 3-5), but there was a marginally significant difference in the attending physicians' evaluation of its relevance (3.7) compared with residents' An evidence request feature (4.2, Pϭ0.05). When similarly queried regarding their agreement with the statement, ''It is useful to see questions that other clinical physicians have asked,'' respondents assigned a mean ranking of 4.1 (range: [3][4][5], with residents indicating a higher level of agreement with this statement than attending physicians (4.5 vs. 3.9, Pϭ0.02).

DISCUSSION
As the biomedical literature continues its rapid growth and demands on clinician time continue to increase, the increasing complexity of health care practice may be viewed as the impetus for a growing shift from knowing ''how to'' to knowing ''how to find out'' [45]. In this changing environment, effective care is supported by just-in-time, informatics-based connections between clinicians and various human and technological knowledge sources. The survey results at the end of this study project indicate that the new communication and marketing interventions developed in this project are well received by clinicians, even though they had no effect on the actual use of the EBM Literature Request tool during the first five months of the new strategies. In response to this positive reception by clinicians, the EBL team has maintained the customized marketing and communications strategies developed during the study and is implementing them with each new clinical area that begins using the EBM Literature Request feature. Given that these survey and focus group results also point to the utility of the service for educational purposes, Vanderbilt librarians also continue to target in-person visits to teams during resident teaching conferences and the teaching portion of resident clinic hours.
This study is subject to the inherent limitations of its observational and survey methods. Though the new marketing and communications strategies represent the only major change in the library's implementation of the service during the study period, additional changes in the clinic and its personnel might also have affected the volume of clinical questions. Generalizability is another consideration. This evaluation of service communication strategies has been conducted with one primary care clinical group, and the service has been provided primarily by one intermediate-level librarian. Although these new communication efforts have not resulted in a noticeable increase in the number of questions asked and the rate of questions remained somewhat low, future studies will include additional clinical areas and a broader pool of librarians as the EBM Literature Request Service is expanded throughout the medical center.
The very modest survey response rate, significantly lower among residents than attending physicians (18% vs. 67%) despite a higher frequency of service use reported by resident respondents, strongly suggests the potential for a response bias. Nonresponders may have been significantly different in terms of their awareness of and opinions about the EBM Literature Request service. Additional studies are needed to develop effective strategies for gathering feedback from this group and encouraging their use of this innovative new service.
The scalability of this novel service is also an issue that has yet to be well understood. To some extent, the current service employs the model: ''one clinician, one answer, one patient.'' In addition to the survey data indicating some ''viral'' spread of the librarian-provided information among clinicians and patients in the APCC, the library has further anecdotal evidence of information sharing among additional clinicians. Clearly much of the information delivered via this EMR messaging tool may interest others besides the individual clinician submitting the question, both in the originating clinical department and other clinical departments dealing with similar patient issues. The systematic sharing of recent questions via the monthly APCC Evidence Alert newsletter will likely need to be supplemented by additional techniques for promoting the service as it is scaled to other VUMC clinical areas.

CONCLUSIONS
In addition to the observational data indicating a steady influx of clinical questions through the EBM Literature Request service, the focus group and survey results reported here provide interesting indications that librarian-provided information and related communications are well received by the small group of clinicians who have used this service. Corresponding with the literature describing the complex factors that influence adoption of new informatics tools and services [46][47][48][49], additional work will be required to understand the adoption curve for services such as the EBM Literature Request utility in the Vanderbilt University Medical Center EMR system and to identify additional strategies for fostering clinician use of these services.

ACKNOWLEDGMENTS
The authors gratefully acknowledge the support of colleagues from the VUMC Eskind Biomedical Library, the Master of Public Health Program, and the Department of Biomedical Informatics in designing and conducting this research, including editing and design suggestions provided by Nila Sathe. This study was supported in part by a training grant (Jerome, T15 LM 007450-03) and a career development award (Rosenbloom, 2K22 LM008576-02) from the National Library of Medicine. The focus group and survey incentives were supported by funding from the National Library of Medicine (5 R01 LM07849-02). The authors also thank Gary D. Byrd, AHIP, for serving as guest editor and coordinating the editing, peer review, and publication decision process, as two of the authors are current members for the Journal of the Medical Library Association Editorial Board.