Barriers and enablers to the acceptance of bioinformatics tools: a qualitative study

One of the most important developments in the biomedical sciences today is the emergence of bioinformatics, the “science of managing and analyzing biological data using advanced computing techniques” [1]. The past decade has witnessed an explosion of biological data stored in large central databases as well as software tools to organize, visualize, and analyze the data [2–5], yet the acceptance and use of these applications by biologists lags behind this proliferation [6]. While some practices, such as the analysis of DNA and protein sequences, have fully diffused in the biomedical community, other bioinformatics practices still face adoption difficulties [7,8]. 
 
Medical libraries are increasingly required to provide services such as resources, training, occasional reference assistance, and individualized consultations to biomedical researchers [1,9–12] and have the potential to play a significant role in facilitating the acceptance and use of bioinformatics software by researchers. To provide effective services, medical libraries can benefit from gaining an understanding of the barriers and enablers to the acceptance of bioinformatics applications by researchers; however, at present there is a small body of literature on this topic [7–9]. 
 
A useful theoretical framework to study bioinformatics acceptance is Rogers' diffusion of innovations theory [13], adapted to the context of information systems by Moore and Benbasat [14]. Like other models of information systems diffusion [15–18], the framework suggests that perceptions of an information system play an important role in explaining end users' intentions to use a system and that intentions are predictors of actual use. Moore and Benbasat [14] propose eight perceptions, which are summarized in Table 1. 
 
 
 
Table 1 Perceptions of information systems innovations as defined by Moore and Benbasat [14] 
 
 
 
The authors previously studied the effect of hands-on training workshops, using either a structured step-by-step method or guided trial-and-error exploration methods, on end-user perceptions and intended use of bioinformatics tools for primer design and microarray data analysis [19]. Hands-on training positively affected perceived ease of use (PEOU) of the primer design tool; however, it decreased PEOU of the microarray data analysis tools. Surprisingly, intention to use both types of software decreased following hands-on training [19]. The present qualitative study was conducted to further increase understanding of the barriers and enablers to biomedical researchers' acceptance of bioinformatics applications, with a focus on the decision process underlying the selection of tools for primer design and microarray analysis and the long-term effect of training on these behaviors.


INTRODUCTION
One of the most important developments in the biomedical sciences today is the emergence of bioinformatics, the ''science of managing and analyzing biological data using advanced computing techniques'' [1]. The past decade has witnessed an explosion of biological data stored in large central databases as well as software tools to organize, visualize, and analyze the data [2][3][4][5], yet the acceptance and use of these applications by biologists lags behind this proliferation [6]. While some practices, such as the analysis of DNA and protein sequences, have fully diffused in the biomedical community, other bioinformatics practices still face adoption difficulties [7,8].
Medical libraries are increasingly required to provide services such as resources, training, occasional reference assistance, and individualized consultations to biomedical researchers [1,[9][10][11][12] and have the potential to play a significant role in facilitating the acceptance and use of bioinformatics software by researchers. To provide effective services, medical libraries can benefit from gaining an understanding of the barriers and enablers to the acceptance of bioinformatics applications by researchers; however, at present there is a small body of literature on this topic [7][8][9].
A useful theoretical framework to study bioinformatics acceptance is Rogers' diffusion of innovations theory [13], adapted to the context of information systems by Moore and Benbasat [14]. Like other models of information systems diffusion [15][16][17][18], the framework suggests that perceptions of an information system play an important role in explaining end users' intentions to use a system and that intentions are predictors of actual use. Moore and Benbasat [14] propose eight perceptions, which are summarized in Table 1.
The authors previously studied the effect of handson training workshops, using either a structured stepby-step method or guided trial-and-error exploration methods, on end-user perceptions and intended use of bioinformatics tools for primer design and microarray data analysis [19]. Hands-on training positively affected perceived ease of use (PEOU) of the primer design tool; however, it decreased PEOU of the microarray data analysis tools. Surprisingly, intention to use both types of software decreased following hands-on training [19]. The present qualitative study was conducted to further increase understanding of the barriers and A supplemental appendix is available with the online version of this journal. enablers to biomedical researchers' acceptance of bioinformatics applications, with a focus on the decision process underlying the selection of tools for primer design and microarray analysis and the long-term effect of training on these behaviors.

Semi-structured interviews
In 2003-2004, semi-structured 60-minute interviews with a convenience sample of 12 of the 115 previously studied participants of the microarray analysis and primer design workshops were conducted 3-6 months after the workshops noted above. Although a small convenience sample was employed, the study participants represented the diversity of the biomedical community. Interviewees included 6 doctoral students, a post-doctoral fellow, a laboratory worker, a faculty member, a scientific manager for a biotechnological company, a manager of a genetics laboratory in a public hospital, and a member of a unit that provides DNA sequencing services for researchers. With the exception of 2 plant scientists and the DNA sequencing unit member, all interviewees were involved in biomedical research. Five interviewees (M1-M5) participated in microarray analysis workshops, 2 by structured hands-on activity and 3 by guided exploration. The other 7 participated in primer design workshops (P1-P7), 3 by structured hands-on activity and 4 by guided exploration. Participation in the study was voluntary, and no compensation was offered to participants.
All interviews were conducted by one researcher (Shachak) in interviewees' offices or laboratories. A predetermined set of questions (Appendix online) was employed to collect data regarding the effect of training, attitudes toward bioinformatics, perceptions, and intended and actual use of specific tools. Additional themes that emerged during the interviews were analyzed as well (e.g., knowledge barriers). Interviews were tape-recorded, transcribed verbatim, and analyzed using qualitative data analysis software (AT-LAS.ti) [20].

Analysis
Two researchers experienced in qualitative data analysis (Shachak, Shuval) scrutinized the data independently. Initial agreement between researchers was high (80%), and open discussions were held until reaching consensus. The framework analysis approach [21] was used. First, researchers read interview transcripts several times to familiarize themselves with the data. Second, a thematic (coding) framework was identified based on both predefined issues (i.e., perceptions, attitudes, intended and actual use, training effect) and emergent themes from the familiarization stage. Next, codes were assigned to the data, and thematic charts [21] were created. The final stage of analysis was data mapping and interpretation [22] in relation to the above predefined categories and emerging themes. To establish validity, triangulation with quantitative data Table 1 Perceptions of information systems innovations as defined by Moore and Benbasat [14] No.
Perception Definition 1 Relative advantage ''The degree to which an innovation is perceived as better than the idea it supersedes'' [13] 2 Ease of use ''The degree to which an innovation is perceived as being difficult to use'' [14] 3 Compatibility ''The degree to which an innovation is perceived as being consistent with existing values, past experiences, and needs of potential adopters'' [13] 4 Trialability ''The degree to which an innovation may be experimented with on a limited basis'' [13] 5 Visibility ''The degree to which an innovation is visible during its diffusion through a user community'' [14] 6 Result demonstrability ''The degree to which the benefits of an innovation are readily apparent to the potential adopter'' [14] 7 Image ''The degree to which use of an innovation is perceived to enhance one's image or status in one's social system'' [14] 8 Voluntariness ''The degree to which the use of an innovation is perceived as being voluntary or of free will'' [14] collected during the workshops was performed to assess agreement with this related data set [23].

FINDINGS
The following findings detail the major themes that emerged during data analysis. Selected quotations are depicted in Table 2.

Perceptions and adoption of bioinformatics tools
All twelve participants considered bioinformatics a valuable discipline for biomedical research, though they questioned the validity and standardization between different methods and databases (e.g., in gene annotations). Moreover, participants realized that ''a good biological question must precede the use of bioinformatics'' and that bioinformatics could not entirely replace experimental research.
Research needs emerged as the driving force behind the use of specific bioinformatics tools. These needs determined which applications to use and were the major reason for participating in training. Results showed that two perceptions of bioinformatics tools were most commonly associated with their intended or actual use: PEOU and perceived usefulness (PU). PEOU often affected the choice of particular software over other equivalent tools. This effect was especially true for participants from the primer design workshops. Participants from the microarray analysis workshop described its complexity as inhibiting their use of microarray data analysis software. All seven users of primer design software commented on the usefulness of primer design tools and reported that using them improved the quality of their work. In contrast, most of the five participants of the microarray analysis workshops did not refer to PU of microarray analysis tools.
Though all the interviewees reported using bioinformatics software, the applications and the extent to which they were used varied greatly between participants. Most interviewees utilized public databases such as PubMed and Online Mendelian Inheritance in Man (OMIM). Many also used sequence analysis tools. However, four interviewees commented on adoption difficulties or said they would have liked to utilize more bioinformatics resources than they actually did. In particular, analysis revealed differences in usage of primer design and microarray analysis tools. After the workshop, six of the seven interviewees from the primer design workshops actually used primer design software. In contrast, none of the interviewees from the microarray analysis workshops made use of the software, although they expressed the intention to use the tools. Except for one researcher who decided not to use microarrays at all, participants from the microarray analysis workshops employed research collaborations or services to analyze their data.

Training needs
Knowledge gaps and extensive learning time emerged as key factors that in combination inhibited the use of bioinformatics tools. In particular, interviewees felt that learning to analyze the results of microarray experiments would require a substantial learning effort and time investment. Therefore, alternative ways to analyze microarray experimental results were sought. Some interviewees reported looking for support and consulting in attempting to overcome knowledge barriers. These individuals looked to colleagues and local experts as well as institutional bioinformatics units. However, they felt that support, especially from local experts, was insufficient.
Interview data revealed three reasons for taking the workshops. Most participants attended the workshops because they fulfilled a specific job need. Participants of the primer design workshops described a positive effect on their decision to use software tools for this purpose. On the other hand, participants of the microarray analysis workshops described a negative or no effect on their usage decision.

DISCUSSION
The present study attempted to illuminate factors affecting the acceptance of bioinformatics software by biomedical researchers. Although data (theoretical) saturation [24] was reached as no new themes emerged when the final participants' responses were analyzed, the small number of participants and the fact that they volunteered for the study might affect external validity. None of the interviewees participated in both microarray analysis and primer design workshops, which might weaken the validity of comparison between the two workshops. Although participants referred to other bioinformatics applications as well, the  primary focus of the interviews was on tools for primer design and microarray analysis. Therefore, results might not be generalizable to all types of bioinformatics tools yet despite these limitations, the study provided insight into their acceptance by researchers. Participants expressed two major perceptions that affected their usage decisions (of the primer design and microarray analysis tools): PEOU and PU. This was consistent with previous quantitative findings, which suggested these two perceptions significantly explained respondents' intention of use. [25]. The quantitative data, however, suggested that PU explained a greater portion of the variance in intended use than PEOU, while in this study the majority of comments referred to PEOU. Gefen and Straub proposed that PEOU has a greater impact on acceptance ''when the task itself is an integral part of an IT in-terface'' [26]. The task, for which bioinformatics applications are frequently used (i.e., managing organizing, visualizing, and analyzing biological data), could be considered to be such an integral task. As many bioinformatics applications suffered from poor user interfaces [27], the current findings suggested that their implementation might benefit from research and development of the human-computer interaction aspects of such tools.
Key factors that emerged as highly important for bioinformatics software acceptance were high knowledge barriers and learning time required for use, especially for complex tools such as applications for microarray data analysis. Instead of analyzing the data themselves, participants in the microarray analysis workshops employed alternatives such as research collaborations and paid services. Attewell suggested that services could ''enable user organizations to adopt a complex technology without (initially) having to acquire a full range of technical knowledge'' [28]. This highlights an opportunity for medical librarians, who are often familiar with the research needs and qualifications of researchers, to take an active role in establishing research collaborations and providing services and resources to facilitate researchers' use of bioinformatics tools. Services may range from bioinformatics training and support to paid services for complex tasks, such as microarray data analysis. Institutions that have employed highly qualified bioinformatics specialists, such as the medical libraries at the University of Washington and Purdue University [9,29], provide valuable service models for these approaches.
The present study finds that training positively influenced usage decisions regarding simple tasks and applications (primer design) but had no effect or a negative influence on adoption of complex tasks and tools (microarray analysis). This finding contrasts somewhat with a study by Yarfitz and Ketchell, in which the establishment of a bioinformatics training program resulted in increased use of bioinformatics resources overall [9]. However, the present study examined the effect of individual, short-term workshops rather than a long-term program, a difference in scope that might account for the different findings.
Previous quantitative data suggest that training moderated the effect of task and system complexity on perceived ease of use and that it might have a negative effect on intended usage of bioinformatics tools [19]. The findings of the present study further support this idea. The authors propose that training, or at least short training as is the case here, should be regarded as a complex intervention that allows potential adopters to better assess the objective characteristics of bioinformatics applications. This proposition is consistent with studies of other information systems that suggested that hands-on experience assists users to better assess systems' usability, thereby allowing them to form more realistic perceptions and expectations [30,31].

CONCLUSION
Effective utilization of bioinformatics in biomedical research has significant implications for discovering the underlying mechanisms of numerous diseases as well as potential treatments. The present study illuminates some of the barriers and enabling factors to the implementation of bioinformatics in biomedical research. Researchers employ the bioinformatics training they receive in their work only when the tools are easy to use and require short learning time. Research collaborations and data analysis services enable researchers to use cutting edge technology (microarray), thus overcoming knowledge barriers and enabling researchers to analyze the data themselves. These findings suggest a number of potential roles for medical libraries in supporting bioinformatics implementation, including infrastructure support, consultation, and training. In addition, libraries could provide services and initiate research collaborations for complex tasks. Future research may use the findings of this study to further examine ways of integrating bioinformatics into biomedical research and developing training modules to improve bioinformatics acceptance.

ACKNOWLEDGMENTS
The study reported here is based on doctoral research conducted in the Department of Information Science, Bar-Ilan University, and was supported by the Bar-Ilan President's Scholarships Program. The workshops referred to in the text were organized by the bioinformatics units of Tel-Aviv University and Weizmann Institute of Science and supported by the Israeli Center of Knowledge for Bioinformatics Infrastructure. Special thanks are due to Eitan Rubin for his ongoing help with this research.

BACKGROUND
In today's fast-paced health sciences (HS) information settings, health professionals need instant access to the best possible medical evidence. The challenge for information professionals is to deliver traditional information services both in person and in electronic venues. The increase in electronic resource publishing and advances in technology allow librarians to provide effective research, information, and instructional services without face-to-face communication. Virtual reference (VR) service, also known as digital reference, facilitates computer-mediated reference assistance [1]. It includes all electronic methods by which libraries fulfill patrons' information needs, such as email, online forms, interactive chat, and Web-browsing software [2]. In this study, live VR service refers to real-time human help delivered through the Internet via chatting software [3], a close simulation of traditional faceto-face reference for users who are not physically present in the library.
According to Dee [4,5], only 21% of academic health sciences libraries provided health-related VR services in 2002. In her follow-up study in 2005 [6], provision of health-related VR services in such libraries remained low at 27%. Most of the literature on chatbased VR services examines their use in academic, public, and school libraries; fewer authors have investigated VR in the health sciences [7][8][9][10][11][12][13][14]. Little discussion focuses on the competencies required for librarians and information professionals to provide effective HS VR services. Lankes [15] discusses the digital ref-  Table 1 and an appendix are available with the online version of this journal. erence research agenda, stating that one of the central questions in digital reference is ''how human expertise can be incorporated effectively and efficiently into information systems to answer information seekers' questions.'' Both the Statewide VRS Training Committee of the Washington Statewide Virtual Reference Project [16] and the Reference and User Services Association of the American Library Association have published practice guidelines for implementing and maintaining VR services [17]. These documents include sets of recommended core competencies for providing general VR services.
This study attempted to determine the expertise and training necessary for providing HS VR service. The intention of this study was to collect evidence from practitioners that could be used to fill knowledge gaps in providing effective chat-based HS VR services. By identifying which knowledge and skills are required, this research may help library and information science (LIS) educators revamp HS education and produce graduates who are better qualified to deliver these services.

METHODOLOGY Questionnaire
A thirty-six-item survey designed by the researcher was sent as an email attachment to the sample population. This research instrument (Appendix online) was developed based on the core competencies required for providing both face-to-face reference services and general VR services [16][17][18][19][20][21]. In addition to responding to general questions, the participants were asked about (1) the knowledge and skills required to provide HS VR service and (2) their view of LIS education and training for such service. Content validity was determined through a review of the instrument by two LIS research faculty with significant expertise in survey research design and methodology, information ethics, and library and information studies. For pilot-testing, two graduate students and the researcher independently completed the questionnaire. After each individual completed the questionnaire, a meeting was held during which wording was modified as necessary to improve clarity of the instrument.

Survey sample
The sample population included information professionals providing chat-based VR services in four health-related information settings: academic biomedical/HS libraries, hospital libraries, special bio-healthrelated libraries, and VR service systems associated with health-related services (e.g., statewide consortia and outsourced VR service systems). Email invitations to complete the survey were sent to thirty-one heads or directors of information and reference services in academic HS libraries affiliated with the Association of American Medical Colleges (AAMC) [22]. These libraries were identified through the AAMC Website as providing chat-based VR services. In addition, an invitation to participate in the survey was sent to several professional mailing lists (i.e., MEDLIB-L, SOCHAP-L, the Australian health librarianship e-list aliaHEALTH, and the UK medical, health care library community, and information workers e-list LIS-MEDICAL).

Participant demographics
While the total number of individuals who received the survey is unknown given changes in mailing list membership and the possibility of additional distribution of the invitation through email forwarding, only 27 individuals returned surveys and 21 provided complete data. Most of the participants were from the United States (n ϭ 16, 76%). Twelve of the 21 subjects (57%) were from biomedical or health sciences libraries, including libraries in academic medical centers. One-third of the participants (n ϭ 7) reported that they completed their initial library education in 2000 or later. These subjects might have had some VR service education in their LIS education programs. The average time providing HS-related VR services was 3.3 years, according to the subjects' self reports.

Resources used and skills and knowledge required
Reference resources in electronic format, such as databases and full-text electronic journal aggregators, were rated as the most important types of resources and the most useful by 71% (n ϭ 15) of the subjects (Table 1 online). All subjects (n ϭ 21) reported that the most essential knowledge included subject analysis, formulation of search strategies, and problem solving ( Table 2). All 21 subjects also agreed that the most important skill set was effective information retrieval, including formulating search strategies, online searching, and using indexes in print and electronic formats. The knowledge perceived as least important was related to management, such as budgeting and cost control of VR services (n ϭ 11, 52%). The skill set perceived as least essential was conducting a collaborative browsing (co-browsing) session with an online user (n ϭ 9, 43%).
Regarding the effectiveness of LIS education related to HS VR, less than half of the subjects (n ϭ 9, 43%) said that they felt satisfied with their LIS education or that their LIS education helped them develop adequate knowledge and skills for providing effective HS VR services. Participants were also asked to give their opinions on what subject matter should be included in LIS curricula. General database searching and search skills were reported as necessary (n ϭ 7), as were reference interview (n ϭ 7) and customer service skills. Four participants (19%) mentioned that it was essential to integrate training in general VR service skills into HS-related curricula, including real-time VR practice and chatting and messaging skills. Several subjects mentioned that it was more vital to have on-the-job VR training (n ϭ 7), and some thought it was not necessary to have extra training in VR before the job (n ϭ 4). The data did not speak to the necessity of providing VR training in continuing education programs. Subject analysis, formulation of search strategies, and problem solving 21 100 Ⅲ Effective information retrieval, including formulating search strategies, online searching, and using indexes in print and electronic formats 21 100 Ⅲ Knowledge of ready-reference materials in print 20 95 Ⅲ Interpersonal communication in online environments and netiquette 20 95 Ⅲ Knowledge of ready-reference materials in electronic format 20 95 Ⅲ Conducting reference interviews and transactions in online environments 20 95 Ⅲ Knowledge of Internet search engines and Web resources 20 95 Ⅲ Engaging in lifelong learning through professional development courses to improve skills in all areas of reference services 20 95 Ⅲ Knowledge of how to formulate policies (e.g., electronic reference service policies) 12 57 Ⅲ Applying electronic reference service policies (e.g., time limits, harassment, etc.) while providing services 11 52 Ⅲ Creating guidelines (e.g., time limits, harassment, etc. for electronic reference service policies 11 52 Least important Least essential Ⅲ Knowledge of budgeting and cost control of virtual reference services 11 52 Ⅲ Conducting a collaborative browsing (cobrowsing session with an online user 9 4 3

DISCUSSION
The scope of this research is limited by its small sample size and specific focus; however, the study is a step toward a better understanding of how to prepare professionals and students in providing effective chatbased HS VR services. Overall, the participants in the current study rated as important the skills noted in studies and projects related to developing VR reference skills or competencies [16,23,24]. These skills included online searching, reference interviews, interpersonal communication, and problem solving. Based on the higher importance ratings for these kinds of skills, it appears that face-to-face reference techniques are also useful in providing effective HS VR services, possibly because the delivery of reference services has certain similarities in both traditional and VR environments. Because high-quality services come from a better understanding of the user's needs, the fundamental knowledge and skill sets for reference services (including reference interviews, interpersonal communication, and online retrieval) should always be central to courses in LIS programs. While user instruction has been incorporated into guidelines for reference services practice in a variety of libraries [25][26][27][28], skill in user instruction through VR was seen as less important in this study. Similarly, in the current study, knowledge of the theoretical framework of reference interviews was not rated as important as knowledge of subject analysis, formulation of search strategies, and problem solving. Knowledge of VR services management-such as how to formulate policies (e.g., electronic reference service policies), budgeting, and cost control-was reported as less or the least important. These findings might reflect the immediacy of VR work, which typically focuses on providing fast, on-the-spot assistance.

CONCLUSIONS
Defining a set of essential knowledge and skills for HS-related VR services may help LIS educators revise health sciences librarianship curricula and produce better qualified graduates as well as assist librarians engaged in VR to develop their skills and knowledge. LIS educators and HS information professionals should work together to develop education and training models to produce qualified future librarians and to help professionals update their knowledge and skill sets. Ultimately, the goal is to ensure that HS librarians and information professionals are able to design and deliver customized and effective HS VR services in fast-paced and challenging health-related environments.