![]() | ![]() |
Formats:
|
|||||||||||
Copyright © 2008, Authors. Evidence-based Medicine Search: a customizable federated search engine Paul J. Bracke, MLS (corresponding author), Email: pbracke/at/purdue.edu, Associate Dean for Information Technology, Purdue University Libraries, 504 West State Street, West Lafayette, IN 47907-2058; David K. Howse, MLS, Email: dhowse/at/ahsl.arizona.edu, Information Services Librarian, Arizona Health Sciences Library, 1501 North Campbell Avenue, P.O. Box 245079, Tucson, AZ 85724-5079; Samuel M. Keim, MD, Email: sam/at/aemrc.arizona.edu, Associate Professor, Department of Emergency Medicine, University of Arizona, P.O. Box 245057, Tucson, AZ 85724-5057 Received August 2007; Accepted November 2007. Readers may use articles without permission of copyright owners, as long as the author and MLA are acknowledged and the use is educational and not for profit. Abstract Purpose: This paper reports on the development of a tool by the Arizona Health Sciences Library (AHSL) for searching clinical evidence that can be customized for different user groups. Brief Description: The AHSL provides services to the University of Arizona's (UA's) health sciences programs and to the University Medical Center. Librarians at AHSL collaborated with UA College of Medicine faculty to create an innovative search engine, Evidence-based Medicine (EBM) Search, that provides users with a simple search interface to EBM resources and presents results organized according to an evidence pyramid. EBM Search was developed with a web-based configuration component that allows the tool to be customized for different specialties. Outcomes/Conclusion: Informal and anecdotal feedback from physicians indicates that EBM Search is a useful tool with potential in teaching evidence-based decision making. While formal evaluation is still being planned, a tool such as EBM Search, which can be configured for specific user populations, may help lower barriers to information resources in an academic health sciences center.
INTRODUCTION Background Libraries and other information organizations have been grappling with the challenge of developing more efficient tools to allow users to discover information relevant to their needs, with many efforts focused on creating tools that facilitate access to a broad range of content to broad audiences. Web search engines from Alta Vista to Google, sometimes referred to as horizontal search engines, exemplify these approaches. Federated search engines implemented by many libraries provide a similar approach to enhancing access to the content of licensed and free article databases and other resources [1–4]. Recently, however, interest has increased in developing vertical search engines that provide access to targeted content to better meet the needs of particular communities or subject domains [5]. On the open web, Google Custom Search is one example of this approach [6]. At the same time, there has been significant interest in clinical medicine and other health professions in better integrating evidence from the medical literature and other sources into clinical practice. The complexities of knowledge translation [7] and the time constraints in clinical settings [8], however, create barriers to adopting evidence-based practice (EBP). Studies have shown that the user experience provided by commercial clinical information platforms can have a positive impact on both clinicians' perception of a search tool and their ability to find answers to clinical questions [9]. In addition to challenges specific to EBP, the convenience of general search engines has created a standard for ease of search, demonstrated in studies of clinician search behavior [10]. In response, many libraries have implemented federated search tools with single, integrated results sets [1–4]. Several projects have also created sophisticated search engines that federate clinical evidence in a single result set [11–14]. Additionally, other health sciences libraries have undertaken projects to create specialized search interfaces for specific types of resources [15, 16]. Librarian and clinician collaboration The Arizona Health Sciences Library (AHSL) felt these approaches could be improved by organizing the user experience around one of the visual constructs used to teach evidence-based medicine (EBM) at the University of Arizona [17]. Further, clinicians and librarians at the institution felt that the commercial clinical search products available to them were inherently limited by not integrating content from multiple publishers equally. This was especially true of electronic book resources. To develop an approach to the problem of integrating information searching and library expertise efficiently into clinical practice, the team chose local development of a tool because no commercial product was available to the institution that provided federated searching of licensed and freely available clinical evidence, provided supported customized display of results, and supported the embedding of power-searching strategies in a manner that was transparent to users. In light of this context, the AHSL, as a library dedicated to improving the integration of clinical evidence into clinical practice, sought to explore the following question: Is it possible to create a search tool that is both simple and convenient to use and sufficiently powerful to provide effective access to clinical evidence? As described elsewhere, it became apparent to the authors that synergies between clinical departments and the health sciences library clearly provided an opportunity for integrating evidence into clinical decision making [18]. Librarians had been involved in EBM efforts for several years in an instructional capacity, and while the team felt that teaching clinicians to effectively search and evaluate information had merits, instruction also seemed to be a limited approach. Given the time constraints inherent in the clinical workplace, it was also seen as unlikely that physicians would apply this instruction in practice. Likewise, models such as the informationist that have been implemented in other settings were seen as unrealistic in the University of Arizona environment, both for financial reasons and the unlikelihood that local clinicians would relinquish control of query formulation at the point of clinical decision making. The goal of the project was to create a scalable model for integrating librarian expertise into clinical decision making, without requiring their availability in real-time. To meet this goal, the team decided to develop a search tool that would allow physicians to control query formulation and result evaluation, but that would embed the expertise of librarians in resource selection and search construction. It would also embed clinical knowledge into the interface design to assist users in making relevance judgments. Evidence-based medicine search development The search tool, EBM Search, was programmed using Cold Fusion with a Microsoft SQL server database backend between February and August 2006. A rapid development process was employed, which focused on prototype development followed by an iterative feedback and modification process. The first iteration of the end-user search tool, developed in two weeks, provided the initial proof-of-concept and incorporated several design considerations. First, the search interface needed to be simple. Second, for practical and philosophical reasons, the team decided that providing a results screen that integrated results from all resources included in EBM Search into one single set was undesirable, in part, because the amount of development that would be required to manage a single results set was seen as unrealistic. More importantly, both clinical and library partners felt that an integrated results set was not optimal for enabling users to quickly select a resource at the appropriate level of evidence for their needs. Accordingly, the interface was organized around the evidence pyramid [16], a well-understood concept among local clinicians. Third, the search tool needed to incorporate librarian-designed search hedges to improve relevance for the target group, thereby reducing the need for expert searching skills from physicians. Fourth, EBM Search would use each resource's native interfaces for vocabulary mapping and relevance ranking, as these features were better developed in the individual resources than could be developed in this project. Finally, where feasible and meaningful, the number of results from each provider was displayed to enhance usability. The initial implementation was designed for the department of emergency medicine (EM) and allowed local emergency department (ED) physicians to execute searches on topics of their choice. Resources were selected through consultation with the EM residency director and interviews with EM residents. Based on this input, the resulting search provided access to pre-executed queries in a number of resources: PubMed Clinical Queries, the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effects (DARE), topic reviews from UpToDate, and textbooks from Stat!Ref. The links to PubMed Clinical Queries were limited in several ways, including a limit to core clinical and EM journals. In this case, core journals were determined by local clinicians to be those most useful for finding clinical answers. Figure 1
Finally, the results were displayed using a ranking system based on an EBM pyramid (Figure 2
INITIAL FEEDBACK AND CONTINUED DEVELOPMENT The prototype was well received by the EM department and generated interest from other departments after presentation in a grand rounds session. Although formal evaluation was still needed, the system clearly had potential and merited further development to be configurable for specialties other than EM. The system should also be modifiable by librarians without the assistance of a programmer. In subsequent phases of development, the user interface remained essentially unchanged. Minor formatting changes were made, and additional resources such as the National Guideline Clearinghouse were added. Some adjustments were also made to the search hedges, primarily to refine the limits applied to PubMed searches. For example, searches for PubMed Clinical Studies were disaggregated to provide separate access to articles on therapy, diagnosis, and prognosis, with each of these further limited to EM journals. Most development at this point focused on the backend tools that would allow librarians, or other managers, to create and configure search tools for specific audiences (Figure 3
Over the course of three months, the backend tools needed to manage EBM Search were developed and refined through a process in which initial prototypes were quickly developed and then modified in an iterative fashion, as reference librarians reviewed and critiqued initial functionality and suggested next steps. The result was a system that allowed nontechnical individuals to create customized federated search tools with only one point of intervention from technical staff: the addition and configuration of new searchable resources. This phase of development was completed in June 2006, although bug fixes and other minor enhancements have continued in the time since. DISCUSSION AND FUTURE DIRECTIONS The design of the EBM Search tool is important for several reasons. First, clinician-initiated queries are enriched with librarians' knowledge of resources and expert searching, and the tool has the potential to provide an added value for specific target groups by allowing convenient but powerful access to clinical evidence. Second, the configurability of the tool encourages collaboration between librarians and audiences to allow for assessing user need and creating a tool that meets these needs. Third, adapting the evidence ranking system from the Oxford Centre for Evidence-Based Medicine [17] potentially reduces the time needed to conduct critical appraisal. In addition, isolating Cochrane Reviews in the interface permits users to navigate to the highest-quality results first, if the source is appropriate for the clinical question. Finally, the evidence pyramid layout can function as a teaching aid, allowing users of the system who are unfamiliar with the principles of EBM to make connections between real search results and levels of evidence. While the EBM Search Tool has demonstrated promise, a number of enhancements could improve its capabilities. First, the usability of the interface for evaluating information sources could be improved. In part, this could be achieved by fetching and displaying result set sizes for more resources. Displaying result set size is a useful heuristic device allowing the selection of a resource that appears to meet a desired level of specificity, while saving time by allowing users to skip empty sets. In other cases, displaying the number of results might be less useful as a point of comparison. For example, knowing the number of results from a textbook search does not allow one to compare it to the results from an article database. In these cases, alternative methods for communicating what a user can expect by following any given link need to be developed. Technical issues, however, limit the number of resources for which results set sizes can be easily retrieved and displayed. Currently, vendor support for web services or other XML gateways for search and retrieval is limited, and Z39.50-based solutions can be complex to implement. In the long run, this issue should be less significant as more vendors support XML-based services such as search/retrieve via URL (SRU) or the National Information Standards Organization Metasearch XML Gateway Standard (MXG) [19, 20]. It is also essential that the tool remain adaptable to evolving clinical workflows. One of the primary reasons for the initial positive response to the EBM Search tool is that its design takes constraints of the clinical workflow into account. More factors, however, might be taken into account to further improve design in future iterations. In addition to the interface enhancements discussed above, it is also important to consider ways in which the search tool can be embedded in other online environments and workflows. For example, mobile versions of the search tool would be a logical future consideration, as would embedding the search tool in portals, hospital intranets, or other information technology resources that clinicians use. Despite the possibility of further enhancements, it would be important to maintain the foundational aspects of the current design that have led to a service that, anecdotally, has provided clinicians with a useful way to quickly access evidence. Continuing to provide a simple search interface and results interface organized around known constructs for grading levels of evidence will be critical. REFERENCES
|
PubMed related articles
Your browsing activity is empty. Activity recording is turned off. |
||||||||||
Ann Emerg Med. 2007 Mar; 49(3):355-63.
[Ann Emerg Med. 2007]BMJ. 2002 Mar 23; 324(7339):710.
[BMJ. 2002]J Med Libr Assoc. 2006 Oct; 94(4):435-41, e206-7.
[J Med Libr Assoc. 2006]J Med Libr Assoc. 2007 Apr; 95(2):156-63.
[J Med Libr Assoc. 2007]J Am Med Inform Assoc. 2006 Nov-Dec; 13(6):593-600.
[J Am Med Inform Assoc. 2006]J Biomed Inform. 2001 Apr; 34(2):85-98.
[J Biomed Inform. 2001]J Med Libr Assoc. 2006 Jul; 94(3):307-13, E192.
[J Med Libr Assoc. 2006]