Evidence-based medicine at the intersection of research interests between academic health sciences librarians and medical educators : a review of the literature

Objectives: In 2008, the Association of Academic Health Sciences Libraries established an Education Research Task Force (ERTF) to plan research addressing research priorities outlined in key Association of American Medical Colleges reports. ERTF members conducted a literature review to describe the state of collaborative research at the intersection of medical education and health sciences librarianship. Analysis of initial results revealed instruction in evidence-based medicine (EBM) was a shared interest and is thus the focus of this review.


INTRODUCTION
In late 2002, the Association of American Medical Colleges (AAMC) established the Institute for Improving Medical Education (IIME) to encourage and foster improvements in medical education.An ad hoc committee was established to direct the efforts of the institute, with the intention of conducting a comprehensive review of the state of medical education and making recommendations for reform.In 2004, the committee released its report and recommendations, Educating Doctors to Provide High Quality Medical Care: A Vision for Medical Education in the United States [1].The report presented a series of strategies for reforming medical education and achieving an ''Ideal Medical Education System.''The AAMC's Group on Educational Affairs (GEA) was invited to respond to the report with ideas for advancing its various recommendations.In 2006, the GEA issued its response in a report, Implementing the Vision: Group on Educational Affairs Responds to the IIME Dean's Committee Report: Educating Doctors to Provide High-Quality Medical Care: A Vision for Medical Education in the United States [2].
The GEA's response considered what was working well in medical education and what needed attention.
Among the areas identified for improvement were increased integration across the continuum of medical education; recognition (academic and financial) of educators; and more ''rational assessment'' across the This article has been approved for the Medical Library Association's Independent Reading Program ,http://www.mlanet.org/education/irp/..A supplemental appendix is available with the online version of this journal.

Highlights
N The Association of Academic Health Sciences Libraries Education Research Task Force identified three key questions for a review of the literature that relates to library teaching programs in medical education.
N There is an ample literature that reveals that librarians are active collaborators in instruction, educational assessment, and medical education research.
N A literature is emerging that addresses successes and failures of specific evidence-based health care instructional initiatives, including reviews and controlled trials.

Implications
N Gaps in the literature suggest a need for longitudinal follow-up and multicentered studies to validate the findings of the literature to date.
N Future studies should include effectiveness research to determine the impacts that evidence-based medicine instruction led by librarians, or in collaboration with medical educators, has on learning outcomes and practice.
continuum.The GEA report identified areas for future research in parallel to the areas of concern expressed in the IIME report.The GEA also articulated the knowledge, skills, and attitudes essential for the successful transition of learners throughout the medical education continuum, among them skills in accessing information, understanding and using evidence-based medicine (EBM), self-directed learning, self-assessment, and reflection.
Among the suggested areas for future research and the acknowledged lifelong learning skills expressed in the report were many of potential interest to the biomedical information and library sciences community.Health information professionals are in fact actively engaged in the full complement of teaching programs on their respective campuses or sponsored by their institutions, with a focus on the acquisition and application of informatics and knowledge management skills.
The IIME and GEA reports identified four key agendas in order to achieve the proposed ''Ideal Medical Education System'': & promote a patient-centered approach to medical care, & ensure that doctors are capable of providing highquality medical care, & improve the efficiency of educational processes, and & improve the effectiveness of the education processes.
The Association of Academic Health Sciences Libraries (AAHSL) [3] Board of Directors embraced the initial IIME report and in 2008 established an Education Research Task Force (ERTF) of the association's Teaching/Learning Oversight Committee (T/ LOC) to respond to both the IIME and GEA reports, with a charge to ''plan and conduct collaborative research that addresses the research priorities outlined in the IIME and GEA Reports.''The ERTF was explicitly charged to define, in partnership with the GEA, a research agenda of mutual interest.
The ERTF membership, which included key leaders from five academic health sciences libraries and an AAHSL Board-assigned liaison and project manager, determined that a critical first step in addressing its charge was to conduct a comprehensive review of the literature to describe the state of collaborative research at the intersection of medical education and health sciences librarianship, focusing on the four agendas identified in the IIME and GEA reports.

METHODS
The ERTF subsequently identified three key questions in constructing its approach to reviewing the scholarly literature: 1.What is the nature of evidence regarding teaching programs in evidence-based medicine (EBM) in medical schools or other curricula?2. What is the nature of evidence for improving the effectiveness of education processes, generally?3. What is the nature of evidence regarding teaching programs in consumer health information in medical schools or other curricula?Initial literature searches, conducted in 2008, focused on these three questions.Search strategies focused on the role of health sciences librarians in teaching and evaluating instructional programs, primarily in medical education.Furthermore, the focus should be on shared interests or collaborative initiatives of librarians and medical educators.Databases searched included: MEDLINE, CINAHL, and Academic Search Premier (EBSCO).Search results for each question were shared in CiteULike databases, where the citations could be ranked for relevancy and tagged with metadata tags describing subject matter and scope.
In assessing both the volume and nature of the literature regarding consumer and patient health, it was clear that while there is some shared interest between the medical educator and health information professional communities, the nature of the scholarship is neither as deep nor as rich in scope as that regarding EBM and lacked emphasis on educational programs.For this reason, the ERTF subsequently recalibrated its energies to focus on instruction and assessment of learning in evidence-based practice.
Creating a CiteULike group database revealed a degree of duplication and an overlapping research base of the EBM and education effectiveness literature.The analysis also showed that EBM was a predominant vein of common interest between authors from the medical education and health sciences librarianship fields of study.When this group database was further analyzed by publication type and research methodology, the articles on EBM emerged as evincing some of the most promising scholarship, so the decision was made to concentrate the review of the literature reported here on this subset dedicated to EBM instruction (hereafter referred to as the EBM Subset).

Evidence-Based Medicine (EBM) subset
The EBM search was updated in February 2012 using an iterative approach.A search was conducted in PubMed, CINAHL, Library Literature, and Google Scholar for all literature published since 1990.The ''Related'' feature of the PubMed database was used to identify articles that might have been missed by the original search strategy, and hand-searching of the references from selected articles was also conducted.The search strategy used for the update is found in Figure 1.
The inclusion criteria for the EBM subset were: 1. article was about instruction in EBM, including ''critical appraisal'' and 2. included librarian author(s) or 3. described research on collaborative efforts in EBM that included librarians (even if the authors were not librarians) Articles about evidence-based practice in fields other than medical education were included if they met the other criteria (however, the search strategies used did not specifically include other disciplines).The set was restricted to papers written in English.

Dorsch and Perry
Exclusion criteria were: 1. editorials or opinion pieces 2. articles primarily about database searching without an EBM focus 3. articles about EBM without an explicit or implied instructional focus 4. articles about evidence-based librarianship without a focus on EBM instruction Meta-data tags were developed to reflect key concepts and themes that emerged from the original searches.Medical Subject Headings (MeSH) were used to assign tags for publication type (PT) and research methodology.Any systematic reviews and meta-analyses were also tagged.PT and methodology tags were only assigned if they were listed in the MEDLINE record for the article.Therefore, not all of the articles were assigned tags for the PT and methodology categories.It should be noted that ''Evidence-Based Medicine'' did not become a MeSH term until 1997.

EBM subset
A total of sixty-eight articles (Appendix, online only) were selected for the EBM Subset after the inclusion criteria were applied.A majority of the articles appeared in the library literature (46/69 or 67%) (Table 1), with the greatest number of these (20) in the Journal of the Medical Library Association and its predecessor the Bulletin of the Medical Library Association.Medical Reference Services Quarterly was second, with 16 articles, followed by Health Information & Libraries Journal and its previous iterations (7) and the Journal of Hospital Librarianship (2).Twentythree articles appeared in a variety of medical education journals, with the greatest number of those appearing in Academic Medicine (5).Twentyfive of the articles, based on institutional affiliations cited in the publications, were coauthored by librarians and medical educators.
Topical patterns that emerged pointed to librarian engagement in various aspects of instruction, including curriculum development, assessment, program evaluation, and educational models (Figure 2).Curriculum development (24/68) was identified as a tag for 35.3% of the articles.Twenty of the articles (29.4%) dealt with ''librarian roles,'' often with the term explicitly stated in the title or abstract.The ''Searching'' tag was assigned to 16 articles (23.5%) that concentrated on teaching searching techniques for finding good evidence to support decision making.Another prominent theme was ''attitudes'' toward EBM and EBM instruction.Eight of the studies looked at attitudes of a variety of populations including medical students, residents, physicians, and librarians.Eight studies were categorized in program evaluation and 6 in assessment, together totaling 20.6%.Librarians were active in a variety of EBM venues and with a variety of audiences.The combined categories for medical students-clinical (6), medicalstudents-preclinical (5), and clerkship (6) totaled 17. Reports of teaching interns and residents numbered 15.Other audiences included nursing (3), allied health (3), dental (2), and veterinary (2) students.In addition to the traditional classroom, other teaching venues included the clinical environment (6), online learning (6), and morning report (3).
The articles showed variance in PT, with the most common being review (11).Three of these reviews were systematic reviews; however, very few studies in these reviews met the inclusion criteria of the systematic review authors.Walczak's review of studies looking at the effectiveness of programs to teach EBM teachers reported that only five studies met his criteria [4].In Parkes's Cochrane review of studies reporting on critical appraisal instruction, only one study qualified according to the set criteria for the review [5].The Cochrane review was updated in 2011, and its authors concluded that critical appraisal teaching interventions might result in modest gains, but that improvement to research examining the effectiveness of interventions was needed [6].The review category was closely followed by 7 randomized controlled trials (RCTs) and 7 evaluation studies.Also represented were controlled studies, including comparative (6) and validation studies (1).
The studies also varied in research methodology.Questionnaires (11/25) accounted for 45% of the articles that had a MeSH research methodology term applied.Articles that did not report the use of a recognized research methodology were tagged as ''descriptive.''Although many of the articles that were given the ''descriptive'' tag might have been classified as case studies, they were not identified as such either in the body of the article or by the assigned MeSH terms.
Significant conclusions were found in several of the areas of investigation, as summarized in Table 2. Of the studies of gaining EBM skills as a result of an educational intervention, results were mostly positive.Gardois (2011) found quantitative evidence of a significant difference in search performance in a clinical setting between residents assisted by a librarian and those searching the literature alone [7].Likewise, a recent validation study by Rana (2011) showed improvement in information-seeking skills using a tool to measure success of an EBM intervention [8].Schilling (2006) found that an electronic learning approach to teaching search skills for finding evidence-based articles can result in higher-quality results [9].Gruppen (2005) reported that even a brief training session can have marked benefit on quality of subsequent EBM literature searching [9].Dorsch (2004) found a statistically significant increase in medical students' self-assessment of EBM skills [11], and Frasca (1992) reported significantly higher scores on a critical appraisal test in a group of medical students who had instruction when compared to a control group [12].Bradley (2002) found that residents improved and retained searching skills six months after an intervention [13].
Although a number of the articles on librarian roles were descriptive or review papers, a few of the papers studied the impact of librarian involvement in EBM.Aitken (2011) found that clinical librarian involvement led to a positive effect on information retrieval and clinical decision making in a clinical environment [14].Mulvaney (2008) reported that decision making was impacted by a library consult service [15], and Banks (2007) found that librarian-led searches at morning report were an effective means of introducing EBM [16].On the other hand, Koufgiannakis (2005) felt the impact of librarian participation in problem-based learning groups did not merit their continued participation [17].Another form of assessment, the Objective Structured Clinical Examinations (OSCE), was reported by Davidson (2004) and Burrows (1999) as an effective method of evaluating EBM and searching skills [18,19].

DISCUSSION
The findings from this review suggest that there is a body of literature addressing EBM instructional initiatives.It is notable that these publications appear in both the library and medical education literature, thus reaching cross-disciplinary audiences concerned with improving education in the EBM process and lifelong learning.Furthermore, many of the publications are coauthored by librarians and medical educators, highlighting the collaboration of these two groups in EBM education and an overlap of research interests.
Topical patterns that emerged from this review show that librarians are actively engaged in curriculum development, deployment, and assessment.The nature of the evidence regarding teaching programs in EBM generally comprises descriptive narratives of how a particular course was developed or curricular change implemented; evaluative studies that measure success of courses or programs based on student satisfaction; and studies that rely on self-reports of competency or one-time course evaluations, with little or no follow up.However, some higher level studies show that librarian-led programs or teaching programs in which librarians are collaborators contribute to building EBM competencies.To be expected, competency in search skills is one of the dominant themes in this literature.This may be due not only to the domain of librarian expertise, but perhaps also because demonstration of search competency lends itself to skill measurement.In fact, four of the seven RCTs measure search performance.However, one of the RCTs reported here went beyond search skill measurement on to examine the effect librarian involvement had in influencing clinical decision making.
The various roles of librarians in EBM emerged as another dominant thread in this review of the literature.These studies, often case based, showed librarians successfully engaged in EBM activities using a great variety of teaching methods; participating in traditional teaching venues, in the clinical environment, and at the forefront of emerging online formats; and reaching audiences across the medical education continuum.With these studies as a foundation, librarians and their collaborators in EBM education can expand their efforts guided by the Aitken, 2011 [14] Clinical librarians led to a positive effect on provider attitudes, information retrieval, and clinical decision making.
Comparative study Gardois, 2011 [7] Found quantitative evidence of significant differences in search performance between residents searching the literature alone and those assisted by a librarian.

Randomized controlled trial
Horsley, 2011 [6] Found modest improvement as a result of teaching critical appraisal skills to health care populations.Recommended more rigorous studies.

Systematic review
Ilic, 2011 [20] Established a protocol for a randomized controlled trial to identify effectiveness of delivering a formal workshop in EBM literature searching skills to third-year medical students.

Randomized controlled trial
Rana, 2011 [8] A validated tool to measure evidence-based information-seeking skills showed improvement following an intervention.
Validation study Walczak, 2010 [4] Only five papers met criteria for inclusion; there is a need for the development of assessment tools for teaching the teacher courses in EBM.

Systematic review
Mulvaney, 2008 [15] Library consult service impacted clinical decision making.Randomized controlled trial Banks, 2007 [16] Librarians at morning report is an effective means of introducing EBM search skills.Evaluation study Leon, 2007 [21] Smartphones with availability of authoritative sources may make EBM in a community hospital setting a reality.
Evaluation study Schardt, 2007 [22] Pilot study too small to demonstrate a statistically significant difference between patient/problem, intervention, comparison, outcome (PICO) templates and standard PubMed search.
Randomized controlled trial, comparative study Scherrer, 2006 [23] Collaborative teaching model is effective as an EBM ''train the trainer'' approach.
Evaluation study Schilling, 2006 [9] An online approach to teaching EBM search skills can result in higher-quality search strategies, identification of higher-quality evidence, and improved confidence in information retrieval and analysis skills.

Randomized controlled trial
Gruppen, 2005 [10] Brief training session can have marked benefit on quality of subsequent EBM literature searching.
Comparative study Koufgiannakis, 2005 [17] Impact of librarian participation in EBM problem-based learning groups was not sufficient to warrant continued participation.
Randomized controlled trial Davidson, 2004 [18] Evaluation of EBM skills can be carried out during a performance-based exam (OSCE).Evaluation study Dorsch, 2004 [11] Pre-and post-test results found a statistically significant increase in medical students' self-assessment of EBM skills.
Evaluation study Linton, 2004 [24] Pre-and post-test results indicate that students learn general principles of practicing EBM, but do not acquire strong practical skills for application in clinical situations.
Evaluation study Jacobs, 2003 [25] An information literacy model in an evidence-based practice (EBP) nursing curriculum proved successful as measured by pre-and post-questionnaire and observed improved outcomes in assignments.
Evaluation study Bradley, 2002 [13] Residents search skills improved after an EBM intervention.Randomized controlled trial Chichester, 2002 [26] Compared the curricular utilization of evidence-based health care philosophies in baccalaureate and non-baccalaureate dental hygiene programs.
Comparative study, review Parkes, 2001 [5] There is evidence that teaching critical appraisal has positive effects on participants' knowledge, but only one study met inclusion criteria.

Systematic review
Earl, 1999 [27] Librarians' involvement in an EBM exercise did not result in a higher degree of sophistication in residents' search results nor attention to the techniques outlined in an EBM tutorial.
Comparative study Frasca, 1992 [12] Critical appraisal course taught by librarians and physicians is effective in teaching skills.
Comparative study EBM research interests of academic health sciences librarians and medical educators experiences and lessons learned presented in this case-based literature.Table 2 highlights conclusions of some of the publications in this review that provide evidence that supports the premise that EBM instruction contributes to competencies in search skills, critical appraisal of the literature, clinical decision making, and ultimately lifelong learning.The studies collected in Table 2 represent RCTs, controlled trials (evaluation, comparative, and validation studies) and systematic reviews.These studies and the case-based studies in this review form a body of research that will only benefit from future longitudinal and multicenter studies that replicate and validate the evidence to date and consider the long-term impacts and outcomes that collaborative teaching is having or could have.

CONCLUSIONS
This review uncovered a scholarship of common interest between peers in medical education and health sciences librarianship in the area of EBM instruction.Coauthorship between the two groups and the distribution of the literature across library and medical education journals points to a productive collaboration.A portion of this literature is case based and describes the collaboration of librarians and medical educators in the development, delivery, and assessment of EBM instruction across the continuum of medical education.An emerging literature of controlled studies that measures the impact of crossdisciplinary EBM instruction points to continued collaboration in the classroom and the promise of a robust research agenda.

Figure 1
Figure 1 Search strategy for Evidence-Based Medicine (EBM) Subset Evidence-Based-Medicine OR journal club OR decision making OR ''critical appraisal'' OR information storage and retrieval OR PubMed OR MEDLINE OR databases, bibliographic OR computer literacy OR ''information literacy'' OR point-of-care-systems.AND Educational measurement OR program evaluation OR teaching OR curriculum OR evaluation studies OR outcome assessment OR models, educational OR clinical competence OR professional competence OR evaluation* OR competenc* OR assess* OR outcomes AND education, medical OR schools, medical OR students, medical OR academic medical centers OR internship and residency OR clinical clerkship AND Librarians OR library services OR libraries, medical OR libraries, hospital OR librar*

Table 1
Journal titles by frequency of articles in the Evidence-Based Medicine (EBM) Subset

Table 2
Highlights of conclusions from studies with designated publication types