|
|
J Med Libr Assoc. 2002 July; 90(3): 327–330. | PMCID: PMC116407 |
Copyright © 2002, Medical Library Association Finding the evidence: teaching medical residents to search MEDLINE *Eric W. Vogel, M.D., Assistant Professor of Medicine, †1,4 Kevin R. Block, M.S., Access Services Manager, 2 and Karen T. Wallingford, M.L.S., Senior Commercial Information Analyst31MCP Hahnemann University 3300 Henry Avenue Philadelphia, Pennsylvania 19129 2Scott Memorial Library Thomas Jefferson University 1020 Walnut Street Philadelphia, Pennsylvania 19107 3Wyeth-Ayerst Pharmaceuticals P.O. Box 8616 Philadelphia, Pennsylvania 19101 4†  To whom correspondence should be addressed Received December 2001; Accepted January 2001. Increasingly, medical residency programs are teaching their trainees how to practice evidence-based medicine (EBM), with one-third of internal medicine programs now offering freestanding EBM curricula [ 1]. Most EBM curricula described in the literature focus on teaching critical appraisal of original journal research [ 2]. However, other aspects of EBM are equally important. One crucial step is efficiently finding the best literature that answers clinical questions. As part of a newly implemented EBM curriculum, the authors have developed a workshop to teach our internal medicine housestaff how to search MEDLINE efficiently and effectively to find the answers to clinical questions arising from patient care. In the fall of 1997, we conducted a needs assessment to ascertain our residents' skills in literature searching. We performed a convenience sample by distributing a survey at various meetings of residents. Thirty-six percent of the respondents described their MEDLINE searching skills as only fair or poor. In addition, 56% felt teaching in this area would be extremely or very useful for them. To address the identified learning needs of our residents, we developed a three-hour workshop on using Ovid's version of MEDLINE to find high-quality information to answer clinical questions. After a pilot test of the workshop, the session was implemented with all second-year medicine residents (N = 42, male = 61.9%, mean age = 30). We taught the workshop six times (once per month over a six-month period), with an average of seven residents per workshop. In advance of the workshop, the residents were assigned to read a series of articles about using MEDLINE and searching to find answers to clinical questions [ 3–5]. The workshop began with a thirty-minute didactic presentation on MEDLINE searching. Topics included Medical Subject Headings (MeSH) versus textword searching, “explode” feature, search term truncation, Boolean operators, and use of the methodological filters developed by Haynes et al. [ 6]. Lastly, we provided an overall approach for converting clinical questions from patient encounters into MEDLINE search strategies [ 7, 8] ( Table 1). The approach emphasized retrieval of the largest possible sets (“mega-sets”) for each of the strategy components to find all of the potentially relevant literature. The methodological filters were then used to identify the most clinically relevant articles. | Figure 1 MEDLINE performance checklist |
The class then moved to the library computer center, where the residents were provided with personal computers for the remainder of the workshop. Searches were performed with Ovid's MEDLINE search interface. We guided the residents step by step in performing two MEDLINE searches for clinical questions from two hypothetical clinical scenarios, using the search functions and strategies described in the earlier didactic session. A handout contained materials from the didactic session, the clinical scenarios, and step-by-step instructions for the MEDLINE searches. Finally, class participants independently performed practice searches from a third scenario, with instructors available for assistance. All scenarios related to problems that would be seen in primary care medical practices. Two instructors supported the workshop; one primarily led the step-by-step instructions, and the other provided individual assistance for participants as needed. One week before the workshop, residents performed a pretest MEDLINE search, based on a clinical question from a hypothetical patient care scenario. Following the workshop session, the residents were assigned new scenarios for searching MEDLINE as a posttest. For example, one of the assigned questions was “In patients with Alzheimer's disease, do any vitamin supplements improve their cognitive function or slow progression of disease?” In addition, one to eleven months after the workshop, for a subsequent session of the EBM curriculum, residents independently performed MEDLINE searches to answer clinical questions from one of their own patient encounters. Performance checklists were designed for the MEDLINE searches. The performance checklists evaluated whether residents demonstrated the correct use of important MEDLINE strategies at least once in their searches and whether they retrieved the appropriate study type and articles to answer the clinical questions (Figure 1). Search evaluations were performed by one of the authors (Vogel). We compared the proportions of residents appropriately completing each searching task before, after, and in “long-term” searches, using the McNemar change test (Analyze-It Software). The residents' use of MEDLINE searching strategies and techniques all increased significantly (P < 0.05) in searches completed in the week after the workshop, compared to baseline. For all of the searching tasks in the long-term search, one to eleven months after the workshop, a significantly higher percentage of residents correctly used MEDLINE searching skills compared to their pretest searching (P < 0.05). However, in long-term searches, residents employed the explode function, textword searching, truncation, and the “OR” function significantly less frequently (P < 0.05) than their posttest search. Written feedback was sought at two times: at the end of the workshop and as part of overall EBM course evaluations a week later at the end of their ambulatory block. The workshop was generally very well received by the residents. In feedback at the end of the ambulatory block, more than half the residents rated the workshop as one of the best learning experiences of their rotation. Two participants suggested using residents' own clinical questions to teach searching in the future. We have developed a workshop that improves residents' ability to search MEDLINE effectively for information to answer clinical questions. For all searching techniques evaluated for the workshop, the percentage of residents using the technique appropriately improved significantly after the workshop. In self-directed MEDLINE searches later in the year, residents continued to demonstrate improved skills compared to their baseline abilities, although there was a significant decline in the use of some skills compared to their searches right after the workshop. A MEDLINE search back to 1990 yielded only a few studies related to developing and conducting classes on MEDLINE searching for residents. Most articles that concerned MEDLINE training discussed it as part of the curriculum in medical school, the more recent articles as a component of EBM instruction [ 9–14]. Other articles attempted to measure improvement in searching skills of residents and attending physicians after participating in MEDLINE training classes. These studies reported some improvement but also concluded that designing a good study about teaching methods was difficult, making additional research needed [ 15–17]. We also found articles discussing classes devoted to teaching more general information retrieval skills, but these articles did not detail structure and content of individual sessions in comparison to what we describe here [ 18–20]. The evaluation of our workshop had several significant limitations. First, there was no control group for comparison of searching skills over time. Therefore, we could not exclude the possibility that apparent improvement in abilities was related to some factor other than our instruction. Second, as only one of the authors evaluated the residents' searches, the assessment of searching skills might have had some bias, possibly resulting in overstatement of the magnitude of the benefit of the workshop. Third, we only evaluated the skills of residents in these structured searching assignments; we could not assess how frequently or thoroughly residents might perform searches on their own as a result of participating in this workshop. In the future, we need to evaluate how teaching MEDLINE searching has an impact on residents' abilities to find information for questions about their actual patients. It would be important to know that participants not only improve their searching skills from workshops like this, but also are more likely to search for information in this way. We hope participants would become more inclined to perform literature searches to find the most up-to-date information to answer clinical questions, as opposed to looking at less recent or less valid resources.
| Table 1 Overall MEDLINE searching strategy presented in workshop |
The authors thank Elias Abrutyn, M.D., and David Brody, M.D., for their thoughtful manuscript suggestions and Edward Gracely, Ph.D., for his guidance on statistical testing. We also acknowledge Margay Grassberger for her helpful suggestions and teaching in pilot testing of the workshop. Dr. Vogel also acknowledges the faculty and staff of the Michigan State University Primary Care Faculty Development Fellowship Program for their invaluable assistance in the development of this curriculum. - Green ML. Evidence-based medicine training in internal medicine residency programs: a national survey. J Gen Intern Med. 2000. Feb; 15(2):129.–33. [PubMed]
- Green ML. Graduate medical training in clinical epidemiology, critical appraisal, and evidence-based medicine: a critical review of curricula. Acad Med. 1999. Jun; 74(6):686.–94. [PubMed]
- McKibbon KA, Walker-Dilks CJ. Beyond ACP Journal Club: how to harness MEDLINE to solve clinical problems [editorial]. ACP J Club. 1994. Mar/Apr; 120:A10.. [PubMed]
- Palmer J, Lusher A, and Snowball R. Searching for the evidence. Genitourin Med. 1997. Feb; 73(1):70.–2. [PubMed]
- Haynes RB, Wilczynski N, McKibbon KA, Walker CJ, and Sinclair JC. Developing optimal search strategies for detecting clinically sound studies in MEDLINE. J Am Med Inform Assoc. 1994. Nov–Dec; 1(6):447.–58. [PubMed]
- Haynes RB, Wilczynski N, McKibbon KA, Walker CJ, and Sinclair JC. Developing optimal search strategies for detecting clinically sound studies in MEDLINE. J Am Med Inform Assoc. 1994. Nov–Dec; 1(6):452.–4.
- Haynes RB, Wilczynski N, McKibbon KA, Walker CJ, and Sinclair JC. Developing optimal search strategies for detecting clinically sound studies in MEDLINE. J Am Med Inform Assoc. 1994. Nov–Dec; 1(6):457..
- Palmer J, Lusher A, and Snowball R. Searching for the evidence. Genitourin Med. 1997. Feb; 73(1):70.–2. [PubMed]
- Barnett SH, Kaiser S, Morgan LK, Sullivant J, Siu A, Rose D, Rico M, Smith L, Schechter C, Miller M, and Stagnaro-Green A. An integrated program for evidence-based medicine in medical school. Mt Sinai J Med. 2000. Mar; 67(2):163.–8. [PubMed]
- Ghali WA, Saitz R, Eskew AH, Gupta M, Quan H, and Hershman WY. Successful teaching in evidence-based medicine. Med Educ. 2000. Jan; 34(1):18.–22. [PubMed]
- Rosenberg WM, Deeks J, Lusher A, Snowball R, Dooley G, and Sackett D. Improving searching skills and evidence retrieval. J R Coll Physicians Lond. 1998. Nov–Dec; 32(6):557.–63. [PubMed]
- Schwartz DG, Schwartz SA. MEDLINE training for medical students integrated into the clinical curriculum. Med Educ. 1995. Mar; 29(2):133.–8. [PubMed]
- Schwartz DG, Schwartz SA. MEDLINE training for medical students in the clinical curriculum. Acad Med. 1992. Jul; 67(7):484.. [PubMed]
- Woods SE, Francis BW. MEDLINE as a component of the objective structured clinical examination: the next step in curriculum integration. Bull Med Libr Assoc. 1996. Jan; 84(1):108.–9. [PubMed]
- Erickson S, Warner ER. The impact of an individual tutorial session on MEDLINE use among obstetric and gynaecology residents in an academic training programme: a randomized trial. Med Educ. 1998. May; 32(3):269.–73. [PubMed]
- Haynes RB, Johnston ME, McKibbon KA, Walker CJ, and Willan AR. A program to enhance clinical use of MEDLINE. a randomized controlled trial. Online J Curr Clin Trials 1993 May 11; Doc No 56:[4005 words; 39 paragraphs].
- McKibbon KA, Haynes RB, Johnston ME, and Walker CJ. A study to enhance clinical end-user MEDLINE search skills: design and baseline findings. Proc Annu Symp Comput Appl Med Care 1991;73–7.
- Teal JB. A library-based information skills rotation for internal medicine residents. Acad Med. 1998. May; 73(5):602.. [PubMed]
- Mangrulkar RS, Whelan CT, and Williams BC. Teaching medical information retrieval skills to internal medicine residents at the terminal-side. Acad Med. 1999. May; 74(5):573.–4. [PubMed]
- Earl MF, Neutens JA. Evidence-based medicine training for residents and students at a teaching hospital: the library's role in turning evidence into action. Bull Med Libr Assoc. 1999. Apr; 87(2):211.–4. [PubMed]
|
This article has been 
