• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of jmlaJournal informationSubscribeSubmissions on the Publisher web siteCurrent issue of JMLA in PMCAlso see BMLA journal in PMC
J Med Libr Assoc. Oct 2006; 94(4): 451–455.
PMCID: PMC1629423

Comparison of top-performing search strategies for detecting clinically sound treatment studies and systematic reviews in MEDLINE and EMBASE*

Sharon S.-L. Wong, PhD, RD, Assistant Professor,1 Nancy L. Wilczynski, MSc, Research Associate,2 and R. Brian Haynes, MD, PhD, Professor of Clinical Epidemiology and Medicine†,2,3

BACKGROUND

Increasingly, clinicians are doing their own searches using large biomedical literature databases [1]. Choice of database can influence the success of a search [2]. MEDLINE is often searched first due to its free access through the PubMed interface and its broad coverage of the biomedical literature, including nursing, dentistry, paramedic professions, reproductive biology, and clinical and experimental medicine. EMBASE searching is not free but has greater coverage of European and non-English language publications and topics such as pharmaceuticals, psychiatry, toxicology, and alternative medicine. The overlap of EMBASE and MEDLINE is estimated to be 10% to 87%, depending on the topic [3].

To date, search strategy development has focused more on MEDLINE [49] than on EMBASE. Search strategies developed for MEDLINE cannot be directly translated for use in other databases because indexing practices vary and thesaurus terms are not equivalent across databases.

In the 1990s, the Hedges Team developed MEDLINE search strategies for a small subset of 10 journals [10, 11]. This work has been expanded using data from 161 journals indexed in MEDLINE in 2000 and a 55-journal subset for EMBASE. The MEDLINE strategies [1218] and some EMBASE strategies [1922] have previously been published. This report compares the sensitivity and specificity of top-performing search strategies for detecting treatment and review articles in MEDLINE and EMBASE.

The methods used in this study have been detailed elsewhere [17]. Briefly, the operating characteristics of search strategies were compared with a manual review of 161 health care journals in 2000 for MEDLINE [23] and a 55-journal subset for EMBASE [24]. Six research assistants manually assessed all articles for studies meeting methodologic criteria in 7 purpose categories (treatment, causation, prognosis, diagnosis, economics, clinical prediction, and reviews). The authors reported purpose category definitions and methodologic criteria in an earlier paper [25]. To evaluate search strategies designed to retrieve studies meeting basic methodologic criteria, index terms and text-words related to research design features were run as search strategies. Search strategies were treated as “diagnostic tests” for sound studies, and the manual review (hand search) was treated as the “gold standard.” Operating characteristics of the search strategies were determined. Top-performing strategies for detecting sound treatment and systematic review articles in MEDLINE [17, 18] and EMBASE [26, 27] were compared.

RESULTS

Tables 1 and and22 present top-performing search strategies that allow for meaningful comparisons between MEDLINE and EMBASE.

Table thumbnail
Table 1 Top-performing strategies yielding high sensitivity and high specificity and minimizing the difference between sensitivity and specificity for detecting treatment articles in MEDLINE and EMBASE in the year 2000
Table thumbnail
Table 2 Top-performing strategies yielding high sensitivity and high specificity and minimizing the difference between sensitivity and specificity for detecting systematic reviews in MEDLINE and EMBASE in the year 2000

Strategies for detecting treatment studies in MEDLINE and EMBASE (Table 1)

In both MEDLINE and EMBASE, strategies used text-words and index terms. The high-sensitivity strategies in both databases performed similarly but used different combinations of terms except for the same text-word, “random:.tw.” The high-sensitivity MEDLINE strategy used the publication type, “clinical trial,” and the exploded therapeutic use subheading, “tu.xs,” neither of which was supported in EMBASE. The high-sensitivity EMBASE strategy used the exploded subject heading, “health care quality,” a term not supported in MEDLINE. Because several terms in these top-performing strategies were uniquely supported in their respective databases, it was not possible to directly test them in the opposite database.

Strategies that yielded high specificity and minimized the difference between sensitivity and specificity performed slightly better overall in MEDLINE than in EMBASE. MEDLINE strategies used the publication type, “randomized controlled trial,” which EMBASE did not support.

Strategies for detecting systematic reviews in MEDLINE and EMBASE (Table 2)

Strategies in both MEDLINE and EMBASE used text-words and index terms. The high-sensitivity strategy in MEDLINE was more sensitive than the comparable EMBASE strategy but had lower specificity. Both strategies included the publication type, “review,” but the MEDLINE strategy included an additional publication type, “meta analysis,” which was not supported in EMBASE. The EMBASE strategy used the subject heading, “methodology,” which was not supported in MEDLINE. In both databases, the textword, “search:.,” was a top-performing term.

The most specific strategies in MEDLINE and EMBASE were similarly specific, but the MEDLINE strategy had better sensitivity. The MEDLINE and EMBASE strategies used similar text-words (“MEDLINE.tw.” and “systematic review.tw”). In MEDLINE, the high-specificity strategy used the publication type, “meta analysis,” and the journal name, “Cochrane Database of Systematic Reviews,” both of which were uniquely supported in MEDLINE. This latter term retrieves Cochrane reviews, all of which would meet the criteria for a sound review. EMBASE did not index this publication. In EMBASE, the single term, “meta analysis.sh.,” also yielded high specificity.

The top-performing strategy for minimizing the difference between sensitivity and specificity in MEDLINE outperformed the comparable EMBASE strategy. The strategies were similar in the use of the terms, “search:.tw.” and “review.pt.,” but MEDLINE used the additional publication type, “meta analysis.”

DISCUSSION

Top-performing filters in MEDLINE and EMBASE were not equivalent in the search terms used, although some overlap existed. MEDLINE search strategies generally outperformed EMBASE strategies, but, in both databases, strategies achieved high sensitivities and specificities, whereas precision peaked at about 50%. Precision is inevitably low in large multipurpose databases [28].

EMBASE incorporated fewer publication types than MEDLINE, and, in the strategies examined, the best MEDLINE strategies contained several publication types that were not supported in EMBASE. Table 3 displays the performance of these publication types, plus “review.pt” (supported in both databases), and compares them with the similar term searched as a subject heading or text-word in MEDLINE and EMBASE. All MEDLINE publication types attained specificities greater than 90% and reasonably high sensitivities (>77%), except the term, “meta analysis.pt” (possibly because the definition used for “reviews” encompassed reviews broader than only meta-analyses). EMBASE subject headings generally yielded better sensitivities than the similar text-words. Text-words had overall lower sensitivity but slightly higher specificity than index terms, a finding that is consistent with previous research [29].

Table thumbnail
Table 3 Comparison of publication type (pt), subject heading (sh), and text-word (tw) search strategies for detecting sound treatment or systematic review articles in MEDLINE and EMBASE in the year 2000

As shown in Table 3, the higher sensitivities achieved by searching with publication types alone over text-words suggest an enhanced quality of indexing for publication types. This gives insight to the strategies in Tables 1 and and2,2, where use of publication types in MEDLINE yielded generally higher sensitivities. It follows that EMBASE strategies might be enhanced by expanding the available range of relevant publication types.

In the absence of a greater range of publication types in EMBASE, comprehensive searches should use subject headings with a methodologic focus over similar text-words to potentially improve sensitivity while maintaining a smaller difference between sensitivity and specificity. Text-word searching, however, remains valuable because text-words are versatile, do not rely on consistent indexing, and can be used to detect citation of “in process” articles that have not yet been assigned indexing terms. Comparing the use of text-words in the two databases is difficult because of differences in the spelling of key words, such as “randomized” in the United States and “randomised” in most European countries, or the different use of various methodologic concepts. An example of different use of methodologic concepts can be made with the term, “overview.” In the United States, an “overview” is considered to be a narrative review (a review that does not meet the methodologic criteria used in this study), whereas in the United Kingdom, an “overview” is considered to be a comprehensive systematic review (a review that would meet the methodologic criteria used in this study). Text-word searching requires the use of exact spelling of text-words in articles. Authors who use commonly understood text-words in titles and abstracts can facilitate a successful search.

Further work is necessary to determine the degree of overlap of citations retrieved using MEDLINE and EMBASE strategies. Better search strategies can probably be developed, but they would likely be more complex than those reported here, and improvements in sensitivity will be offset by decreases in specificity, or vice versa.

CONCLUSION

Optimal filters in MEDLINE and EMBASE for detecting treatment and review articles attained high performance using different search terms. Extra publication types in MEDLINE appeared to increase search sensitivity, and MEDLINE filters generally performed a little better than EMBASE filters. Considering the unique content coverage and search terms available in MEDLINE and EMBASE, information professionals would be best served by searching the database that is most relevant to their setting or searching both databases to improve a comprehensive search.

AUTHOR CONTRIBUTIONS

Haynes and Wilczynski prepared grant submissions in relation to this project, supplied intellectual content to the collection and analysis of data, and were involved in data analysis and staff supervision. Wilczynski and Wong participated in the data collection. All authors drafted, commented on, and approved the final manuscript.

Acknowledgments

The funding organization (National Library of Medicine, USA.) had no role in the design and conduct of the study; in the collection, analysis, and interpretation of data; or in the preparation, review, and approval of the manuscript. The Hedges Team includes Angela Eady, Brian Haynes, Susan Marks, Ann McKibbon, Doug Morgan, Cindy Walker-Dilks, Stephen Walter, Stephen Werre, Nancy Wilczynski, and Sharon Wong, all in the Health Information Research Unit, Department of Clinical Epidemiology and Biostatistics at McMaster University, Hamilton, Ontario, Canada.

Footnotes

* Source of funding: National Library of Medicine, grant no. 5R01LM06866-04.

REFERENCES

  • Westbrook JI, Gosling AS, and Coiera E. Do clinicians use online evidence to support patient care? a study of 55,000 clinicians. J Am Med Inform Assoc. 2004.  Mar–Apr; 11:113–20.Epub 2003 Dec 07. [PMC free article] [PubMed]
  • McDonald S, Taylor L, and Adams C. Searching the right database. a comparison of four databases for psychiatry journals. Health Libr Rev. 1999.  Sep; 16(3:):151–6. [PubMed]
  • Royle P, Bain L, and Waugh N. Systematic reviews of epidemiology in diabetes: finding the evidence. BMC Med Res Methodol. 2005.  Jan 8; 5(1:):2. [PMC free article] [PubMed]
  • Jadad AR, McQuay HJ. A high-yield strategy to identify randomized controlled trials for systematic reviews. Online J Curr Clin Trials 1993 Feb 27;doc no. 33. [PubMed]
  • Marson AG, Chadwick DW. How easy are randomized controlled trials in epilepsy to find on Medline? the sensitivity and precision of two Medline searches. Epilepsia. 1996.  Apr; 37(4:):377–80. [PubMed]
  • Adams CE, Power A, Frederick K, and LeFebvre C. An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care. Psychol Med. 1994.  Aug; 24(3:):741–8. [PubMed]
  • Dumbrigue HB, Esquivel JF, and Jones JS. Assessment of MEDLINE search strategies for randomized controlled trials in prosthodontics. J Prosthodont. 2000.  Mar; 9(1:):8–13. [PubMed]
  • Nwosu CR, Khan KS, and Chien PF. A two-term MEDLINE search strategy for identifying randomized trials in obstetrics and gynecology. Obstet Gynecol. 1998.  Apr; 91(4:):618–22. [PubMed]
  • Robinson KA, Dickersin K. Development of a highly sensitive search strategy for the retrieval of reports of controlled trials using PubMed. Int J Epidemiol. 2002.  Feb; 31(1:):150–3. [PubMed]
  • Wilczynski NL, Walker CJ, McKibbon KA, and Haynes RB. Assessment of methodologic search filters in MEDLINE. Proc Annu Symp Comput Appl Med Care 1993:601–5. [PMC free article] [PubMed]
  • Haynes RB, Wilczynski N, McKibbon KA, Walker CJ, and Sinclair JC. Developing optimal search strategies for detecting clinically sound studies in MEDLINE. J Am Med Inform Assoc. 1994.  Nov–Dec; 1(6:):447–58. [PMC free article] [PubMed]
  • Wilczynski NL, Haynes RB. for the Hedges Team. Developing optimal search strategies for detecting clinically sound causation studies in MEDLINE. AMIA Annu Symp Proc 2003:719–23. [PMC free article] [PubMed]
  • Wong SSL, Wilczynski NL, Haynes RB, and Ramkissoonsingh R. for the Hedges Team. Developing optimal search strategies for detecting sound clinical prediction studies in MEDLINE. AMIA Annu Symp Proc 2003:728–32. [PMC free article] [PubMed]
  • Haynes RB, Wilczynski NL. for the Hedges Team. Optimal search strategies for retrieving scientifically strong studies of diagnosis from MEDLINE. BMJ. 2004.  May 1; 328(7447:):1040. Epub 2004 Apr 08. [PMC free article] [PubMed]
  • Wilczynski NL, Haynes RB. for the Hedges Team. Developing optimal search strategies for detecting clinically sound prognostic studies in MEDLINE: an analytic survey. BMC Med. 2004.  Jun 9; 2(1:):23. [PMC free article] [PubMed]
  • Wong SS, Wilczynski NL, Haynes RB.. Developing optimal search strategies for detecting clinically relevant qualitative studies in MEDLINE. Medinfo. 2004;2004:311–6. [PubMed]
  • Montori VM, Wilczynski NL, Morgan D, and Haynes RB. for the Hedges Team. Optimal search strategies for retrieving systematic reviews from MEDLINE: analytical survey. BMJ. 2005.  Jan 8; 330(7482:):68–73. [PMC free article] [PubMed]
  • Haynes RB, McKibbon KA, Wilczynski NL, Walter SD, and Werre SR. Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey. BMJ. 2005.  May 21; 330(7501:):1179. Epub 2005 May 13. [PMC free article] [PubMed]
  • Holland JL, Wilczynski NL, and Haynes RB. Hedges Team. Optimal search strategies for identifying sound clinical prediction guides in EMBASE. BMC Med Inform Decis Mak. 2005.  Apr 29; 5(1:):11. [PMC free article] [PubMed]
  • Wilczynski NL, Haynes RB. Optimal search strategies for detecting clinically sound prognostic studies in EMBASE: an analytic survey. J Am Med Inform Assoc. 2005.  Jul–Aug; 12(4:):481–5. [PMC free article] [PubMed]
  • Wilczynski NL, Haynes RB. Hedges Team. EMBASE search strategies for identifying methodologically sound diagnostic studies for use by clinicians and researchers. BMC Med. 2005.  Mar 29; 3(1:):7. [PMC free article] [PubMed]
  • Haynes RB, Kastner M, and Wilczynski NL. Hedges Team. Developing optimal search strategies for detecting clinically sound and relevant causation studies in EMBASE. BMC Med Inform Decis Mak. 2005.  Mar 22; 5(1:):8. [PMC free article] [PubMed]
  • Health Information Research Unit, McMaster University. Full title of journals indexed in MEDLINE. [Web document]. [cited 9 Jun 2006]. <http://hiru.mcmaster.ca/hedges/Medline%20journals%20read.pdf>.
  • Health Information Research Unit, McMaster University. List of EMBASE journals. [Web document]. [cite 9 Jun 2006]. <http://hiru.mcmaster.ca/hedges/List%20of%20EMBASE%20Journals.pdf>.
  • Wilczynski NL, McKibbon KA, Haynes RB.. Enhancing retrieval of best evidence for health care from bibliographic databases: calibration of the hand search of the literature. Medinfo. 2001;10(pt. 1:):390–3. [PubMed]
  • Wong SSL, Wilczynski NL, and Haynes RB. for the Hedges Team. Developing optimal search strategies for detecting clinically sound treatment studies in EMBASE. J Med Libr Assoc:[accepted for publication]. [PMC free article] [PubMed]
  • Wilczynski NL, Haynes RB. for the Hedges Team. EMBASE search strategies achieved high sensitivity and specificity for retrieving methodologically sound systematic reviews. J Clin Epidemiol:[accepted for publication]. [PubMed]
  • Bachmann LM, Estermann P, Kronenberg C, and ter Riet G. Identifying diagnostic accuracy studies in EMBASE. J Med Libr Assoc. 2003.  Jul; 91(3:):341–6. [PMC free article] [PubMed]
  • Lowe HJ, Barnett GO. Understanding and using the Medical Subject Headings (MeSH) vocabulary to perform literature searches. JAMA. 1994.  Apr 13; 271(14:):1103–8. [PubMed]

Articles from Journal of the Medical Library Association : JMLA are provided here courtesy of Medical Library Association
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • Cited in Books
    Cited in Books
    PubMed Central articles cited in books
  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...