Non-biomedical Sources for Systematic Reviews of Pharmaceutical Policy

It has been well documented that reaching beyond MEDLINE into a diversity of databases enhances search results [1–3], but a chronic question in comprehensive and systematic searching is how far, and where, to search [4]. When published in business or economics sources, articles focusing on cost outcomes of health and health policy interventions may not be indexed in the biomedical databases that are traditionally consulted for clinical systematic reviews [5]. The goal of this dual case study is to explore and document the significance of non-health sources, specifically databases that index economic research, in comprehensive searching for two systematic reviews of pharmaceutical policy interventions. 
 
Due to ethical and logistical constraints, as well as lack of political will, controlled experimental studies of health policy interventions are rare [6]. The classic systematic review hierarchy of evidence may not apply to many policy reviews, as interventions of this type often can only be studied by using “natural experiments” and observational study designs [7]. Thus, systematic reviews of health policy interventions present methodological challenges to the investigators that are important for the librarian to be aware of—for example, the decision as to exactly which study designs or types of data are acceptable. 
 
Once the inclusion and exclusion criteria are clearly defined, the librarian must determine which sources are likely to be most productive. Search strategies for health policy reviews necessarily require a broader approach than that used in a classic clinical systematic review search, as sources such as the Cochrane Central Register of Controlled Trials (CENTRAL) and tools such as search filters for identifying randomized controlled trials will typically not be as fruitful for policy reviews as they are for reviews of therapeutic interventions. In an ideal world, one would always exhaustively search any source of possible relevance. However, when balancing time and resource constraints, it is useful for librarians to be aware of which sources are likely to provide the highest return on their investment.


INTRODUCTION
It has been well documented that reaching beyond MEDLINE into a diversity of databases enhances search results [1][2][3], but a chronic question in comprehensive and systematic searching is how far, and where, to search [4].When published in business or economics sources, articles focusing on cost outcomes of health and health policy interventions may not be indexed in the biomedical databases that are traditionally consulted for clinical systematic reviews [5].The goal of this dual case study is to explore and document the significance of non-health sources, specifically databases that index economic research, in comprehensive searching for two systematic reviews of pharmaceutical policy interventions.
Due to ethical and logistical constraints, as well as lack of political will, controlled experimental studies of health policy interventions are rare [6].The classic systematic review hierarchy of evidence may not apply to many policy reviews, as interventions of this type often can only be studied by using ''natural experiments'' and observational study designs [7].Thus, systematic reviews of health policy interventions present methodological challenges to the investigators that are important for the librarian to be aware of-for example, the decision as to exactly which study designs or types of data are acceptable.
Once the inclusion and exclusion criteria are clearly defined, the librarian must determine which sources are likely to be most productive.Search strategies for health policy reviews necessarily require a broader approach than that used in a classic clinical systematic review search, as sources such as the Cochrane Central Register of Controlled Trials (CENTRAL) and tools such as search filters for identifying randomized controlled trials will typically not be as fruitful for policy reviews as they are for reviews of therapeutic interventions.In an ideal world, one would always exhaustively search any source of possible relevance.However, when balancing time and resource constraints, it is useful for librarians to be aware of which sources are likely to provide the highest return on their investment.

METHODS
This dual case study was based on searches conducted by a librarian embedded in an interdisciplinary health policy research centre located at a large Canadian university.In both systematic reviews, the Both literature searches were done in several databases, including a mixture of core biomedical databases and others thought to yield relevant clinical and/or economic results.While most databases are common to both searches, some more ''niche'' sources were specifically selected for one or the other.As is common for systematic review searching, both search strategies also included citation snowballing, expert referral, gray literature searching, and some degree of hand-searching in core journals.
The search question in review #1 was: What are the clinical or economic outcomes (including health services use) of implementing either a reference pricing scheme or tiered formulary for prescription medications?Acceptable study designs included: randomized controlled trials, before-and-after or pre/post studies with nonrandomized comparison groups, interrupted time series analyses with or without comparison groups, and pre/post studies without a comparison group [8].
The investigators searched in a fairly standard suite of biomedical databases (CINAHL, Evidence Based Medicine Reviews, EMBASE, International Pharmaceutical Abstracts, and MEDLINE) as well as the business database ABI/Inform, the economics database EconLit, the general database Web of Science, ProQuest's Digital Dissertations & Theses (PQDT) database, and the Public Affairs Information Service databases (PAIS International and PAIS Archive).In addition to these major subscription databases and indexes, we searched a host of gray literature sources and online search tools.
The search question in review #2 was: What are the clinical or economic impacts (including health services use) of any form of direct-to-consumer advertising (DTCA) of prescription drugs?Acceptable study designs for this review were randomized controlled trials including cluster randomization trials, controlled before-after studies with contemporaneous or historic control, nested or non-nested case-control studies, comparative cross-sectional studies, and interrupted time series analyses.
We searched in the same suite of biomedical databases detailed above and all the same additional subscription databases with two exceptions: ABI/ Inform was abandoned as unlikely to contribute usable, unique studies after pilot searches on the topic, and Worldwide Political Science Abstracts was added, as the investigators wanted to attempt to capture relevant studies published abroad in law or political sciences sources.The lists of Internet and gray literature sources searched in the two reviews differed slightly as well; for example, the DTCA review searched in the Drug Promotion Database (DPD), which was selected specifically for applicability to this second review but not seen as relevant for the first.
All citations retrieved for the reviews were tagged with their sources of origin (e.g., database name or other method of discovery).Absolute number and percentage of relevant citations were tracked after the initial title-level review of citations, after the full-text review, and after the reviewers' final assessment of the articles meeting criteria to be included in the review.Citation sources for each review were analyzed for number of results from the source that were included in the review, number of unique results (found in no other source) included in the review, and ''missed'' results-in other words, citations that turned up in another search and ''should'' have been duplicated in that database search as well.Table 1 illustrates the number of citations, as well as the number of unique citations from no other source, for each database that furnished studies included in the two reviews.

Review #1: the impact of reference pricing and tiered formularies
Results indicate that MEDLINE and EMBASE, while highest in number of relevant citations, provided no unique results appropriate for inclusion in the final review.Non-biomedical sources, such as economic and business databases, on the other hand, garnered relevant results not indexed in biomedical databases.Citation snowballing offered more unique results than any database.For this review, neither PAIS nor our extensive web-based gray literature searching added results that could be included.
Of the twelve articles included in the reference pricing and tiered formularies review, only three were unique to a particular database.Those databases were EconLit, ABI/Inform, and PQDT.Five additional articles were uniquely identified through citation snowballing.The remaining four articles appeared in multiple data sources.
Review #2: the impact of direct-to-consumer advertising of prescription drugs Similar to review #1 described above, core biomedical databases yielded the largest number of results, but few unique items.Interdisciplinary databases such as Web of Science and PQDT did provide unique citations.In this review, EconLit was less fruitful and ABI/Inform was not searched.Neither political science database (PAIS or WPS) garnered any citations appropriate for inclusion in the review.However, gray literature sources such as the OAISter meta-repository search and DPD provided a unique result, as did citation searching.
Of the twenty-six articles included in the DTCA review, nine were unique to a particular database, and of those nine, six were found in PQDT.Two were in Web of Science and one in MEDLINE.The remaining three were found via OAIster, DPD, and citation searching.

DISCUSSION
Economic outcomes are increasingly evaluated alongside clinical outcomes in health policy research and reviews [9].Including both types of outcomes may increase policy relevance of the research, as decision makers struggle with questions not only of clinical efficacy, but also real life effectiveness and cost efficiency of interventions [10].
Large biomedical databases such as MEDLINE and EMBASE, which are common first sources to consult in a health-related review, are certainly inadequate as sole sources.Further, as demonstrated in this study, they may not be as essential in identifying unique articles for systematic literature searching as smaller, niche databases in tangential fields, due to the large amount of duplication of content among the large, comprehensive databases.As has been the case in other studies, citation snowballing is affirmed here as a critical element of the systematic review search process.Gray literature search tools are emerging as highly useful in some searches as well, particularly topics such as DTCA, which are relatively new interventions with strong appeal to a broad, interdisciplinary community of researchers.
While this study focused on two reviews of pharmaceutical policy interventions, cost effectiveness and resource-related outcomes of health technologies, methods of organizing health care, and population health interventions are of broad concern; thus, the results of this study will likely be applicable to many health policy topics.Investigators seeking to complete systematic reviews of pharmaceutical and other health policy interventions should strongly consider including non-biomedical databases, particularly economic and interdisciplinary databases, in their search strategies, along with citation snowball-ing and gray literature searching.When considering wise use of scarce available resources, it may be a better investment to look in ''niche'' databases or gray literature sources, rather than multiple core biomedical databases with large portions of overlap.
Brief communications: Wessel et al.intention was to examine clinical and economic outcomes of pharmaceutical policy interventions.

Table 1
Included and unique studies, by source*