Citation analysis of Minnesota Department of Health official publications and journal articles: a needs assessment for the RN Barr Library.

OBJECTIVE
The paper describes the information needs of a state public health agency, compares needs to its library's collection, and evaluates collection development policy accordingly.


METHODS
A citation analysis of journal articles authored by Minnesota Department of Health staff as well as official publications from 2002 to 2004 was conducted. Fifty-six publications fitting the criteria for inclusion in the study were identified using PubMed and library records. Information on each cited reference was recorded, including reference type, relative age of citation, and journal name, if applicable. The library's collection and collection development policies were analyzed in regard to the results.


RESULTS
As expected, journals were the most heavily cited format: 63% (n = 897) of all citations were to journal articles. Most cited materials were between 2 and 5 years old. The 897 journal citations represented 265 different journals. The top 10 cited journals (4% of all titles) accounted for 36% (n = 320) of all citations; 62% (n = 320) of journals were cited only once. Of the total journals cited, the library subscribed to 70% (n = 627).


DISCUSSION
Overall, no large gaps appeared in the RN Barr Library's journal collection. The analysis confirms that the library's collections budget for serials and books reflects the cited use of these materials.


Implications
• Lengthy materials retention may be necessary for libraries serving public health practitioners. • Original research journals are a critical format for disseminating public health practice knowledge. • This citation analysis is unique research in public health information behavior and demonstrates how public health practitioners' information needs and behaviors can be analyzed using citation analysis, though locating all relevant publications may be challenging. • Multiple sources of library and journal use data are valuable indicators for systematic collection assessment.

INTRODUCTION
Founded in 1872 simultaneously with the creation of the Minnesota State Board of Health (now called the Minnesota Department of Health), the RN Barr Library has provided Minnesota's public health community with document delivery, literature searching, research, and other library services for over 130 years. The RN Barr Library has multiple clienteles, including the 1,400 employees of the Minnesota Department of Health (MDH), over 3,500 local public health employees in Minnesota, approximately 700 school health professionals, and MDH grantees and interlibrary loan clients. MDH itself has 4 Twin Cities metropolitan locations and 7 district offices throughout the state. In fiscal year 2003, local public health employees and school health employees constituted 8.2% and 4.4% of the library's client interactions, respectively. MDH staff, on the other hand, composed over 70% of the library's client interactions. Like other public health professionals, MDH employees' information needs vary due to the nature of public health and the many different types of work done by the department. Public health contains elements from the hard sciences, including chemistry, water sciences, engineering, and toxicology; the natural sciences, including microbiology, biochemistry, and genetics; the social sciences, including social work, education, and psychology; medical sciences, including infectious diseases, chronic diseases, epidemiology, and biostatistics; and much more [1]. MDH employs researchers, policy analysts, microbiologists, epidemiologists, statisticians, health educators, public health nurses, and engineers.
In 2004, the library spent about 80% of its limited collections budget (approximately $50,000) on 143 serial subscriptions. The library also manages subscriptions for MDH divisions, and approximately 150 of these subscriptions are housed in the library. Every year, the library reexamines its journal subscriptions with extensive cost-benefit analyses to assess how the collection fits the needs of this diverse clientele and the library budget. Due to the political nature of public health funding, MDH staff projects vary greatly from year to year, necessitating new journal subscriptions and rendering other subscriptions unnecessary. In addition to these normal yearly requirements, a 2006 move to a smaller location and its attendant pressures to winnow the collection provided impetus to formally analyze journal subscriptions and holdings. Previous journal collection decisions at the RN Barr Library have been based on use and interlibrary loan figures. This paper looks at MDH's information needs using citation analysis. Citation analyses study the patterns of citations in documents, an objective method for gathering data about information needs [2]. Rather than focusing on users' perceived needs, as in a questionnaire-type needs assessment, citation analysis gives the researcher a picture of actual needs as reflected by the items referenced in formal publications by the library's users.
To more objectively identify MDH's information needs, a citation analysis was performed to explore patterns of cited materials by MDH authors. In addition, this analysis examined the age of cited materials to see if the library's current retention patterns fit clients' needs. Lastly, this study compared cited journals with the library's journal collection to identify any gaps.

METHODOLOGY
This study is limited to publications by MDH staff. Although the RN Barr Library serves Minnesota's local public health and school health professionals as well, those clients are less likely to use the library and are harder to identify due to lack of a comprehensive directory. MDH staff publish both in official MDH publications and in more traditional publishing venues, particularly in academic and professional journals. To represent the largest range of publication types, both official documents and journal articles published between 2002 and 2004 were examined for this citation analysis.
To identify appropriate journal articles, two methods were used. First, the author examined all MDHauthored journal articles previously identified by RN Barr Library staff. The library examines new journal issues regularly to identify staff publications and requests that MDH employees notify the library whenever an article is published. To identify any articles missing from the file, a PubMed search using the term ''Minnesota'' as an affiliation was performed; to narrow the search, citations including ''Mayo'' or ''University'' in the affiliation field were excluded from the results. A total of 618 records were retrieved and examined for MDH affiliation. Because each article's lead author would presumably be the main researcher and writer, articles with lead authors outside MDH were excluded to focus on articles for which the MDH library would likely be the primary library utilized. Exceptions to this rule were made if the publication had an equal or greater amount of MDH authors than non-MDH authors. Authors without affiliation information listed in PubMed or the article, where applicable, were checked against MDH employee rosters. A total of 44 articles fit these criteria.
Official MDH documents were identified using the 2002 to 2004 issues of the RN Barr Library's newsletter, Check It Out [3], which contains a section listing newly cataloged MDH documents. Twelve documents containing any references to materials other than original data were included in this study.
The bibliography, references, footnotes, and endnotes from each of the fifty-six total publications (journal articles and official documents) by MDH staff were analyzed and entered into an Excel spreadsheet. Each reference was ascribed one of six types: journal article, book, book chapter, Website, conference proceedings, or other. These categories were chosen based on categories from published citation analyses [4,5]. For this study, the ''other'' category included newspaper articles, technical or government documents, datasets (e.g., Youth Risk Behavior Surveillance Survey data), and any other reference not fitting in the first five categories.
The age of each citation in relationship to the citing publication by MDH staff was also recorded using the categories: same year, previous year, two to five years old, six to ten years old, eleven to twenty years old, and twenty-one or more years old. These particular age ranges were chosen based on known frequency patterns of in-house use. Citations without dates were categorized as unknown. In this study, citations were counted as ''same year'' only if the article was published in the same calendar year (i.e., 2003 article with 2003 citation). This practice was used for consistency, because some MDH publications lacked month or seasonal information. Only references used in the MDH publications were included; lists of related resources or Websites with more information were not included in the analysis.
Collection information was used to identify whether each cited journal article was available in the library collection. The results of the citation analysis were compared with other library data sources, including records of journal titles used by library staff, a study sampling print journal title use by date of publication conducted during four months in the period April 2003 to July 2004, and interlibrary loan request records. Finally, the journal citation data were grouped into Bradford zones using the methodology employed in the Mapping the Literature of Nursing Project [6] to further examine the concentration of citations to specific journal titles.

RESULTS
A total of 56 MDH publications, including references to 1,434 citations, were examined. Of those, a check of

Figure 1
Types of citation by format

Figure 2
Age of cited materials authorship revealed the most publications (n ϭ 18) were authored by the MDH Infectious Disease Epidemiology, Prevention and Control (IDEPC) Division ( Table 1).

Type of cited material
As expected, journals were the most heavily cited format: 63% (n ϭ 897) of all citations were journal articles. Other citations (including technical reports, data sets, and assorted citations) garnered 15% (n ϭ 220) of the cited references, followed closely by Websites with 13% (n ϭ 193). Books, book chapters, and conference proceedings accounted for 5% or fewer citations (n ϭ 72, 31, and 21, respectively) ( Figure 1).

Age of cited materials
The library use statistics indicated that clients requested journal articles most commonly in the current year of publication, followed by the previous year. After this time period, requests fell sharply off, though decadesold materials were still used. In the examined publications, however, MDH employees cited materials 2 to 5 years older (39%, n ϭ 562) and materials 6 to 10 years older (21%, n ϭ 297) more frequently than the current and previous year combined (21%, n ϭ 295) ( Figure 2). Almost a fifth (17%, n ϭ 246) of all cited references with dates were more than 10 years old, a far greater percentage than in library data regarding material requests filled internally (less than 5%).

Journals
Publications by MDH staff cited 897 journal articles from 265 different journals. The top 10 cited journals (4% of all titles) accounted for 36% (n ϭ 320) of all citations. The top 25 cited journals (9% of all titles) accounted for 54% (n ϭ 487) of all citations. In this study, only 7% (n ϭ 19) of total journal titles were cited more than 10 times. The great majority of all cited journals, 62% (n ϭ 167), were only cited once. Figure 3 displays the number and percent of journal titles by the number of times a title was cited. Of the total cited journal articles (n ϭ 897), 70% (n ϭ 627) are to journals currently subscribed to by MDH. An additional 8% (n ϭ 73) of citations are to journal titles to which the library owns partial runs but does not currently subscribe ( Figure 4). The RN Barr Library currently subscribes to 18 of the top 25 cited journals (Table 2). An additional 5 journals on this list are purchased by MDH divisions and donated to the library. MDH does not subscribe to only 2 titles in the top 25, Applied and Environmental Microbiology and Antimicrobial Agents and Chemotherapy. Both of these titles were previously subscribed to, but cancelled for lack of use and cost. Neither of these titles was requested more than 5 times via interlibrary loan from January to August 2004, and both are freely available online after 6 months.
The top twenty-five cited journals were compared to the library's internal data regarding the top twentyfive copied journals to determine if there were major differences in use patterns (Tables 2 and 3). Not surprisingly, twelve journals were represented in each list, including the major medical publications (New England Journal of Medicine, JAMA, and Lancet). The most heavily cited journal, MMWR (Morbidity and Mortality Weekly Report, the CDC's epidemiological journal), was not in the top twenty-five copied journals, likely due to personal or departmental subscriptions and online avail-  Titles in bold are represented in Tables 2 and 3. ability from 1983 to the present. The remainder of cited journals that were not copied as heavily were specialty publications (e.g., Diabetes Care, Journal of Infectious Diseases). Similarly, the copied journals that were not heavily cited focused on subjects outside the scope of the MDH publications studied here, such as school nursing (e.g., Journal of School Nursing, Archives of Pediatrics and Adolescent Medicine).
Of the top 25 cited journals, many were available to MDH employees online ( Table 2) Looking at all cited journals, only 76 of the library's current 143 subscriptions were cited. An additional 5 titles that had changed titles were cited, and the library subscribes to them in their current form. Two journals that were in the top 25 copied list, Ethnicity and Disease and American Family Physician, were not cited, however.
To further analyze the RN Barr Library data in a wider context of public health literature, the cited journal titles were sorted by citation frequency and divided into Bradford distribution zones as shown in Tables 4 to 6: Zone 1, journals with the top third of citations (9 titles, range of citations 16-70); Zone 2, journals with the middle third of citations (36 titles, range of citations 4-15); and Zone 3, the journals in the bottom third (220 titles, range of citations 1-3). Because of the smaller amount of citations in this study, the difference between appearing in Zone 2 and Zone 3 was only 1 citation for several titles, indicating that a larger sample size would be helpful for more concretely differentiating Zone 2 and Zone 3 titles.

DISCUSSION
The collected data indicate a number of interesting phenomena. First, MDH patrons use materials from a range of dates in their formal publications, including materials more than ten years old. Alpi and Adams have found a similar dependence on older materials in their study of public health nursing literature [7]. This is important to note because the library moved to a new location in 2006, for which the facilities management staff at MDH encouraged discarding older journals, replacing paper journals with microfiche, and using off-site storage. Currently, the library houses complete runs of all public health journals it subscribes to but limits runs of journals outside public health. Clearly, however, the library's clients do require access to older materials; if these were moved off-site or withdrawn, library staff would have to perform more work for patrons to acquire needed materials. Indeed, it may be prudent to lengthen retention for some of the highly used general medicine journals such as JAMA and New England Journal of Medicine (currently housed for thirty years). Libraries with larger serials budgets may also wish to consider purchasing electronic backfiles of certain key journals.
The most important findings, however, pertain to the journal collection. In a 2001 article about the information needs of public health professionals, Rambo et al. broadly state, ''The public health workforce does not need bibliographic references. They do need carefully filtered and synthesized information that can be applied to solving problems'' [8]. Though this may be true of some public health professionals, particularly those in local public health, library statistics in the current study show that MDH employees use journal articles far more than any other type of information. This is roughly comparable to findings in other multidisciplinary citation analyses [4,5]. Studies examining citation patterns in specific public health disciplines likewise show comparable percentages of citations to journal articles [7,9]. These numbers are considerably lower than those found in the hard sciences or medicine, which have been found to be close to 80% or higher [10,11], likely because of the interdisciplinary nature of public health. As in studies relying on focus groups and surveys for findings [12][13][14], this study also identified public health practitioner dependence on Web resources, gray literature, datasets, and early reports, such as conference proceedings.
The RN Barr Library appears to be spending an appropriate percentage of its collections budget on serials as opposed to books. Currently, the library operates at approximately 80%/20% spending for serials and books, a ratio that is quite similar to the cited materials. In fact, only 7% (n ϭ 103) of all cited materials were books. Other heavily used materials, namely Websites and miscellaneous materials like datasets, are outside the scope of the library's collection development scope and budget.
As with other published multidisciplinary and discipline-specific citation analyses [10,15,16], this study found that publications by MDH staff cite articles from a large number of journals. Also similar to other studies [10,[15][16][17], the journal article citations found in this study were not spread evenly across these publications; very few journals contributed the majority of citations, a phenomenon often called the 80/20 rule or Bradford's Law of Scattering [6].
The journals the library currently subscribes to appear to be quite appropriate. MDH does not currently subscribe to 164 of the cited titles, but only 5 of those titles were cited more than 5 times. In fact, of those 164 titles, 130 were cited only once, results that largely conform to the Bradford distribution. Thus, proposing new subscriptions to these titles based on the citation analysis seems unnecessary. The number of current library subscriptions not cited might indicate that the library subscribes to many unnecessary titles, though this may be due to the limited user population studied. The only real surprise is the overwhelming popularity of MMWR, an artifact missing from library article request statistics and electronic journals data, likely due to its open access availability.
Others have also devoted attention to identifying key journal titles for supporting public health professionals, such as Alpi and Adams' and Scholman's bibliometric analyses of public health nursing and health education literature, respectively [7,9]. The Core Public Health Journals Project provides a more complete guide for libraries by listing journals important to public health academia and practice. Title selections, however, have been made by experienced editors and might not necessarily reflect the actual importance of specific journals to public health practitioners [18]. Indeed, three of the most highly cited titles in this investigation-Antimicrobial Agents and Chemotherapy, Diabetes Care, and Journal of Food Protection-are not included in the Core Public Health Journals Project list, Key Journals for All Public Health [18].
This study has several limitations. First of all, it is likely that not all publications by MDH staff were identified. Searching in PubMed for an affiliation is helpful in identifying many journal articles, but because affiliations may not be listed for all papers, include multiple variations of the department name, or may even be listed only by section or program name, it is hard to comprehensively identify all articles. Furthermore, because the affiliation listing is often for first authors only, articles without MDH first authors are difficult to identify. Lastly, though PubMed is the most used database for public health in general, it does not adequately cover many areas of public health. It is likely that many authors publish in journals not indexed in PubMed [18].
Though department policy dictates that MDH divisions notify the RN Barr Library about all official publications, some publications may not have been reported. This limitation may have skewed the data to favor particular disciplines at MDH, though with the exception of the Environmental Health (EH) Division employees, who are heavy library users, the publications roughly corresponded with amount of library use. The EH Division may be underrepresented because the majority of their library use is for risk analysis, which may not show up in formal publications. Conversely, the IDEPC Division may be overrepresented. The division director strongly encourages publication, resulting in a large number of recent publications including a series of smallpox articles in the Journal of Clinical and Laboratory Medicine and multiple articles on food safety in a Clinical Infectious Diseases supplement. However, IDEPC staff are also the li-brary's heaviest users, so the citations may be representative. In addition, because this study is limited to only one group of library clients, it is expected that some important journals are not represented in the citation analysis, particularly those dealing with school health.
Citation analysis is a controversial methodology because it does not represent all possible needs or uses for information, such as keeping up to date or background reading [15]. Critiques from 1944 on have pointed out flaws in citation analysis as a tool to establish quality of materials and their value to scholars [19]. Though valid criticism of citation analysis exists, several authors have shown that citations correlate with other methods of collection analysis, including impact factors, circulation statistics, in-house use, and user surveys [20][21][22]. Citation analysis cannot be used to assess collections in a void but should instead be used as one tool in the repertoire of collection development techniques [10].
Focused only on materials cited in publications by MDH staff, this citation analysis may overlook other critical information sources used by MDH's public health professionals. Because of the limitations of citation analysis, it is necessary to use this study's data judiciously and in context of other known information. The RN Barr Library's long history and close relationship with its clients means that much information is known about its clientele's information needs from other sources, such as interlibrary loan statistics, electronic journal use, and personal knowledge. Nevertheless, this study produced several new pieces of information that will be valuable for RN Barr Library staff in their collection development and management efforts.

CONCLUSION
In an age of spiraling journal costs and shrinking budgets, local citation analysis is one tool for enabling libraries to develop strong and cost-effective collections. It can be a useful technique for identifying potential collection development weaknesses, particularly any glaring omissions in collections. In this study, citation analysis has identified several points of unique information that were not discernable from other library statistics and helped to analyze the information needs and uses of a particular user population, MDH employees. Other library statistics merge multiple user populations, making it difficult to winnow out the needs of only one population.
As one of the few state health department libraries in the United States [23], the RN Barr Library already had some insight into what information needs public health professionals have and the uses they make of that information on a daily basis. Unlike many places in which public health professionals have little or no access to library services, Minnesota's public health professionals have been served with free document delivery (both from the RN Barr Library's collection and interlibrary loan), literature searches, research, current awareness services, and more for decades. Because of this long history of information accessibility, MDH authors may use information differently than other public health professionals. Nevertheless, this study has shown a number of similarities to other studies that researched public health information needs.
In contrast to other research on the information needs and use of public health professionals [8], MDH employees depended heavily on journal articles, possibly because MDH employees are at the state level and therefore responsible for synthesizing research for their local public health colleagues. The number and range of cited journal titles also confirms the diversity of public health information needs. In the current study, Zones 1 and 2 alone represented a large swath of public health disciplines, from environmental health to maternal and child health, as well as titles in medicine, science, and the social sciences.
In a recent surge of research on public health information needs and behaviors, this study is unique: it is the only citation analysis performed on a group of public health practitioners' publications. As such, it provides a unique vantage point for looking at public health information-seeking behavior, in addition to collection development feedback that is library specific. For librarians serving public health professionals, this study may have particular interest. Librarians serving areas without a history of information services could perform similar local citation analyses as a step toward improving or building new collections. Public health departments interested in subscribing to journals to support their work could use the most highly cited journal list as a starting point, in addition to broader lists such as those provided by the Core Public Health Journals Project. Because of the varied and often underserved needs of public health professionals, no one study can answer all the questions, but this preliminary look at one state health department's information needs is another piece of the puzzle.