Evaluating a health sciences library residency program: what have we learned?

INTRODUCTION In difficult economic times, institutions must make decisions about their core values and where they should concentrate their limited resources. Survival of vital programs can take precedence over other programs that are recognized as valuable, but not essential. Funding for immediate needs can, and often does, preclude funding for future needs. Such was the issue facing the University of Illinois at Chicago (UIC) regarding its Academic Resident Librarian Program. Initiated in 1981 as a postgraduate appointment for librarians who were new to the profession and lacking librarianship experience , the program was suspended in 2007 for reevaluation and study. UIC has estimated that it costs the university approximately $14,000 per resident for recruiting and training, in addition to their salaries. This study was designed to determine what the value of the program is to the former residents, whether the goals of the participants in the UIC Library of the Health Sciences (LHS) residency program matched or reflected those of UIC, and if UIC should continue investing in the program. A survey was designed to probe the initial motivation for those entering the residency, their satisfaction with the program, their greatest challenges, their biggest rewards, new skills they developed, their retention rate in health sciences librarianship, and the most and least valuable aspects of the program. The results provide insight for institutions considering establishing a similar residency or in evaluating current programs. The responses also provide a view of the issues that new health sciences librarians face during the beginning years of their careers. published a Systems and Procedures Exchange Center (SPEC) survey on internships, residencies, and fellowship programs [1]. At the time of publication eleven libraries offered residency programs. In 2001, twenty-two academic librarian residency programs were identified in the United States [2]. The ARL's SPEC kit defines a residency as ''post-degree work experience designed as an entry-level program for professionals who have recently received a graduate degree.'' This study adheres to that definition. The stated goals of the UIC program were: 1. increasing the pool of qualified academic librarians with an emphasis on traditionally underrepresented groups 2. providing an opportunity to work with leading academic library and information science professionals in a guided environment 3. preparing emerging academic library professionals, emphasizing strong commitment to research The decision by an institution to continue supporting a residency program is significant and has farreaching implications for the hosting institution, the profession, …


INTRODUCTION
In difficult economic times, institutions must make decisions about their core values and where they should concentrate their limited resources.Survival of vital programs can take precedence over other programs that are recognized as valuable, but not essential.Funding for immediate needs can, and often does, preclude funding for future needs.
Such was the issue facing the University of Illinois at Chicago (UIC) regarding its Academic Resident Librarian Program.Initiated in 1981 as a postgraduate appointment for librarians who were new to the profession and lacking librarianship experience, the program was suspended in 2007 for reevaluation and study.UIC has estimated that it costs the university approximately $14,000 per resident for recruiting and training, in addition to their salaries.
This study was designed to determine what the value of the program is to the former residents, whether the goals of the participants in the UIC Library of the Health Sciences (LHS) residency program matched or reflected those of UIC, and if UIC should continue investing in the program.A survey was designed to probe the initial motivation for those entering the residency, their satisfaction with the program, their greatest challenges, their biggest rewards, new skills they developed, their retention rate in health sciences librarianship, and the most and least valuable aspects of the program.The results provide insight for institutions considering establishing a similar residency or in evaluating current programs.The responses also provide a view of the issues that new health sciences librarians face during the beginning years of their careers.

BACKGROUND
In 1992, the Association of Research Libraries (ARL) published a Systems and Procedures Exchange Center (SPEC) survey on internships, residencies, and fellowship programs [1].At the time of publication eleven libraries offered residency programs.In 2001, twenty-two academic librarian residency programs were identified in the United States [2].The ARL's SPEC kit defines a residency as ''post-degree work experience designed as an entry-level program for professionals who have recently received a graduate degree.''This study adheres to that definition.The stated goals of the UIC program were: 1. increasing the pool of qualified academic librarians with an emphasis on traditionally underrepresented groups 2. providing an opportunity to work with leading academic library and information science professionals in a guided environment 3. preparing emerging academic library professionals, emphasizing strong commitment to research The decision by an institution to continue supporting a residency program is significant and has farreaching implications for the hosting institution, the profession, and those librarians hoping to begin a library career in an academic setting.This is especially true in the health sciences where two years experience is often a prerequisite for employment, making residency programs highly competitive.Over one hundred new library graduates applied each year the UIC program was offered.The minimum requirement was the recent completion of a master's degree from an American Library Association (ALA)-accredited program.Approximately one-third of the UIC applicants expressed an interest in the health sciences, and these were the applicants who were interviewed for the LHS positions.This brief communication reports on the evaluation of only one part of the UIC program, that of residents who completed at least one year of the program at LHS between 1997 and 2007.All the LHS residents served in the information services department whose services include reference, teaching, and outreach.

METHODOLOGY
Seventeen librarians completed at least one year of the residency program at LHS between 1997 and 2007.One has since died.The sixteen remaining librarians were contacted by email and asked if they would participate in a study evaluating the residency program.They were informed that the study had been approved by UIC's Institutional Review Board (IRB) and that participation was voluntary and would in no way affect their relationship with UIC should they decide to participate or not to participate.
All sixteen former LHS residents agreed to participate.A research assistant, with no previous contact with the residents, administered the survey.This was done with the intention of eliminating any influence on how the residents responded.
After receiving a positive response to participate from each former resident, the research assistant arranged for a mutually convenient time to call and administer the survey (Appendix, online only) over the telephone.The participants were told it would take approximately 15 minutes for completion and that results would be anonymous.All 16 (100%) of the A supplemental appendix is available with the online version of this journal.

J M L A
P r e p r i n t N o t F i n a l V e r s i o n former residents did participate although not all answered every question.

RESULTS
The survey provided both qualitative (Table 1) and quantitative information with respect to the UIC residency program.
When asked retrospectively what factors they remembered as important in accepting the UIC residency position, 14 (87%) of the former LHS residents remembered the desire for professional experience in an academic library as somewhat or very important.This was closely followed by 12 (75%) reporting health science experience as somewhat or very important.
When asked what they valued once they had completed the program, all 16 of the former LHS residents identified professional experience as somewhat or very important, followed by 15 (94%) listing reference experience and entry in the field as somewhat or very important.Networking opportunities were listed next in importance by 14 (87%) while 13 (81%) cited introduction to professional colleagues and teaching experience as next in importance.
Although few responded to the question asking what the least valuable aspect of the program was, the consensus of those responding was that a more structured program including well-designed seminars would improve the experience.
At the time of the survey in 2009, 14 (87%) were employed in libraries; of these, only 4 (25%) were employed in academic health sciences libraries.One (6%) was employed in an academic library.Of the 16 participants in the LHS program, only 2 (12%) were from underrepresented groups and neither of them was in an academic library at the time of the survey.
When asked if they had engaged in their own research since leaving UIC, 11 (73%) of the 15 who responded to the question said yes; of these, 6 (40%) have published in a peer-reviewed journal.

DISCUSSION
The first goal of the UIC program, increasing the pool of qualified academic librarians with an emphasis on traditionally underrepresented groups, met with mixed and discouraging results with only 5 (31%) of the former residents being employed in an academic library.Of the 16 participants in the LHS program, only 2 (12%) were from underrepresented groups and neither of them was in an academic library at the time of the survey.
The second goal, providing an opportunity to work with leading academic library and information science professionals in a guided environment, proved successful from the perspective of both the residents and the institution.Previous research has shown that residents are largely motivated by the desire to gain experience.Lanier and Henderson, who studied residents of three programs, National Library of Medicine (NLM), Library of Congress, and UIC in 1997 found that ''respondents seemed to agree that the reason they entered the internship was to gain experience, network, and acquire the skills that would allow them to specialize in their careers'' [3].These desires were reflected in the LHS residents' decision to accept the UIC residency position.
The third goal, preparing emerging academic library professionals with strong commitment to research, met with mixed and encouraging results.While the number of librarians who stayed in academic libraries was relatively low, the number committed to research, whatever their subsequent library setting was high with 11 (73%) librarians engaging in their own research since leaving UIC.
There were other findings not as closely tied to the library's goals that were of interest.The fact that lack of teaching experience was listed as the top challenge to new residents points to a continuing need for library students to be trained in techniques for teaching adult learners.A 2003 study indicated that the biggest change in health sciences librarianship in the previous ten years was the increasing emphasis on teaching [4].This need for librarians to possess teaching skills has not abated.
This survey had a number of limitations; for example, it did not include resident librarians at other UIC libraries, nor did it include all those who participated from its inception in 1981.And participants were asked to remember motivations and responses from events that occurred sometimes as long as ten years ago which may have changed the experience.

CONCLUSIONS
This survey demonstrated that the residency program provided many benefits to the newly graduated librarians who gained experience and expressed satisfaction with that experience, even though not all followed an academic health sciences career.Besides needing training in techniques for teaching adult learners, new librarians also need guidance in time management and in learning how to balance the many demands of their professional positions.Working in a real setting provides new librarians with the practical experience to decide if that setting is the right one for them, while they may in fact decide it is not.How libraries and librarians can broaden this opportunity for experiences needs to be explored and new programs devised.Some models for offering practical experience already exist such as the NLM Associate Fellow program, but these are not sufficient to meet the demand or interest.Instead new models need to be devised.NLM through the National Network of Libraries of Medicine (NN/LM) or professional organizations such as the Medical Library Association or ALA might sponsor the financial overhead of one or two year programs, while various institutions continue to pay the residents' salaries.Placements could be on a rotating basis at many institutions, so that the new librarians receive a variety of experiences, in which all hosts benefit from the innovations new blood inevitably brings, but no one host would bear the complete financial burden.Individual libraries need to connect with graduate library schools to initiate internships or practicum placements.At the time of this publication, UIC has not decided whether to reinstate its program but LHS is proposing a collaborative practicum with another academic health sciences library for students interested in health sciences librarianship.As former residents rise through the ranks of the library profession and go to institutions not previously hosting a residency program or practicum, they need to offer to serve as mentors as they themselves were once served.

BACKGROUND
A bill designed to improve health care access and quality for all citizens of Washington State was passed and signed into law in May 2007 [1].Many of the bill's provisions stemmed directly from recommendations made by the governor's Blue Ribbon Commission on Health Care Costs and Access.One of the commission's recommendations was that health care providers follow evidence-based standards of practice, where available.This recommendation was incorporated into law by mandating that the University of Washington (UW) Health Sciences Library (HSL) develop a web portal where eligible providers across the state could access evidence-based information to support patient care.To fund this portal, the law required that the Washington State Department of Health (DOH) impose a maximum $25 per year add-on to the license fees of health care providers.*State associations representing licensed care providers were contacted by legislative staff about whether or not their members should be covered by this legislation.Based on those efforts, 14 professions were named in the enabling legislation, totaling more than 150,000 providers (Table 1).A number of other professional groups either were not considered, could not be contacted in the limited time available, or, in the case of pharmacists and dentists, decided on behalf of their members to opt out.A contract was negotiated between DOH and HSL in 2008 covering the development and maintenance of the web portal and the transfer of fees to support this work.The DOH-HSL contract runs through June 30, 2011.The enabling legislation does not sunset.

METHODS
The HSL hired an experienced medical librarian to develop this program in November 2007.By that time the program had become known as HEAL-WA: Health Evidence Resource for Washington State ,http://www.heal-wa.org.. Prior to resource selection and site development, librarians conducted surveys and informal interviews with health providers in the three professions with the largest represen-* This surprising convergence of events came about due largely to connections between Sherrilynne Fuller, FMLA, then University of Washington Health Sciences Library director; the then president of the state medical association; members of the governor's staff; and an enlightened nurse legislator.
Brief communications: Scherrer

302
J Med Libr Assoc 98(4) October 2010 tation (physicians, registered nurses, and massage therapists) to learn about their information-seeking patterns as well as their familiarity with and use of electronic health information resources.Librarians at other educational institutions, including those in complementary and alternative medicine, were consulted about resources in their respective fields.
Because of the diversity of included professional groups, particular effort was made to select resources that would be of value to the broadest number of users.In light of the legislative mandate, resources were also selected, as far as possible, on the basis of their ease of use at the point of care.Plone [2], an open-source content management system (CMS), was selected as the platform for the HEAL-WA portal.A CMS was chosen rather than a static hypertext markup language (HTML) site, because the CMS provided greater flexibility in categorizing content, along with greater ease of maintenance.Initial site design was based on that of other sites that group resources by type, content, and targeted users, including HealthLinks, the HSL website ,http://healthlinks.washington.edu..
As the state agency governing the licensing of health professionals, DOH was a key player in making HEAL-WA a reality.DOH and HSL worked for several months in collaboration with the central UW technology services unit to develop work agreements that would cover the transfer of eligible users' data from DOH to UW to be used for authentication purposes, as well as the transfer of fees collected.The work agreements addressed concerns about privacy and appropriate use of user data, data transfer methods, and frequency.

RESULTS
The HEAL-WA site was launched on January 1, 2009.One year after the site's launch, approximately 4% (6,100) of roughly 150,000 eligible users had registered for a user ID.Daily site use statistics show gradually increasing traffic.Aside from the site's main page, the Physician, Registered Nurse, and Mental Health Professionals' Toolkit pages are viewed more than other pages.
Communication with providers about HEAL-WA began on an awkward note.As providers received their license renewals with the add-on fee in mid-2008, many expressed resentment about having to pay the fee and doubts about the portal's usefulness to their practice, especially because the portal was not yet available.Early information about the program from several sources had given many providers the impression that HEAL-WA would provide remote access to the UW HSL's entire electronic collection, which was contractually impossible.However, this misunderstanding contributed to professionals' antipathy toward the program and the license fee addon.Besides individual providers, the lobbyists for a few of the professional associations have been vocal and persistent with both the legislature and DOH about decreasing or eliminating the HEAL-WA fee or removing their provider group from the program.These professional groups represent, in most cases, a minority of the eligible licensed care providers in the state, but their access to the legislature is formidable and presents a significant challenge to the program.
Because HEAL-WA was a unique resource with a previously untried user model, negotiating contracts for resources presented challenges.Vendors were uncertain about how to structure pricing given the unusually large number of potential users and the unknown number that would eventually use the licensed resources.After discussion with selected resource providers, HSL was able to negotiate favorable contracts for a number of licensed summary and bibliographic databases, some of which included full-text journal access (Table 2).Negotiating access to additional full-text journals has been more difficult,  Health sciences librarians around the state expressed concerns that HEAL-WA could adversely affect their budgets or their staffing.Those reservations were amplified during the project's early stages by misunderstandings such as the impression that HEAL-WA would provide access to UW Libraries' resources.One misstep was a survey sent to physicians by the state medical association that referred to HEAL-WA, prior to staff notifying librarians that the survey was about to be distributed.This increased librarians' concerns about the effect HEAL-WA would have on their services.Communication with librarians has improved since that time.HEAL-WA has been a featured topic at several meetings of various librarians' associations.There is now an email list that staff use to inform librarians about HEAL-WA updates as well as any impending publicity about the portal.Health sciences librarians in Washington State also have access to HEAL-WA for training and demonstration purposes.
Because HEAL-WA originated with the Washington State Legislature, any changes to the program, including the amount of the fee or which professions are eligible, must go through the legislative process.Staff monitor legislative activity closely and spent significant time during the 2009 and 2010 sessions responding to legislative actions and proposed bills that could affect HEAL-WA.
User feedback has indicated the layout and structure are not intuitive for this new audience.Users' comments have focused on the difficulty of choosing among many links, uncertainty in selecting what resources to use from the available options, and inconvenience of having to perform multiple searches in different resources to find information.Given these comments, a redesign of the site is in progress that will simplify its appearance and include a prominent federated search function to facilitate users' ability to access the resources.
Although HEAL-WA had access to postal mailing addresses for all eligible professionals, publicity efforts were hindered by lack of access to email addresses for most eligible professionals.DOH uses a legacy database system for professional licensing and did not begin routinely collecting email addresses for its licensees until around the time HEAL-WA was implemented.Because there was no cost-effective, efficient way to contact every potential user and because first-year funding was directed primarily toward obtaining the resources that would populate the site, publicity during the critical first year was severely limited.A comprehensive publicity plan has now been developed to increase professionals' awareness of this resource and encourage increased use.HSL has contracted professional marketing services to implement this plan.

OPPORTUNITIES
Education and training in the use of individual resources is necessary, as the level of information literacy varies widely among providers, depending on many factors.Based on in-person and website feedback, many of the professionals who are the most vocal and resistant to paying the add-on to their license fees do not understand why evidence-based resources are of value.Staff are offering training sessions through hospitals, focusing on those institutions that do not have library services, and given the geographic dispersion of HEAL-WA users, online tutorials will be made available as well.Staff are alert to other opportunities to promote HEAL-WA as a support to the specific information needs of care providers.For example, a new initiative from the state nursing commission would reinstate a continuing education requirement for registered nurses.HEAL-WA provides access to many continuing education modules and will be promoted as a support to registered nurses in obtaining the required hours.

CONCLUSION
There are several considerations in developing a statewide program of this kind.Major considerations are the source and procedure for obtaining funding.Working directly with a state's professional licensing agency has the advantage of reaching every eligible practitioner and can work well, once procedures are developed to transmit both the designated funds and the necessary data about eligible users for authentication purposes.
Negotiations with publishers were facilitated by providing representatives with clear descriptions of the nature of the project and as much information as was available at any given time about numbers of eligible professionals, emphasizing the pilot status of the project, and highlighting the fact that user numbers would not likely approach the total number of eligible users within the first several years, if ever.A concurrentusers option was available from some publishers and was taken advantage of.Usage will be monitored, and this will be adjusted in later contracts if usage exceeds the contracted numbers.Staff will examine user numbers for all resources to compare pricing with actual use when negotiating future contracts.
Publicity should begin well ahead of the initiation of such a program, continue well after its launch, and highlight other elements, in addition to the evidencebased resources, that affected professionals might see as benefits, for example, access to patient education materials and continuing education credits.Getting support ahead of time from a more representative sampling of the affected parties would be ideal in developing similar programs and could enhance early publicity efforts.Because legislative staff worked with health professional associations to determine which provider groups would be included, and not directly with affected providers, that was not an option in this case.
Health sciences librarians throughout the state need to be considered in early publicity efforts as well.Those publicity efforts should emphasize the value- added services that professional librarians can deliver not only in clinical care but also in administrative decision making, patient safety and risk management issues, and many other areas.It has also been important to emphasize to librarians that HEAL-WA will not be made available as a substitute for an institutional purchase of resources, that institutional access to HEAL-WA will not be given, and that HEAL-WA access does not include access to UW HSL resources.
Despite the challenges encountered in developing and launching this program, HEAL-WA has enormous potential to benefit health care providers, particularly those in underserved areas, and improve patient care for the citizens of Washington State.

INTRODUCTION
The past decade has seen a major shift in the journal collections of academic health sciences libraries.Libraries gained access to vast numbers of journals as never before possible when they moved into licensing large electronic journal packages instead of the traditional title-by-title print journal selections.These changes have been a boon to researchers, who presumably no longer have to travel across campus or even down the hall to the library for articles.Instead, they retrieve articles with just a few clicks on their home or office computers.
Libraries are reducing their print collections due to decreased utilization and increased access to online journals [1][2][3].The convenience of accessing the online collection remotely instead of having to walk into the library has been cited as a reason [4].A previous study that examined the impact of online journals on the citation patterns of medical school faculty found that when medical faculty had access to a small print collection and a large online journal collection, the use of the print collection significantly decreased.However, when faculty had access to a large print collection in addition to a large online collection, the impact on the use of the existing print collection was minimal [5].A 2006 study that surveyed health sciences librarians also found that researchers are still using print journals in this electronic age [6].
The purpose of this study is to determine how online journal collections are impacting the citation patterns of researchers in dentistry, nursing, and pharmacy.Journal citation patterns before and after the introduction of online journals will be examined to determine whether researchers are more likely to limit the journal articles they cite to those journals available online rather than those available only in print.

METHODS
This study examined citation data from a large urban university with a health sciences campus that included colleges in medicine, nursing, pharmacy, public health, and allied health.A retrospective, longitudinal study compared citation patterns of dentistry, nursing, and pharmacy faculty over time to determine if the advent of online journals affected the articles cited in their published research.The time period of 1996 to 2008 allowed the opportunity to examine the use of journals prior to the proliferation of online journals and after.
At the study institution, only 15 online biomedical journals were available to faculty, staff, and students in 1998.By the year 2000, more than 3,000 journals were available online, and by 2008, more than 30,000 journals were available online.In 2004, Serials Solutions OpenURL links had been made available in the majority of the institution's online database collections, providing direct links to the online articles when available.Of the 3 disciplines studied here, dentistry had the least number of available print journals (311), compared to nursing (381) and pharmacy (517).Nursing had the largest number of available online journals (369), compared to dentistry (106) and pharmacy (271).
Searches by author affiliation were performed in Web of Science to find all articles written by faculty members from the colleges of dentistry, nursing, and pharmacy.To equalize the number of journal articles  1996, 1999, 2002, 2005, and 2008.Journals cited in these articles were separated into five categories based on their availability at the university: print only, online only, online and print, not owned, and dropped.See the methodology from De Groote's 2008 study for journal and citation inclusion criteria and an explanation of assigned journal categories [5].
Because the purpose of this study was to determine the impact online journals had on the use of the print journal collection, only print journals for which the studied library maintained continual subscriptions from 1986 through 2008 were included.Only citations to articles published in the previous ten years were examined to ensure an equitable comparison for each studied year.(The 1986 date provides ten years of citation data in 1996.)Journals that ceased or began publication during the years studied were excluded.Journals licensed online between 1999 and 2005 had to be in continuous publication between 1986 and 2008 to be included in the study.

RESULTS
The number of journal articles cited per year continued to increase each year in all of the disciplines examined in this study.The average number of articles cited per year increased the most for the pharmacy discipline: pharmacy increased from 21.32 in 1996 to 33.1 in 2008; nursing increased from 20.9 in 1996 to 25.0 in 2008; and dentistry increased from 20.5 in 1996 to 24.6 in 2008.

Dentistry
A total of 461 distinct journals were cited by dentistry researchers; 224 journals remained in the analysis because they met the study criteria.Of the 224 journals, 57 were available only in print, 3 were available only online, and 164 journals were available both online and in print.Table 1 presents the total and mean number of times journal articles were cited each year by authors in dentistry.In dentistry, where the available print journal collection was the smallest of the disciplines studied, use of the print-only journal collection decreased.The ANOVA test was performed to examine the effects of online journals on the citation patterns of authors for this discipline.There was a statistically significant interaction between the year of citation and the publication status (F54.485,P,0.001).The number of cited references decreased during the study years for the print-only journals, while the cited references of the online-and-print journals increased.

Nursing
A total of 665 distinct journals were cited by nursing researchers; 295 journals met the study criteria.Of the  1 presents the total and mean number of times journal articles were cited each year by authors in nursing.In nursing, the number of cited print-only journals decreased from 1996 to 2008, while the number of cited journals available in online format increased, although the change was not statistically significant.

Pharmacy
A total of 1,276 distinct journals were cited by pharmacy researchers; 515 journals met the study criteria.Of the 515 journals, 98 were available only in print, 27 were available only online, and 390 journals were available in both online and in print.Table 1 presents the total and mean number of times journal articles were cited each year by authors in pharmacy.
In pharmacy, where the largest number of journals was available in print, there was no statistically significant change in the number of print-only or online journals cited over the years studied.Pharmacy was the only discipline where the number of printonly journals cited increased over the years, although the change was not statistically significant.

DISCUSSION
The number of journal articles cited over the years increased from 1996 to 2008 for all of the disciplines studied.It is possible that online databases such as PubMed, CINAHL, Web of Science, and EMBASE facilitate awareness of potential research articles and have contributed to the increase in articles being cited over time.
The average number of journal articles cited per journal varied by discipline.While dentistry and nursing appeared to rely more on the journal articles published in an online format, pharmacy researchers continued to cite articles from titles available to them only in print.Citation of print-only articles increased over time for the articles produced by the pharmacy researchers.In the 2008 article examining the citation patterns of medical researchers at the same institution, it was observed that when researchers had access to a large print journal collection, their use of the print journals did not decrease [5].This may explain why in pharmacy the use of the print collection did not decrease over time.Having a large number of journal resources available only in print may make a trip to the library for research purposes worthwhile, particularly when there is a potential for obtaining multiple materials.
Again comparing to the earlier study [5], the small size of the print dentistry collection might be responsible for the decrease in its use; however, there were additional factors of interest.The decrease in the print dentistry journal collection occurred despite the fact that the number of print-only journals remained larger than the online dentistry collection.Also of note was that the dentistry authors cited more online journals than were provided for dentistry through the institution's online subscriptions intended for dentistry.The institution had approximately 134 online dentistry journal subscriptions, but a total of 167 online journals were cited.This would suggest a willingness to use the research materials from other disciplines.One study did note that as researchers continued to grow in their reliance on online resources, there was ''a relatively high use of interdisciplinary references'' [7].
It cannot be fully concluded if the observations made in this study were a result of disciplinary differences or if they were a result of the size of the collection provided to the authors.For example, both dentistry and pharmacy had small online collections, compared to the print collections.It is unknown if the limited number of online journals in the discipline is a result of the overall availability of electronic journals or a limit in the subscriptions of the institution.Therefore, it is not possible at this time to generalize the results to other institutions.Nonetheless, while the dentistry print-only collection was the smallest of the three and the online journal collection was also not as substantial as the nursing and pharmacy collections, the number of online journals cited by dentistry researchers increased significantly.At the same time, pharmacy continued to cite the print-only journals at the same rate throughout the study period.Further study is needed to determine how the size of a collection and easy access to online journals in other disciplines impacts the cross-disciplinary or multidisciplinary journals cited.

CONCLUSION
The number of articles cited by each of the disciplines has increased from 1996 to 2008.In some disciplines, the impact of online journals can be seen in the citation patterns of research faculty.Dentistry and nursing faculty appear to be less frequently citing journals that are available only in print.Pharmacy citations do not exhibit the same citation pattern.Medical faculty also do not appear to be less frequently citing print-only journals when the available print collection is large [6].While the decreased use of the physical print collection is not in question, some specific uses of the print collection, such as for research purposes, are impacted in different ways and to different extents, depending on the discipline of the user. .CLHS-R is a regional academic health sciences library for the University of Illinois at Chicago (UIC).The mission of the UIC University Library includes ''ensuring that faculty, students and other users have access to a broad and diversified range of scholarly resources'' and ''engaging in outreach to the community'' [1].In keeping with this mission, CLHS-R welcomes members of the community into the library and provides borrowing privi-leges to Illinois residents with proper identification.The collection at CLHS-R primarily focuses on the curriculum and research needs of the faculty, students, and staff from the colleges of medicine, nursing, and public health.The secondary focus of the collection is on consumer health.At CLHS-R, the consumer health collection has been pulled from the regular collection and resides in a quiet reading area in the back of the library that offers privacy.Providing health information to consumers is a common practice among academic health sciences libraries [2,3].The purpose of this study is to examine the trends in reference services usage provided by CLHS-R from 1990 through 2009.Numerous articles can be found in the literature discussing the most efficient method of staffing the reference desk [4,5].Many studies have also asked if the reference desk should be eliminated completely [6][7][8].At CLHS-R, the traditional reference desk has been phased out even though patrons still walk in the door and need assistance.Using a triage approach, patrons come to the user services desk to request assistance.The library staff member at the user services desk determines the difficulty of the question and handles the query directly or refers the patron to a reference librarian.Training of the library staff is critical so that the reference questions are handled properly.
Over the past two decades, the method of providing reference services has changed dramatically [9][10][11][12].The explosion of the Internet has changed the way libraries provide reference services today [13][14][15][16].Overall traditional reference transactions have declined, while electronic reference questions have increased [17].The author asked, ''Who is really using the reference services at CLHS-R and why?''Knowing who is using reference and why should help reference librarians plan for the future.

METHODS
Reference statistics were gathered monthly by the entire library staff from 1990 through 2009.A standardized form was used to record the number of transactions with any faculty, staff, students, or members of the public who used library services inperson or by telephone, email, or chat.The reference statistics form had a section to indicate types of reference transaction questions that included the options: directional, ready reference, in-depth reference, and demonstrations/consultations.Directional questions referred to questions regarding location of services, policies, collections, and materials contained in the building or university.Ready reference questions were questions that could be answered quickly in five minutes or less.In-depth reference questions were those that required more than five minutes to answer and/or involved the use of several resources.Demonstrations/consultations included demonstrations of one or more print or online resources or consultation sessions on conducting research on a project or topic and might be individual or small group.This form also included a section to indicate This article has been approved for the Medical Library Association's Independent Reading Program ,http://www.mlanet.org/education/irp/..

RESULTS
The utilization data showed an intermittent decline in the aggregate number of reference transactions requested over the entire 20-year time frame.The total number of reference transactions for 1990 was 8,738, but in 2009 it was 2,505.The decline in the totals for each year was more pronounced over the most recent years, 2007 through 2009.The majority of these questions fell into the category of directional and ready reference, with each averaging 37% of the total amount of reference questions.Each year, the consultation/demonstration category had the fewest numbers, averaging only 5% of the total.The in-depth questions for the 20-year period averaged 21% of the total number of reference transactions.Other than the overall decline, there was no consistent pattern of usage for the different question types during the years of this study.The others user group accounted for the most reference transactions as a percentage of the total, averaging 42% during the 20-year period.In comparison, over this same time frame, UIC students accounted for 36% and the UIC faculty/staff group accounted for 26% of the total.At the outset, 1990 through 1998, others patrons were responsible for 44% to 52% of the total number of reference transactions.This number steadily declined, so that by 2007 through 2009, this group only accounted for 23% to 27% of the total number of queries (Figure 1).
The percentage of questions posed by UIC students steadily increased in over the 20-year period.By 2008, they represented the most predominant user group and reached their highest utilization level (41%).For the majority of the 20-year period, the UIC faculty/ staff group asked the lowest amount of reference

DISCUSSION
Overall utilization of reference services has decreased over the twenty-year time frame of this study and is consistent with reports in other articles discussing the issues of decreasing reference questions [18][19][20].The distribution of reference transaction by question type was consistent throughout the twenty-year study.Directional and ready reference questions were the most popular types of questions asked.Because these types of questions are answered in less than five minutes, they accumulate rapidly.The most common directional questions deal with library hours, location of a book, and printing or technical issues.Ready reference questions usually refer to which database to use to search for articles.Once a library staff member points the patrons in the right direction to begin their searches, the patrons are left to browse and search at their leisure.Library staff can easily overlook recording these types of questions because they happen so regularly and quickly.This number might be a larger percentage than actually reported.In-depth questions involve a deeper reference inquiry.Usually, the patron is beginning a research project and does not know where to start.These questions are usually referred to the reference librarians who have the expertise and training to determine how to assist the patron.The number of transactions has fluctuated over the last twenty years.This fluctuation might have occured because each incoming class of students has varying levels of computer expertise.The consultation/demonstration category was the least frequently requested.Consultation sessions are usually research sessions that are scheduled in advance.Demonstration sessions are sometimes scheduled in advance but more often are impromptu opportunities that arise to perform a oneon-one teaching session on relevant databases.These totals have stayed constant over the twenty-year time frame, with some minor decreases in 2000 and 2001.
For the first 13 years of the 20-year study, the CLHS-R addressed more reference questions for nonaffiliated patrons than UIC patrons.The percentages for the nonaffiliated patrons were between 39% and 52% during this time frame.The author believes this change was due to the more limited availability of medical information to the general public in these early years.With the introduction of the Internet and the increasing availability of medical websites, those in the others user group have gained more access to convenient consumer health information and rely less on the university library.The Rockford Public Library provides many public computer stations to accommodate those patrons who do not have a computer or access to the Internet in their homes.It also encourages members of the public to utilize the CLHS-R when searching for consumer health information.
In contrast, the UIC students, faculty, and staff usage has gained momentum.The inception of chat and email reference services has broadened the patron base to include the UIC Daley Library, Science Library, and Library of the Health Sciences at Chicago, Peoria, and Urbana.This increase of patrons across many locations might explain why the UIC student and UIC faculty/staff user groups' percentages have grown.Even though nonaffiliated patrons can use chat and email, they may not realize this option is readily available to them.Librarians promote these digital reference services to UIC students, faculty, and staff during their library orientation, tours, and database workshops.
Another factor that might be contributing to the increasing percentages in the reference transactions for UIC students, faculty, and staff is the dynamic changes in acquiring their education and research information.The adaptation of electronic resources and databases has created an immense need for library instruction.The learning curve has been extended with some of the resources requiring hands-on instruction.However, over time, the existing faculty, staff, and senior students should gain the skills and knowledge to utilize these resources and databases effectively.The need for instruction will still exist as incoming students enter the system and as new resources become available.Library instruction is a vital part of all academic health sciences libraries.One-on-one instruction provided during reference assistance is essential for all users.

CONCLUSION
The data from this twenty-year period at the CLHS-R confirm the findings of studies at many other academic libraries.Overall reference transactions have declined.This analysis has highlighted the shift in utilization by different patron types.The general public now has convenient access to the Internet to locate the consumer health information they desire from their homes.In contrast, the UIC students, faculty, and staff are consistently exposed to a dynamic environment that may require additional library instruction in obtaining their educational and research information needs.
Balancing reference services between affiliated and nonaffiliated users can be a daunting task.Knowing who is requiring the reference services and what types of questions they are asking may help improve library services by providing valuable information to the library staff.Continuing education of the library staff will help ensure the quality of assistance given to the library patrons.

INTRODUCTION
The University of Illinois Crawford Library of the Health Sciences at Rockford (CLHS-R) has been serving University of Illinois College of Medicine faculty, students, and staff and the area community since 1972 which of three categories of user groups were assisted: UIC faculty/staff, UIC students, and others.The others category included Rockford area community residents and nonaffiliated students from local universities, community colleges, or high schools.

Table 1
Summary of survey results

Table 1
HEAL-WA numbers of eligible versus registered users, April 2010
Brief communications: Lawrence et al.compared in each discipline, an equal number of articles were randomly selected from the total number of available articles.The random sample of articles was based on the lowest number of articles published each year in a discipline.The searches were conducted for the following years:

Table 1
Articles cited* by status of cited journal and year of article publication 295 journals, 52 were available only in print, 8 were available only online, and 235 journals were available in both online and in print.Table