A case study : planning a statewide information resource for health professionals : an evidence-based approach

Methods: In January 2008, the authors began planning a statewide eHL for health professionals following the five-step process for evidence-based librarianship: formulating the question, finding the best evidence, appraising the evidence, assessing costs and benefits, and evaluating the effectiveness of resulting actions. Main Results: The authors identified best practices for developing a statewide eHL for health professionals relating to audience or population served, information resources, technology and access, funding model, and implementation and sustainability. They were compared to the mission of the eHL project to drive strategic directions by developing recommendations.


SETTING
The eHL project was coordinated by the HSL in close partnership with Minitex.The HSL plays a prominent role in providing health information outreach to the state.It supports the goal of the University of Minnesota Academic Health Center to ''expedite the dissemination and application of new knowledge into the promotion of health and delivery of health care in Minnesota'' [1].The HSL is a Resource Library for the National Library of Medicine's National Network of Libraries of Medicine, with an additional designation as an Outreach Library.The HSL also serves the state through its support of the Academic Health Center's land grant mission, which includes the primary role of educating health care professionals and generating and disseminating new knowledge to improve the health of Minnesotans [2].
Minitex is a publicly supported network of academic, public, state government, kindergartenthrough-twelfth-grade school, and special libraries working cooperatively to improve library services for their users in Minnesota, North Dakota, and South Dakota.It is an information-and resource-sharing program of the Office of Higher Education and the University of Minnesota Libraries that is funded by the Minnesota Legislature.The Minnesota State Library Services, a unit of the Minnesota Department of Education, provides additional funding to support services for Minnesota libraries.
With the combined goals of disseminating new knowledge to promote health and improving library services for Minnesotans, a partnership between the HSL and Minitex was a natural fit for developing an eHL for the state.The HSL's partnership with Minitex provided the added benefit of building on an established infrastructure made available through the Electronic Library for Minnesota (ELM) ,http://www .elm4you.org.. ELM is an information portal that provides streamlined access to information resources for Minnesotans via statewide Internet protocol (IP) authentication.
The partnership between the HSL and Minitex provided sufficient capacity to support an eHL for the state.The timing for the project was also found to be appropriate in terms of infrastructure, need, stakeholder support, and strategic alignment.Regarding infrastructure, research has indicated that the state's health professionals and Minnesota households have sufficient access to computers and the Internet [3,4].Regarding need, inequalities in patient care have been seen both nationally and throughout Minnesota [5][6][7][8].Additionally, the state's 40,000 health professions students and 160,000 licensed health professionals have benefited from different levels of access to evidence-based clinical information, depending on their institutional affiliations [9,10].Regarding stakeholder support, the concept of a statewide eHL has received strong support from university administration, health care organizations and representatives, and health sciences and other libraries across the state.Finally, regarding strategic alignment, the eHL project aligned with several health care reform initiatives in the state during 2007 and 2008 [5,11,12].The final reports from these initiatives incorporated language that related to the importance of evidence-based health information, the involvement of patients in the health care process, and the goal of implementing a statewide electronic health record system that could serve as a delivery mechanism for eHL materials.This evidence was taken into account when determining the viability for an eHL in Minnesota as each of these aspects would impact its ultimate success.

METHODOLOGY
The eHL project activities followed the five-step evidence-based librarianship (EBL) process as defined by Eldredge [13]: 1. formulate a clearly defined, relevant, and answerable question; 2. search for an answer in both the published and unpublished literature, plus any other authoritative resources, for the best available evidence; 3. critically appraise the evidence; 4. assess the relative value of expected benefits and costs of any decided upon action plan; and 5. evaluate the effectiveness of the action plan.

Formulating the question
The question formulation process was guided by the EBL setting, perspective, intervention, comparison, and evaluation (SPICE) template for question building [14].Using this framework, the following question was developed to guide the eHL planning process: What is the best model for providing equitable access to relevant information resources for all health professionals in Minnesota, as compared to the best practices used by existing area-wide eHLs from outside of the state, that would align with local needs and resources?
In this case, the setting is Minnesota, the perspective is health professionals, the intervention is the model of equitable access, the comparison is existing best practices, and the evaluation is alignment with local needs and resources.

Finding the evidence
After developing a structured question to guide the EBL process, work was done to find relevant evidence.Because evidence was lacking in traditional publication venues, a multistep process was employed to locate other information sources that could address the original question.This included an environmental scan and an information resource assessment.These tasks were aimed at gathering best practices in the following areas: audience or population served, information resources, technology and access, funding model, and implementation and sustainability.

Performing an environmental scan*
The environmental scan was a two-step process that included a competitive analysis and a questionnaire that surveyed health sciences librarians about projects that license clinical information for unaffiliated health professionals [15].For the competitive analysis, a review of projects that provided area-wide services to non-affiliates was completed.A total of nine projects were identified through an Internet search using selected keywords with the Google search engine (Table 1).The project websites were reviewed to gather information related to population served and eligibility, available information resources, technology and access restrictions, and funding models.This information was placed in a matrix, and an additional column was added to capture information relating to the implementation and continued sustainability of eHLs.A literature review was also conducted as part of the competitive analysis to identify information that would supplement the findings from the review of project websites.A total of twelve articles were located by searching library literature databases (Library, Information Science & Technology Abstracts, Library Literature & Information Science) and medical literature databases (PubMed), as well as gray literature (Google Scholar), on keywords related to the selected projects (e.g., AZHIN, Ohio-LINK) [15].New information located through this process was added to the competitive analysis matrix in the categories described above.
For the second part of the environmental scan, a questionnaire was developed to serve as an additional means of gathering information about statewide initiatives outside of Minnesota that license clinical information for unaffiliated health professionals [16].The 36-question instrument focused on best practices in the following categories: population served, information resources, technology and access, funding models, and sustainability.An additional comments section was included to gather information not addressed in the structured questions.This questionnaire was reviewed and approved by the HSL * Supplemental material can be found in the My Health Minnesota: Electronic Health Library collection, housed in the University Digital Conservancy, the University of Minnesota's Institutional Repository ,http://conservancy.umn.edu/handle/47090/browse-title.. Collection documents include the environmental scan (which incorporates the competitive analysis), the information needs assessment, the best practices survey, a promotional information sheet, the request for information, and the final report.
A case study director.Institutional review board approval for the survey protocol was granted through the University of Minnesota's Office of the Vice President for Research, and the questionnaire was distributed via a web-based survey tool.Respondents were recruited based on their membership in the Association of Academic Health Sciences Libraries (AAHSL).The AAHSL email discussion list was used to send out an announcement of the upcoming survey, the questionnaire, and 2 reminder notices.This procedure followed the Dillman total design survey method, modified to accommodate a quick turn-around time [17].Forty-eight of the 143 AAHSL member institutions responded to the survey, for a response rate of 33%.Following the close of the survey, aggregate and individual data were generated using the web-based survey tool.The survey data were analyzed by the project manager.Key findings were summarized, and identifiable information was removed before it was distributed to the project team and the AAHSL email discussion list in a final report [16].

Analyzing information resources
As part of the information resource analysis, a questionnaire was developed to identify a set of resources that would meet the information needs of the state's health professionals [18].The methodology for the information resources survey matched that of the best practices survey described above, including the process for the survey design, approval, implementation, analysis, and distribution of results.The 19-question instrument focused on topics related to project support, current resource usage, information needs and gaps, and best practices, and it included a comments section to gather information not addressed in the structured questions.Respondents were recruited based on their membership in the Health Sciences Libraries of Minnesota (HSLM) association.Thirty-five of 71 HSLM members responded to the survey, for a response rate of 46%.
The information resources survey results were combined with the findings of a 2005 survey on the information needs of community-based preceptors working for the University of Minnesota Academic Health Center [3].Questions in this survey related to the value of access to online resources, resources currently available, and ranking of specific resources.The nearly 500 respondents represented a broad range of health professionals, including family practitioners, nurses, physical and occupational therapists, genetic counselors, medical technicians, and pharmacists.Additionally, usage statistics were analyzed to identify the high-use resources that the HSL's patrons access.
The findings from the environmental scan and information resource analysis were used to identify resources, access, pricing, and technology requirements that addressed local needs.Emails were then sent to vendors to inquire about their ability to meet these criteria.Vendors had varied responses to the emails.While some expressed initial support, others voiced concerns about losing individual subscriptions, incurring liability for providing clinical information to health consumers, and controlling access.In some cases, initial emails were followed by conference calls with vendor representatives to provide additional details and to address concerns.After the initial and follow-up conversations with vendors, a request for information (RFI) was issued through the University of Minnesota's Purchasing Services to gather structured information on the capacity of information vendors to respond to the eHL's specific project requirements [19].

Best practices
The multistep process used to gather evidence related to eHL implementation helped identify best practices in the areas of population served or audience, technology and access, funding model, and implementation and sustainability (Table 2).In summary, it was found that eHLs typically serve health professionals based on institutional affiliation, that the service is usually provided by academic health sciences libraries, that a mixed model approach is typically used to fund eHLs, and that eHLs should be supported by two to five full-time equivalents and employ a governing body for oversight.Through the RFI process, it was found that several information vendors were able to respond favorably to the project requirements that reflected the information needs of health professionals in the state, as well as specific technology, access, and cost requirements.The criteria included in the RFI stated that the suite of resources provided by the vendors must: & represent the needs of a broad range of health professionals (physicians and nurses in particular); & provide access to the following categories of resources: evidence-based medicine and evidencebased nursing point-of-care products, clinical drug references, full-text medical and nursing electronic journals, full-text medical and nursing electronic books, general medical and nursing bibliographical databases, and the Cochrane Library; & be provided at a realistic and feasible cost, to be evaluated based on the annual price per health professional or health professions student user; & be accessible via the ELM portal's statewide IP authentication system; & allow unlimited access or access for a large number of concurrent users; and & be reasonable for a staff of one to two full-time equivalents to manage (Note: one to two full-time equivalents were detailed in the RFI, as opposed to the two to five recommended by best practices because a portion of the workload would overlap with the current responsibilities of the HSL and Minitex staff) [19].

Appraisal of the evidence
Once the best practices were identified, work was done to appraise the evidence.The appraisal was conducted by comparing the best practices with the mission of the eHL, while taking into consideration the project team's expertise as health information professionals working in Minnesota.During the appraisal process, it was found that the timing for eHL implementation in Minnesota was appropriate in terms of capacity, infrastructure, need, stakeholder support, and strategic alignment, as described in the ''Setting'' section of this case study.However, the best practice of providing access based on institutional affiliation would significantly limit the project's mission of providing equitable access for all of Minnesota's health professionals.Additionally, institutional-based access would limit use by the state's health consumers.As a result, this best practice would not support the project's goal of providing accurate health information to the citizens of the state so that they can become engaged in the patient-care partnership and make informed health decisions.
The findings of the appraisal process led to the formation of recommendations that were based on the best practices gathered through the EBL process, modified to reflect the project goals and knowledge of the local environment (Table 3).Most notably, the eHL project team recommended that resources should be selected based on the needs of the project's primary audience, the state's licensed health professionals, but that access should be granted to all health practitioners, researchers, and students, regardless of institutional affiliation, and to all of Minnesota's five million citizens.

DISCUSSION
The EBL process of formulating the question, finding the best evidence, and appraising the evidence provided sufficient data to develop recommendations A case study for eHL implementation in Minnesota.However, as indicated by EBL guidelines, it is important to examine the value of the data by assessing costs and benefits and evaluating the effectiveness of resulting actions.

Assessing costs and benefits
With recommendations in hand, work was done to evaluate their feasibility through a cost-benefit analysis.The eHL promised many positive results in terms of social benefits.These included contributing to an integrated and evidence-based health care system in Minnesota, which would enable patients to become partners in care, support continuous learning through the education process and into clinical practice, encourage recruitment and retention of Minnesota's rural health professionals, and support collaborative and practice-based health research, among others.The eHL would also have positive financial benefits as it would leverage economies of scale and buying power for equitable access to resources, leverage existing investments in information management and technology provided by the HSL and Minitex, and ultimately, reduce health care costs.In addition to these social and financial benefits, it was determined that eHL implementation would carry little risk, as it was supported by best practices, reflected the needs of the state, and was fiscally responsible.However, the eHL recommendations were built on certain assumptions that would have to be addressed to ensure successful implementation.This was done by completing a risk assessment that detailed project assumptions along with their related mitigating actions and dependencies (Table 4).

Evaluating the results
The planning process for the eHL concluded in September 2008, with implementation set to begin in spring 2009.The timetable incorporated a 2-phase implementation schedule.Phase I of the plan was estimated to cost $1.2 million a year, based on the numbers provided by vendors in their responses to the RFI.With approximately 200,000 primary users of eHL resources (i.e., licensed health professionals and enrolled health professions students), this equated to an estimated annual cost per user of $6.00.These funds were to be requested for a 2-year period from Minnesota's health systems with additional contributions from the University of Minnesota Academic Health Center.This shared cost-funding model would have supported the licensing of both an evidencebased medicine and an evidence-based nursing product that would be made available to Minnesota's health professionals and to every citizen of the state.
The two years of funding provided by Minnesota's health systems would have allowed for an outcomes- based evaluation of the eHL, determining its successes and areas for improvement.The use of a proof-ofconcept approach that would incorporate evaluation measures would illustrate the value of the eHL to the state's governing bodies and help prepare for phase II of the project, which would include a legislative proposal for recurring state funds in 2011.
The timing for the two-phase implementation plan coincided with the severe economic downturn that occurred in fall 2008 that affected not only the university, but the state's health systems as well.As a result, the implementation process has been delayed until the state's economy proves to be more stable.
While the delayed implementation of the eHL limited formal evaluation of outcomes, the EBL process was deemed successful for several reasons.First, it was found that the question formulated by following EBL guidelines was indeed answerable.Second, by comparing the findings with professional expertise and the mission of the eHL project, recommendations were generated that aligned with local needs and resources, thereby satisfying the evaluation measure identified in the original EBL SPICE question.Finally, the recommendations formed the basis of a plan that was sensitive to possible risks, that has proved to remain relevant over time, and that can be put into practice when resources become available.

CONCLUSION
Through the EBL process, data were gathered that strengthened the authors' confidence in their ability to select, license, and deploy a suite of electronic resources that aligned with best practices and met local needs.The evidence-based feasibility study also showed the importance of integrating sustainability planning into an eHL project in order to support longterm success.This was done by engaging stakeholders in the planning process, ensuring adequate capacity and infrastructure to support an eHL, and aligning the proposal with statewide health initiatives.As resources become available to implement Minnesota's eHL, additional work will be done to evaluate the program and illustrate the extent to which equitable access to clinical information can support quality, cost-effective health care.

Mitigating actions Dependencies
The partnership between the HSL and Minitex will continue.
Development of a memorandum of understanding to be signed by both partners.
Partners can identify agreeable terms.
Resources will be made available through the ELM portal, operated and administered by Minitex.
Identification of vendors who can work within the eHL model.
Vendors are willing and able to provide access according to the eHL's ''equal access'' model.Recurring funds for the eHL will be secured.
Development of a legislative proposal for eHL support.
Buy-in from key stakeholders in the academic health center, the University of Minnesota, and elsewhere.The funding model will cover the costs of resources and staff.
Incorporation of salary for 2 full-time equivalents; negotiation of fixed pricing, secured through a formal request for proposal (RFP) process.
Resource pricing and salary estimates are accurate and will remain relatively stable over time.
Resources will be licensed for an annual cost of $5-$10 per health care user, which is equivalent to a total annual cost of $1-$2 million.
Selection of resources to match projected costs.
Vendor quotes align with cost estimates.
The eHL will work under a model of shared governance, made up of a steering committee and subcommittees.
Identification of the makeup of the steering committee and initial contact to test feasibility.
Support from health care organizations, health professions programs, and health sciences and other libraries across the state.User support will be coordinated by the HSL but carried out by partner libraries in Minnesota.
Development of an informal partnership agreement that identifies an eHL champion in each partner organization.
Support from health sciences and other libraries across the state.

Table 1
Electronic health library (eHL) competitive analysis projects

Table 2
Best practices for eHLs Audience or population served & eHLs typically restrict access to a defined set of health professionals and often require that individuals belong to a member institution to benefit from services.Information resources & eHLs typically license resources that are evidence based and that support the information needs of a broad range of health care providers.Technology and access & eHLs are typically coordinated by academic health sciences libraries.& Onsite access is typically granted to member organizations through Internet protocol (IP) verification.& Offsite (remote) access is typically provided by member organizations to their affiliates through the use of a proxy server, when available.Funding model & eHLs are typically funded using a mixed-model approach, with revenue coming from membership fees, grants, and government funding.& eHLs are typically more sustainable over time when they receive the bulk of their financial support from recurring state funds, though they often supplement costs in other ways (e.g., grants, membership fees).& eHLs have typically received recurring state funds from departments of health and/or education.Implementation and sustainability & eHLs would ideally employ between 2 and 5 full-time equivalents.& eHLs are typically guided by governing bodies, with work being carried out by the project team and by subcommittees working toward a specific charge.& eHLs typically provide value-added services, either for free or at an added cost.

Table 3
Minnesota eHL recommendations The eHL should be made available to all of Minnesota's health professionals (i.e., health practitioners, researchers, and students) regardless of their institutional affiliation.& The eHL should be made available to all of Minnesota's 5 million citizens.& The primary audience for the eHL should be all licensed health professionals in Minnesota.*Information resources & eHL resources should be relevant to health professionals.Resources that are specifically aimed at a general (consumer) audience should not be considered.*& eHL resources should represent the needs of a broad range of specialists (e.g., administrators, physicians, nurses, pharmacists, mental/behavioral health specialists), and physicians and nurses in particular.*& eHL resources should support evidence-based practice.*& Specific resources should be selected with input from stakeholders.& The eHL should provide access to following categories of resources: evidence-based medicine/point-of-care product, evidence-based nursing/point-of-care product, clinical (professional-level) drug reference, full-text medical and nursing electronic journals, full-text medical and nursing electronic books, general medical and nursing bibliographical databases, and the Cochrane Library.Technology and access & The University of Minnesota University Libraries, represented by the Health Sciences Libraries (HSL) and Minitex, should coordinate statewide access to resources.*& The eHL should be made available through the Electronic Library for Minnesota (ELM), which is operated and administered by Minitex.& eHL resources should support IP access.Resources requiring individual login/password should not be considered.*& eHL resources should allow access for every citizen of Minnesota.Resources that limit use to health professionals or health professions students should not be considered.& eHL resources should allow unlimited access or access for a large number of concurrent users.Funding model & The funding model should cover the costs of the licensed resources, as well as the staff to support initial implementation and long-term maintenance.& A mixed funding model with multiple revenue streams should be employed to support the eHL.* & Recurring state funds should be sought as one eHL revenue stream.*& Varying methods of recurring funds should be explored with stakeholders to see which model(s) might be best received by the legislature and other funding agencies.& Selected information resources must be provided at a realistic/feasible cost, to be evaluated based on a $5.00-$10.00annual price per health professional/health professions student.& License agreements should be negotiated for a 3-5 year period to leverage buying power.& The funding model should include support for maintaining Go Local as part of the My Health Minnesota suite of resources.Implementation and sustainability & The eHL should be supported by a minimum staff of 2 full-time equivalents.*& The eHL should be guided by a governing body made up of key stakeholders.*& The work of the eHL should be carried out by subcommittees with representatives from medical and other libraries.* * Based on best practices.Aspinall et al.