Making meaningful connections: evaluating an embedded librarian pilot project to improve nursing scholarly writing *

The Health Sciences Library, University of Tennessee Health Science Center (UTHSC), serves 6 colleges (medicine, nursing, dentistry, pharmacy, allied health, and graduate health sciences) consisting of approximately 2,000 students and 800 faculty and researchers. The library liaison program has been in place for years, whereby each librarian is assigned as the key contact between the library and campus colleges and is readily available to answer questions in various ways (e.g., telephone, email, text message, and online chat). However, the use of traditional library service is declining, which signifies changing needs that lead to new ways to provide library services 1. Embedding librarians in academic courses might be one of the approaches to meet the changing needs 1. 
 
The term “embedded librarian” describes librarians' involvement at macro (college, department, or program) and micro (course) levels 2. Key characteristics of embedded librarians include going out of the library and into library user groups, building strong relationship, providing customized and value added services, working as team members and collaborators, and contributing to the success of ongoing projects 3. Kesselman and Watstein described “embedded” as intensive integration into the user group, where patrons have multiple opportunities to interact with the librarian 4. They summarized embedded librarianship with two important factors: “integration and collaboration” 4. 
 
A search of the literature found numerous publications reporting the integration of librarians into nursing courses and their collaboration with nursing faculty to assist with student learning and information literacy. Librarians at the Welch Medical Library integrated information resources and library instruction throughout the Johns Hopkins University School of Nursing curricula 5. Dorner, Taylor, and Hodson-Carlton described their collaboration with nursing faculty to design a tiered approach to building nursing student research skills 6. Librarians at Himmelfarb Health Sciences Library, George Washington University, have been embedded in online classes in its School of Nursing and School of Medicine and Health Sciences and are providing library services to their distant students 7. At Auburn University Montgomery, a librarian and the School of Nursing faculty collaborated to create a library database search module and integrated it into a course, “Computers in Nursing,” to enhance students' database searching skills via their course management system 8. Carlock and Anderson reported their instructional program whereby a librarian worked with nursing faculty to teach students evidence-based practice searching skills 9. However, only a few articles used the term “embedded” to describe the librarian's involvement and participation in nursing courses 7, 10, 11. Few articles were found in the literature specifically reporting librarians integrated into a graduate nursing online course to provide in-depth research and writing assistance at the point of need. Few high-quality research studies reported the outcomes of embedded programs 12. 
 
This paper describes an embedded librarian pilot project and seeks to evaluate the project's benefits and value from multiple outlets. It provides a model that can be used by others to develop and provide much needed evaluation of the embedded librarian program. Lessons learned and challenges are also discussed.


INTRODUCTION
The Health Sciences Library, University of Tennessee Health Science Center (UTHSC), serves 6 colleges (medicine, nursing, dentistry, pharmacy, allied health, and graduate health sciences) consisting of approximately 2,000 students and 800 faculty and researchers. The library liaison program has been in place for years, whereby each librarian is assigned as the key contact between the library and campus colleges and is readily available to answer questions in various ways (e.g., telephone, email, text message, and online chat). However, the use of traditional library service is declining, which signifies changing needs that lead to new ways to provide library services [1]. Embedding librarians in academic courses might be one of the approaches to meet the changing needs [1].
The term ''embedded librarian'' describes librarians' involvement at macro (college, department, or program) and micro (course) levels [2]. Key characteristics of embedded librarians include going out of the library and into library user groups, building strong relationship, providing customized and value added services, working as team members and collaborators, and contributing to the success of ongoing projects [3]. Kesselman and Watstein described ''embedded'' as intensive integration into the user group, where patrons have multiple opportunities to interact with the librarian [4]. They summarized embedded librarianship with two important factors: ''integration and collaboration'' [4].
A search of the literature found numerous publications reporting the integration of librarians into nursing courses and their collaboration with nursing faculty to assist with student learning and information literacy. Librarians at the Welch Medical Library integrated information resources and library instruction throughout the Johns Hopkins University School of Nursing curricula [5]. Dorner, Taylor, and Hodson-Carlton described their collaboration with nursing faculty to design a tiered approach to building nursing student research skills [6]. Librarians at Himmelfarb Health Sciences Library, George Washington University, have been embedded in online classes in its School of Nursing and School of Medicine and Health Sciences and are providing library services to their distant students [7]. At Auburn University Montgomery, a librarian and the School of Nursing faculty collaborated to create a library database search module and integrated it into a course, ''Computers in Nursing,'' to enhance students' database searching skills via their course management system [8]. Carlock and Anderson reported their instructional program whereby a librarian worked with nursing faculty to teach students evidence-based practice searching skills [9]. However, only a few articles used the term ''embedded'' to describe the librarian's involvement and participation in nursing courses [7,10,11]. Few articles were found in the literature specifically reporting librarians integrated into a graduate nursing online course to provide in-depth research and writing assistance at the point of need. Few highquality research studies reported the outcomes of embedded programs [12].
This paper describes an embedded librarian pilot project and seeks to evaluate the project's benefits and value from multiple outlets. It provides a model that can be used by others to develop and provide much needed evaluation of the embedded librarian program. Lessons learned and challenges are also discussed.

DESCRIPTION OF THE EMBEDDED LIBRARIAN PILOT PROJECT
Over the years, the UTHSC library liaison's involvement and participation in nursing courses has gained trust from the nursing faculty. In late 2011, the nursing faculty proposed a pilot project to embed a librarian in an online graduate course, ''Professional Issues,'' the first writing-intensive course in the master's of science in nursing (MSN)-clinical nurse leader (CNL) curriculum. This seven-week, accelerated, online course was taught in summer 2012. The faculty wanted to improve the students' experience, outcomes, and information literacy skills in courses with a research and writing-intensive component and sought a model that would integrate course research and writing assignments with library resources. A librarian embedded within the course would have a unique perspective of the students' specific needs for literature, research, and writing skills. Additionally, an embedded librarian would enable faculty to focus more on teaching the nursing content of the course. The embedded librarian's role was clarified, and the embedded services were added to the course syllabus: & providing assistance and training on locating, retrieving, and evaluating information & working with course instructors to deliver specialized lectures & assisting with the use of information management tools such as EndNote & answering questions related to searching, research, writing, and using the Publication Manual of the American Psychological Association (APA), sixth edition & linking library resources to the course by creating a course research guide

METHODS
The participants consisted of seventy-seven first-year nursing students enrolled in the MSN-CNL program. The project was guided by the question: ''Is the embedded librarian pilot project effective and valuable?'' Data collection used multiple methods: course evaluation, pre-and post-survey questionnaires, a content analysis of email questions, a comparison of students' scores on the two written papers, and usage statistics for the course research guide.
The course evaluation was built into Blackboard (BB), a course content management system. The evaluation asked students two questions: (1) list three strengths of the lecture course, and (2) list three weaknesses of the lecture course and three suggestions for improvement.
Pre-and post-survey questionnaires were created with Google Forms and were distributed to students via BB and email at the beginning and end of the course. Excel was used to collect survey data. The presurvey contained fourteen questions (Appendix A, online only); the post-survey asked fifteen questions (Appendix B, online only). Both surveys included six pairs of questions asking students to self-rate their skills and knowledge in conducting health research, searching databases, and using APA format to write papers. In addition, the pre-survey collected demographic information and asked whether students had experiences with a librarian embedded in any previous academic course, how many scholarly papers they published, and what they would expect an embedded librarian to do. The post-survey asked whether students used the embedded librarian services and whether the services met their expectations.
Both surveys provided open-ended questions for students' input and comments on the embedded services.
Students were encouraged to send their questions to the embedded librarian via email, which were recorded in an Excel spreadsheet. A content analysis of the email questions was conducted using NVivo, version 10, qualitative research software.
Students' scores on their first and second written papers were compared to see if embedded librarian's services made an impact on their score improvement. The APA style manual, sixth edition, was used to grade course assignments. The first paper provided an opportunity for students to state their own philosophy of nursing and their hopes or visions for the profession through the lens of the CNL. The second paper required students to provide a description of advance directives for end-of-life decisions in the health care context and concluded with implications for the CNL as a member of the health care team. Targeting students' commonly made APA errors from their first paper, the librarian worked with the course faculty to deliver a face-to-face presentation, to organize a question-and-answer session, and to review students' APA references and citations on their second papers.
The course research guide ,http://libguides.uthsc .edu/nsg510/. was created using LibGuides, a content management and publishing system created by SpringShare.com. The guide was created based on the course content, student needs, and faculty suggestions. Usage of the course research guide was tracked to see which section was used or accessed frequently.
Instruments used in this project were reviewed and approved by the UTHSC Institutional Review Board (IRB).

RESULTS
Fifty-nine (77%) out of 77 students voluntarily completed the course evaluation. Of the 59 responding students, 23 (39%) answered the first question. They pointed out that the embedded librarian was one of the strengths of the course: ''Love the librarian! She is amazingly helpful and accessible.'' ''I enjoyed having the librarian be involved, very helpful!'' ''The on-site APA reference tutoring session with the librarian was helpful!'' Of the 77 students, 15 (19%) completed the presurvey and 19 (25%) responded to the post-survey. In the post-survey, 17 (89%) students reported that they used the embedded librarian services. Students identified librarian's assistance related to writing, research, and APA as the most important aspect of the pilot project. In the pre-survey, 8 (53%) of the students rated their skills and knowledge in using APA for scholarly writing as poor. In the post-survey, 18 (95%) rated their skills and knowledge in using APA as above average or average. In the post-survey, 13 (68%) of the students rated their skills and knowledge as excellent or above average in conducting health research and searching scholarly databases.
Student comments from the post-survey indicated that the pilot project was well received. One student commented, ''I thought it was a great idea and would like to see the embedded librarian used in every class where we are required to do research and write papers.'' Survey results also suggested that students needed help in using library resources and searching for information. One student suggested creating tutorials on how to perform a concise and productive library and/or web search.
During the course, the research guide got a total of 813 hits. Except for the welcome page, the APA frequently asked questions (FAQ) page, which was created based on students' APA questions, was accessed most often. One student suggested keeping the website up and running for future classes and updating it as necessary. The course faculty expressed gratitude to the embedded librarian: ''This is simply wonderful!!! You are such an asset.'' Comparing students' scores on their first and second papers revealed that students' scores improved in their second paper. Sixty-one (79%) students attended the librarian's presentation, and 46 (75%) of them increased their scores in their second paper. Among the 30 (39%) students who emailed the librarian for assistance, 19 (63%) increased their scores in their second paper. One student commented in her email to the course instructor, ''I met with the librarian today and she was able to point out where I lost the additional points in APA formatting. I am certain this coming one will be more accurately written to earn an A!'' This student did receive an A on her second paper. The course faculty members were pleased with overall student score improvement, which demonstrated that the librarian's assistance at the point of need did improve students' learning outcomes.

DISCUSSION
The multi-method evaluation of the pilot project suggested that the embedded librarian service was effective and valuable. Students' feedback indicated that library instruction and services would be more effective when provided at the point of need. This is especially true when students had a special need for their research projects or papers. The project findings support the statement that ''research-intensive courses are naturally the most practical for librarians to provide support throughout the entirety of the class'' [1]. As no rigorous publication evaluating the impact of embedded librarianship was found in the literature [12], this multi-method evaluation of the embedded librarian pilot project has contributed greatly to understanding the effectiveness of embedded librarianship, especially in the context of health sciences libraries.
The study has some limitations. The low response rate from the pre-and post-survey questions might not reflect the thinking of the majority of students regarding improvements they made during the course. The anonymity of the pre-and post-survey made it impossible to pair the responses with improvement in individual students' self-rated skills and knowledge.
For the embedded librarian, this experience was intensive at times and required a considerable time commitment. Table 1 details the tasks, activities, and estimated time and effort for the embedded librarian to prepare and implement the project for this writing course. Table 1 also provides a blueprint for librarians who are interested in collaborating with faculty to develop an embedded librarian model in their institution. For teaching faculty who are interested in collaborating with librarians, Table 1 might also help them learn about the kind of assistance an embedded librarian can offer.
Embedding a librarian into a course was a win-win situation for the students, course faculty, librarian, and library. For the students, the librarian's presence throughout the course opened a readily accessible information service point, both virtually and physically, whenever they needed it. The librarian's shared responsibilities as co-instructor provided the course faculty significant support in teaching the health care content of the course. Being integrated into the course and working closely with the students and course faculty helped the librarian develop a better understanding of what students really needed. The library's value and relevance to the campus was strengthened and made more visible via the faculty-librarian collaboration. As a result of the successful pilot, the librarian was invited to embed in another 2012 graduate course, ''Concepts of Research Methodology,'' offered in the College of Allied Health and in two 2013 nursing courses: ''Concept & Theory Analysis'' (doctor of nursing practice course) and ''Theory Construction'' (doctoral course). The project helped pave the way for the library to provide services and resources in a more meaningful way in a changing environment.