A survey of collection development for United States Medical Licensing Examination ( USMLE ) and National Board Dental Examination ( NBDE ) preparation material

Methods: The investigators surveyed librarians supporting American Association of Medical Colleges (AAMC)–accredited medical schools (n558/125) on the USMLE and librarians supporting American Dental Association (ADA)–accredited dental schools (n523/56) on the NBDE. The investigators analyzed the data by cross-tabulating and filtering the results using EFM Continuum web survey software. Investigators also surveyed print and electronic USMLE and NBDE preparation materials from 2004– 2007 to determine the number of publications and existence of reviews.


INTRODUCTION
Sponsored by the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners (NBME), the United States Medical Licensing Examination (USMLE) functions as the standardized examination for assessing medical students.Composed of three examinations, or steps, the USMLE must be passed to receive medical licensure from state and territorial medical boards in the United States [1].Similarly, the National Board Dental Examination (NBDE) from the American Dental Association (ADA) qualitatively evaluates dental students [2] and satisfies the written examination requirement for licensure by state dentistry boards [3].Institutionally, medical and dental school educators use USMLE and NBDE scores to comparatively evaluate the strengths and weaknesses of their curricula [4][5][6][7][8].On an individual level, higher scores on the USMLE correlate to higher scores and passage of medical specialty board examinations [9][10][11][12][13][14], higher clinical performance ratings [15][16][17][18][19][20], and preferred residency attainment [6,[21][22][23][24][25].
Due to the ''high stakes'' nature and immense professional influence of these licensing examinations, exam takers feel intense pressure to score well in order to pursue their desired career plans.Medical students who use commercial USMLE preparation texts and medical textbooks indicate their utility [26] and demonstrate markedly higher USMLE scores than their counterparts who use other preparation methods [27].Dental students cite print NBDE materials, such as flashcards and old examinations, as most helpful to their preparation [28].A Wayne State University study reveals that a majority of medical students view electronic testing through their personal digital assistants (PDAs) as effective preparation for the electronically administered USMLE [29].Moreover, experience and a higher comfort level with computer-based testing may help reduce anxiety associated with the USMLE and may increase preparedness for the USMLE among medical students [30].Conversely, medical students who attend commercial test preparation courses [31][32][33] or use course materials and school-produced USMLE preparation materials do not show markedly higher USMLE scores [27].Similarly, attending a preparation course did not correlate to higher score on the NBDE for dental students [28].

Collection development of examination preparation material
The acquisition of print, CD, and DVD examination preparation resources poses several collection development quandaries for health sciences libraries in an age of shrinking budgets and inflationary materials costs, including: & crafting flexible collection development policies and procedures for examination preparation resource [34] & qualitatively assessing preparation materials for unfamiliar examinations & maintaining a collection of resources characterized by limited currency & selecting from among the overwhelming number of resources published yearly & identifying the considerable number of earlier edition reprints & selecting resources without many in-depth evaluative reviews & constructively dealing with aggressive promotion by publishers targeting students and librarians [31,33] & simultaneously acknowledging the great importance of examination preparation resources to students and tempering the associated demand & preserving resources that have a great probability of defacement or theft & balancing financial costs associated with judiciously replacing static formats [35] Despite the fact that online test preparation resources offer a more robust and flexible format, immediate updates, and simulations of actual exam conditions, they, too, create problems for health sciences libraries due to the aforementioned assessment and demand issues.In light of this environment, this paper addresses health sciences libraries' collection development policies and procedures regarding print and electronic USMLE and NBDE test preparation materials based on survey data collected in the spring of 2007.

METHODOLOGY A survey of examination preparation materials
To survey the examination preparation materials landscape and identify titles expressly written to prepare medical students for the USMLE and NBDE, the authors searched WorldCat, Doody's Core Titles, Google Book Search, and Amazon.com.The results from these searches were stored and deduplicated in EndNote.All formats (print book, electronic book, CD-ROM) of a specific title were counted separately.Table 1 summarizes the number of print and electronic USMLE and NBDE preparation resources published from 2004 to 2007.Books and other printed materials, such as flashcards, constituted 394 (72%) of the USMLE resources.Electronic resources, constituting 24% (n5144/546) of identified USMLE preparation resources, were usually an electronic version of a previously published print work.Other formats included CD-ROMs, audio CDs, and web-based resources.For the NBDE, publishers issued considerably fewer (12) print preparation resources and no electronic or online resources by March 2007.

Reviews of materials
Scholarly reviews of information products assist librarians in making collections decisions.Doody's

Highlights
N From 2004-2007, the authors identified the publica- tion of 546 and 12 print or electronic United States Medical Licensing Examination (USMLE) and National Board Dental Examination (NBDE) preparation resources, respectively.
N Sixty-two percent of responding AAMC libraries (n558) provide at least 1 electronic or online USMLE preparation resource, in contrast to 1 ADA library (2%, n51/23) that provides an electronic or online NBDE preparation resource.
N Twice as many respondents (n518/58) felt online USMLE preparation resources were a good value as opposed to being too costly.
N Some librarians reported concerns about the quality of questions in examination preparation materials due to poor grammar, inappropriate difficulty level, and irrelevance to the actual examinations.

Implications
N More reviews and usability tests must be conducted and published to assist other libraries in their collections decisions.
N Librarians must partner with individuals familiar with the examinations' content to more effectively evaluate the quality and appropriateness of examination preparation resources.

Hendrix and Hasman
Review Service [36] serves as the only reliably updated review resource for print and CD-ROM USMLE preparation materials.From 2004-2007, 10% (n555/546) of all published USMLE preparation titles and no NBDE preparation titles were reviewed in Doody's.Mostly written by authors who have medical or doctoral degrees, many reviews critically analyzed the content of the titles, though reviews did vary in quality.
Searches of MEDLINE; EMBASE; Web of Science; Library Literature and Information Science Full Text; Library, Information Science and Technology Abstracts; and ERIC uncovered very few reviews of test preparation resources, and the vast majority of those focused on college entrance examinations (e.g., Scholastic Assessment Test, ACT, Advanced Placement) and the Graduate Record Examination (GRE).The authors found only five reviews of USMLE preparation resource [37][38][39][40][41] and no reviews of NBDE preparation resources.A plethora of user-generated reviews of USMLE and NBDE preparation resources populate online forums, such as Studentdoctor.net[42,43], though questions concerning authority and authenticity of reviews arise.Without many traditional or authoritative resources to consult, health sciences librarians often use professional email lists [44] and newsletters [45] to seek advice from their colleagues and discuss user satisfaction, coverage, format, quality, and licensing.

Surveys of collection development policies
The investigators designed and distributed two surveys: one focused on preparation materials for the USMLE and the other on questions specific to the NBDE (Appendixes A and B online).Almost identical, the surveys differed in length due to one extra question on the NBDE survey, which asked about specific print examination resources.The authors anticipated insufficient data regarding electronic or online NBDE preparation resources due to preliminary research in WorldCat that revealed no library ownership of these products, thus the extra question specifically gauged what libraries purchase.Both surveys sought to discover if medical, dental, and health sciences libraries have official collection development policies for USMLE and NBDE preparation materials, their justifications for the policies or collection development decisions (if applicable), and their methods for evaluating preparation resources.Questions also sought to determine factors that go into collection development decisions regarding preparation resources.
The web-based survey software, EFM Continuum, was employed to facilitate ease of distribution, foster a higher response rate, and analyze data.The link to the survey, sent via email, remained live for two weeks.A reminder email was sent out after the first week of the survey.EFM Continuum's analysis function was used to compile the survey data, run cross tabulations, and filter the data by specific criteria.As a backup, the raw survey data were exported to an Excel file.
The population for the USMLE survey consisted of collection development librarians, electronic resources librarians, or medical school liaison librarians at the 125 health sciences libraries that supported American Association of Medical Colleges (AAMC)-accredited medical schools.The authors identified 1 collection development-focused librarian at each library via a search of the libraries' websites.If a collection development librarian dealing exclusively with medicine was not found, the authors proceeded to contact the collection development director, electronic resources librarians, medical school liaison librarians, and library directors, in that order.In March 2007, the USMLE survey was sent to 125 librarians.
For the NDBE survey, subjects included the collection development librarian and/or dental school liaison at the fifty-six health sciences libraries that support ADA-accredited dental schools.Due to the smaller number of dental schools, the authors identified two librarians, the head of collection development and the dental liaison, when that information was available.To ensure results were not unduly influenced by multiple responses from one institution, EFM Continuum sent each librarian a unique URL, so that the authors could track multiple responses.The NBDE survey was sent to seventy-five librarians.This research was determined to be exempt from review by the University of Buffalo Institutional Review Board.

Response rate
Fifty-eight of 125 librarians receiving the USMLE questionnaire completed the survey, constituting a 46% response rate.Twenty-six of 75 librarians receiving the NBDE instrument completed the survey, representing a 35% individual response rate.However, 3 pairs of surveys came from the same institution, so the last received survey in each pair was discarded, resulting in 23 (41%, n523/56) institutions responding.Some respondents chose not to answer all the questions.

United States Medical Licensing Examination (USMLE) preparation materials
Printed USMLE materials.Among the 58 responding libraries, the mean and median number of print Exam preparation materials USMLE preparation materials purchased was 11.6 and 7.5 per year, respectively.Libraries received an average of 3 gift USMLE materials per year.The number of print materials purchased varied widely as evidenced by a standard deviation of 13.7.For example, 17 libraries reported that they purchased no print USMLE preparation materials, while the annual purchased titles ranged from 18 to 60 at the 15 libraries purchasing the most materials.Among the 41 libraries indicating purchase of print USMLE preparation materials, the average number of materials purchased was 16 (SD513.6).The data collected on 2004-2007 USMLE examination preparation print resources indicate that an average AAMC library annually bought 12% (n5394) of all published titles.
Analysis of the data indicated differences between libraries that had official collection development policies in regard to examination preparation materials and libraries that did not have an official policy.Of the 58 respondents, 23 (40%) reported that their libraries had an official policy regarding examination preparation materials.Libraries without official collection development policies for examination preparation materials bought almost double the amount of print USMLE preparation materials annually (mean514.4resources) as opposed to libraries with an official policy toward these types of materials (mean57.5 resources).Furthermore, these data might reflect the fact that 11 of the 23 (48%) libraries with official policies had policies prohibiting purchases of these types of materials.The acquisition of gift USMLE preparation materials was not related to the presence of an official policy.
To justify their collection development decisions in regard to print USMLE preparation resources, respondents most often cited users' needs, institutional goals, and the resources' good value as rationales for their decisions to purchase.A third of the respondents reported the fleeting currency of materials and frequency of theft and defacement of materials as reasons not to purchase print USMLE preparation materials (Table 2).

Electronic or online USMLE preparation materials.
Of the responding AAMC libraries, 62% (n536/58) licensed at least 1 electronic or online USMLE preparation resource and 16% (n59/58) licensed 2. The top licensed resources include Exam Master [46] (45%, n526/58) and USMLEasy [47] (33%, n519/58).Conversely, 17% (n510/58) of librarians responded that policy dictated that they never buy electronic or online examination preparation resources.Analysis of the free-text comments revealed that a small percentage of libraries 7% (n54/58) reported that their affiliated medical schools were officially responsible for purchasing USMLE preparation materials.
When evaluating electronic or online USMLE preparation resources, a majority of responding librarians reported that they employed feedback from Hendrix and Hasman medical students (64%, n537/58), feedback from medical faculty (60%, n535/58), or online trials (55%, n532/58).Additionally, 10% (n56/58) of respondents reported that no evaluation of these products went on at their institutions; this figure included 3 libraries that owned or subscribed to the USMLE preparation products (Table 3).The survey data showed that using more evaluation methods increased the likelihood of subscribing to electronic or online USMLE preparation resources.For example, a substantial majority of subscribing libraries consulted with their students (78%, n528/36) or faculty (75%, n527/36) before deciding on a USMLE product, as opposed to 38% (n58/21) and 43% (n59/21) of nonsubscribing libraries, respectively.When asked to describe justifications for their collection development rationale for electronic or online USMLE preparation resources, over half of responding librarians stated that electronic or online USMLE preparation resources were needed by library users (52%, n530/58).In contrast, not one librarian championed the opposite view that their users did not need electronic or online USMLE preparation resources.In addition, over 3 times as many respondents (n523/58) believed that collecting electronic or online USMLE preparation resources concurred with their institutions' missions and goals as opposed to those who did not, and twice as many respondents (n518/ 58) felt online USMLE preparation resources were a good value as opposed to being too costly (Table 2).
Respondents were asked to rank a list of eleven factors involved in the decision to subscribe or purchase electronic or online USMLE preparation resources from most important to least important.After filtering out the respondents whose libraries did not subscribe to these resources, assessing the cost and value of an electronic or online USMLE preparation resource ranked as the most influential factor by a considerable margin followed by student requests, quality of the questions, and faculty requests.Conversely, vendor information and contact ranked as the least influential factors in the decision-making process (Table 4).

National Board Dental Examination (NBDE) preparation materials
Print NBDE resources.The majority of respondents (52%, n512/23) to the NBDE survey reported that they did not purchase print NBDE preparation resources.No library bought more than 4 titles annually.There were also comments noting that their libraries: ''purchase all that are published,'' ''whatever is current,'' and ''all that we can identify,'' which was within reason given the small amount of published titles.Across all libraries, the average purchases of print NBDE preparation resources amounted to slightly less than 1 title per year.Among the 11 libraries that did purchase these resources, an average of 2 NBDE titles were purchased annually.
Split almost evenly, 10 responding libraries had official NBDE materials collection development policies (44%) and 11 libraries (48%) did not.However, there were differences in the purchasing decisions for the institutions that did and did not have official policies.Among libraries with official collection development policies, policies tended to dictate extremes: either purchasing no or purchasing all available NBDE resources.Thus, while 50% (n55/ 10) of libraries with official policies reported never buying NBDE resources, 30% (n53/10) reported that that they attempted to buy or add everything.Libraries that did not have official collection development policies reported a lesser propensity to extremes, with 27% (n53/11) reporting that they never bought NBDE preparation resources and 9%  Exam preparation materials (n51/11) noting that they attempted to buy everything available.Table 2 shows the justifications for collection development of NBDE preparation materials for the entire respondent pool.Most respondents noted that concern over theft or defacement (22%, n55/23), followed by responsibility for providing access to NBDE testing materials residing with the dental school (17%, n54/ 23) and users' need for print NBDE materials 13% (n5 3/23), as a key reason for purchase decisions.Equal numbers of respondents reported that collecting print NBDE materials was and was not in accordance with the library's mission or goals (13%, n53/23).
Cross tabulation of these justifications with data from the 9 libraries whose policies prohibited purchase of NDBE preparation materials revealed the following the justifications for not purchasing: & Print NBDE preparation resources were frequently stolen or defaced (33%, n53/9).& Collecting print NBDE preparation materials ws not in accordance with the library's mission or goals (33%, n53/9).& The dental school took responsibility for providing access to all NBDE testing materials (22%, n52/9).
The 11 libraries with no official restriction to the purchase of NDBE preparation materials cited the following justifications for purchasing: & Collecting print NBDE preparation resources was in accordance with the library's mission or goals (33%, n53/11).& On my campus, the library took responsibility for providing access to all NBDE testing materials (22%, n52/11).& Print NBDE preparation resources were needed by library users (22%, n52/11).
When respondents were asked what methods they employed to evaluate NBDE preparation resources, the most cited were feedback from dental faculty (57%, n513/23), feedback from dental students (43%, n5 10/23), online trials (30%, n5 7/23), and discussions on email lists (30%, n5 7/23).Also, 35% (n58/23) of respondents reported that they did not evaluate these resources (Table 3).The 15 libraries that evaluated NBDE preparation materials used an average of 3.7 of the listed methods.As with the USMLE survey, respondents were asked to rank a list of factors that most influenced their selection of NBDE preparation resources.Responding libraries indicated that faculty requests, student requests, and cost and value were the most influential.
Electronic or online NBDE resources.While a plethora of electronic or online USMLE preparation resources exist, there are few similar options available for the NBDE.Nearly all respondents, 96% (n522/23), reported that they do not provide electronic NBDE preparation resources.One library reported they subscribed to the Exam Master NBDE product (which was to be released shortly after the time of this survey distribution in March 2007).Additionally, several other products (e.g., Crack the NBDE) available in March 2007 were not being purchased by libraries.
Table 2 illustrates that libraries most often reported that the dental school, not the library, was responsible for providing electronic preparation materials (22%, n55/23) as the rationale behind their collection development decisions.Costliness (17%, n54/23) and incongruence with the library's mission or goals (13%, n53/23) followed.No respondents reported that their library was responsible for collecting all online NBDE preparation materials.

Publication differences
Recent reports from the AAMC and ADA present similar numbers of medical students (17,759) [48] and dental students (18,315) [49] in the United States.However, the search data reveal that publishers produced 45 print and electronic or online USMLE preparation products for every 1 NBDE preparation product.Possible explanations include differences in the ramifications of board examinations to the 2 student groups, differences in study behavior between the 2 groups of students, differences in the role dental and medical schools play in board examination preparation, differences in consumption patterns between the 2 groups of students, or the presence of an untapped dental student market.

Responsiveness to user needs
The majority of responding libraries seem to address their user groups' potential needs in regard to examination preparation materials.Analysis of the data and comments in Table 5 attest to the fact that the majority of respondents factor student requests into their decision-making process.Moreover, some libraries have integrated student and faculty requests into their collection development policies.As illustrated in Table 5, many respondents lauded examination preparation materials for helping their clientele with the USMLE and NBDE.

Effect of official collection development policies
Among both USMLE and NBDE responses, official collection policies, in some cases, may prevent the purchase of examination preparation resources.Eighty percent of dental libraries with official policies (n58/10) reports that the policy dictates either of the 2 extremes: never buy or buy everything.In these libraries, official policies reduce the role of librarians' professional discretion in regard to NBDE preparation resources.Libraries without official policies report a greater likelihood of relying on the discretion of librarians when deciding on USMLE and NBDE preparation materials.

Reasons for not collecting
Besides official policies, this study's quantitative and qualitative data imply that budgetary concerns underlie many of those policies and decisions.The data and comments in Table 5 suggest other possible explanations for not purchasing examination preparation materials, including quality of questions, individual library roles at the parent institution, need for frequent updating, and concerns about theft and vandalism.Moreover, some electronic or online examination preparation materials can only be licensed on an individual basis and have no institutional options.

Evaluating quality
As outlined in the literature review, these licensing tests carry enormous weight in the direction of a professionals' career, therefore, purchasing preparation materials of questionable quality does a great disservice to students and the parent institution.However, evaluating mock examinations and question quality can be problematic for libraries due to a number of factors.
Many collection development librarians try to accommodate requests for these materials as budgets allow [50].However, responsiveness to library users should be tempered with critical evaluation.The tenor of the surveys' comments, anecdotal information, and personal experiences of the authors suggest that the overwhelming majority of requests for these products come from first-or second-year dental or medical students.In most cases, these students have not taken their board examinations and therefore are not in an ideal position to judge quality and utility of a question or clinical vignette.As targets of aggressive marketing campaigns by publishers and examination preparation companies [31,33], students may recommend that their libraries purchase heavily advertised resources and often may not consider the quality of the resource.Respondents report a large majority of library staff making decisions on USMLE preparation products at their institutions have not gone to medical school (95%, n53/58) or taken the USMLE (97%, n52/58) and are therefore unfamiliar with what constitutes a quality question or clinical vignette.This is even more pronounced with the NBDE: none of the respondents have attended dental school or taken the NBDE.
A large majority of examination preparation materials located for this study (90%, n555/546) have not been reviewed by qualified members of the medical and dental education fields.Doody's Review Service provides the only consistent source for reviews of examination preparation materials, albeit a small percentage of the published materials.Unfortunately, Doody's Review Service does not include reviews of online USMLE or NBDE preparation resources.As Doody's is a subscription service, the number of libraries with access is likely limited.Web and proprietary database searching did not retrieve a single review of online examination preparation materials, beyond discussions on online medical and dental forums.
Because of the dearth of reviews and their inexperience with the USMLE and NBDE, librarians use alternate methods to evaluate electronic or online examination preparation products.For example, the survey data indicate that 48% (n528/58) of responding libraries utilize online trials in conjunction with faculty and student feedback as a prime evaluation method.A large percentage of libraries that set up online trials to evaluate electronic or online USMLE preparation resources (75%, n524/32) end up sub-Cust # 1536-5050 N ''We have heard that some of the questions are too easy and the explanations are either too lengthy or not detailed enough.''N ''potentially detrimental to students' performance on these exams.''N ''there were grammatical and spelling errors all over the place and they used words that I've never seen used in medicine in America.''N ''It asked questions about portions of medicine I've never heard of and seriously doubt are even related to anything relevant for this exam.''N ''We had one patron who found a couple of grammatical errors and thought the questions were not well-written.''N ''Faculty do not want us getting into it online.'' Exam preparation materials scribing.A small number of responding libraries (12%, n57/58) engage in usability testing, arguably the most comprehensive method to evaluate an online product [51], resulting in 5 of 7 (71%) of these libraries subscribing.
As illustrated in Table 5, most respondents spoke positively about the use of the resources, their users' feedback, and the features that online examination preparation products afford, but no respondent addresses question quality positively.Some comments reveal various levels of dissatisfaction with examination preparation resources and question quality.Some comments refer to the bad grammar of the questions, limited currency of questions, and difficulty level of the questions or medical school faculty resistance to subscriptions.
Similarly, after online trials of the major preparation products at the authors' home institution, online USMLE preparation products were rejected by the health sciences library in consultation with the office of medical education due to concerns about quality and lack of published qualitative analyses and reviews.Though the majority of feedback came from first-and second-year medical students who stated a desire for online examination preparation materials.Medical school faculty and students who had taken the USMLE criticized the products.These criticisms included improper grammar, irrelevant content and questions, and potential harm of substandard preparation materials.In a December 2006 summary of responses on MEDLIB-L, a medical libraries email list, similar critiques of online USMLE preparation products surfaced, though a majority remarked that they had not received any complaints [44].

Opportunities
Due to student demand for examination preparation materials, the authors believe that outreach opportunities exist for libraries to partner with the medical and dental schools in providing reliable and vetted resources.Librarians' strengths, such as product licensing and computer-human interaction, paired with the expertise of medical and dental faculty and students conversant with the examinations ensures that an appropriate decision can be made.Cooperatively conducting usability tests or instituting structured feedback collection during product trials are examples of evaluative options that libraries should consider before making a purchasing decision.This study's findings reveal that only 7% (n54/58) of responding libraries report that they maintain a homegrown USMLE and/or NBDE web page, so opportunities also exist for libraries to collaborate on a web-based clearinghouse of reputable links.Libraries could also provide the infrastructure for a wiki-based [52] USMLE or NBDE question bank.

Limitations of the survey
The authors recognize the methodology's limitations.Firstly, the questions on the two surveys did not mirror each other exactly.For instance, the USMLE survey let respondents fill in any number when asked how many print materials they buy annually, whereas the dental survey presented this question in multiple choice format with number ranges.The dental survey was sent to multiple librarians at the identified institutions, while the medical survey was sent to one librarian per institution.Due to the population selection method, the contacted librarians might not have had anything to do with purchasing examination preparation materials, which might have resulted in a lower response rate or inaccurate responses, and the surveys reflected a snapshot of self-reported data.The surveys' response rates also limited the generalizability of the findings.

FURTHER RESEARCH
One of the challenges this study revealed is the dearth of qualitative reviews of test preparation materials.As more electronic or online examination preparation resources become available, there is a need for research on the quality of the content.Further studies could correlate the use of these products to trends in examination scores, user preparedness, and user satisfaction.As most board examinations are now administered via computer, future research could further explore differences between students using print preparation materials versus students using electronic or online preparation materials.Due to the diversity of the surveyed libraries, future research could also subdivide the population into libraries affiliated with a larger library system and those that serve a medical or dental school exclusively.
Though many libraries employed online trials or usability testing in tandem with faculty and student feedback, this survey was not designed to capture the quality of the faculty and student feedback.Thus, in addition to considering allusions to question quality in the comments section, the authors can only speculate based on their own experiences.Future research could delve deeper qualitatively by having medical and dental educators and students with different levels of board examination experience evaluate various electronic or online examination preparation products.
For librarians supporting medical or dental programs, this study has several implications.Librarians' responsiveness to their users' requests must be tempered with a more robust evaluative process.Involving faculty and students who have taken the licensure examinations in the decision-making process is essential as most librarians do not have the same familiarity with the nuances of the examinations.A more vigorous dialogue among librarians is needed in regard to the quality of examination preparation resources, results of usability tests, and user feedback from online trials.The sharing of this information can compensate for the negligible amount of impartial information currently available and offer opportunities to showcase the resource expertise of librarians.

Table 4
Ranked list of factors involved in USMLE preparation collection development decisions (n542)

Table 5
Representative respondent comments regarding examination preparation resources ''all of our Exam Master seats are always all in use.''N ''Students always want USMLE resources online, especially if they don't have to pay for it'' N ''the usage statistics indicate they are popular with students.''N ''There is a demand for them and we are about to re-examine our policy.''N ''Because the [institution's] curriculum does not include much testing, students are anxious about the USMLE exams and use the prep materials heavily.''Examination preparation materials user feedback N ''Most responses are positive indicating that they give them experience in taking the test since it is now online.They like the variety of questions and the rigor.Some do not like them, because they prefer to study their course material.''N ''50% liked Exam Master, the other 50% said it didn't help.''N ''Students have made it clear that they appreciate having access to these resources.''N ''Mostly positive feedback by virtue of the fact that we were at least providing this resource electronically so that it could be accessed remotely.The questions prepared the students well for the exam.''N ''Positive feedback on manipulation of results; ability to track questions and responses; generated reading lists.''Concerns regarding currency of examination preparation materials N ''released boards are not current.''N ''Most faculty express concern about the datedness of the older tests.''N ''some print materials are out of date, but Mosby's and First Aid are more recent review materials.N ''print material was outdated.School prefers to provide preparation resources to their own students.''Budgetary concerns regarding examination preparation materials N ''budget priorities have been/are the primary concern leading us to choose not to collect such materials.''