• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of cellbioledLink to Publisher's site
Cell Biol Educ. 2004 Winter; 3(4): 270–277.
PMCID: PMC533131

Survey of Undergraduate Research Experiences (SURE): First Findings

William Wood, Monitoring Editor

Abstract

In this study, I examined the hypothesis that undergraduate research enhances the educational experience of science undergraduates, attracts and retains talented students to careers in science, and acts as a pathway for minority students into science careers. Undergraduates from 41 institutions participated in an online survey on the benefits of undergraduate research experiences. Participants indicated gains on 20 potential benefits and reported on career plans. Over 83% of 1,135 participants began or continued to plan for postgraduate education in the sciences. A group of 51 students who discontinued their plans for postgraduate science education reported significantly lower gains than continuing students. Women and men reported similar levels of benefits and similar patterns of career plans. Ethnic groups did not significantly differ in reported levels of benefits or plans to continue with postgraduate education.

Keywords: undergraduate, research, learning, career, science

Attempts to determine an empirically established set of benefits generated by undergraduate research experiences in the sciences are fairly recent (Lopatto, 2003a blue right-pointing triangle; Seymour et al., 2004 blue right-pointing triangle). Lopatto surveyed science undergraduates at four liberal arts colleges over a period of three summers. The quantitative results of these surveys fit well with qualitative data drawn from student interviews at the same four institutions by Seymour et al. (2004) blue right-pointing triangle. Having grounded the benefits of undergraduate research in both quantitative and qualitative data in these pilot studies, it remains for researchers to establish that the findings apply to a broader range of institutions. The opportunity for a more extended study was presented by the Howard Hughes Medical Institute (HHMI), which funds grant activity for undergraduate science education at a variety of institutions.

The undergraduate research experience is widely touted as an effective educational tool for enhancing the undergraduate experience (Mogk, 1993 blue right-pointing triangle; Tomovic, 1994 blue right-pointing triangle) with multiple benefits (Lopatto, 2003a blue right-pointing triangle), the most instrumental of which is an increased interest in a career in the science, technology, engineering, and mathematics workforce (Fitzsimmons et al., 1990 blue right-pointing triangle; Zydney et al., 2002 blue right-pointing triangle). Undergraduate research experiences are associated with increased persistence in pursuit of an undergraduate degree (Nagda et al., 1998 blue right-pointing triangle); increased levels of pursuit of graduate education (Hathaway et al., 2002 blue right-pointing triangle; Kremer and Bringle, 1990 blue right-pointing triangle); and alumni retrospective reports of higher gains than comparison groups in skills such as carrying out research, acquiring information, and speaking effectively (Bauer and Bennett, 2003 blue right-pointing triangle). Several studies have supported the hypothesis that undergraduate research helps promote career pathways for members of underrepresented groups by increasing the retention rate of minority undergraduates (Nagda, et al., 1998) and increasing the rate of graduate education in minority students (Hathaway et al., 2002 blue right-pointing triangle).

This study was motivated by three research questions, primarily concerning but not limited to HHMI-funded undergraduate research experiences in the sciences. These questions are: 1) Is the educational experience of undergraduates being enhanced? 2) Are undergraduate research programs attracting and supporting talented students interested in a career involving scientific research? 3) Are undergraduate research programs retaining minority students in the “pathway” to a scientific career?

In terms of this survey, question 1 was answered by investigating both the general response to the undergraduate research experience and the specific gains reported on 20 potential benefits. Question 2 was examined by asking students their plans for postgraduate education. Are students in undergraduate research programs declaring that they intend to continue their science education or to seek a science-related career? Furthermore, can we identify differences in the experiences of those students who intend to continue in science compared with those students who assert that they have lost interest in science careers? Question 3 was examined by looking for differences among ethnic groups in proportion to each group that intends to continue in science. Included in this examination were differences between women and men because women are an underrepresented group in some sciences.

METHOD

Description of the Respondents

The survey was completed by 1,135 undergraduates representing 41 universities and colleges in late summer and fall of 2003. A preliminary survey of HHMI program directors at participating institutions indicated that a total of 1,526 students participate in summer undergraduate research programs. Thus, the overall response rate was 74%. The 41 institutions included 19 universities (Carnegie Classification Doctoral/Research Universities—Extensive), 15 colleges (14 Carnegie Classification Baccalaureate College—Liberal Arts, 1 Carnegie Classification Baccalaureate College—General), and 7 master's level institutions (Carnegie Classification Master's College and Universities I). Demographic characteristics of the respondents are given in Table 1.

Table 1
Demographic description of the student respondents

As is true of every tabular or statistical presentation in this report, there are missing cases. Students might have failed to indicate their institution, declined to specify personal characteristics, or left an evaluative question unanswered because it did not apply to them. Table 1 shows the categories of institutional type, sex, and ethnicity. Overall, including cases missing from Table 1, 258 respondents attended a college, 59 respondents attended a master's university, and 787 respondents attended a research university. Thirty-one respondents did not specify their institution. Analysis of the data in Table 1 indicates that the distribution of men and women is uniform across institutional types. The distribution of men and women is also uniform across ethnic categories. The distribution of ethnic categories across institutional types is nearly uniform. A statistical analysis of ethnicity by institutional type yields a significant chi square (χ2 = 34.4; df = 12; p < .05), with higher than expected frequencies of Asian American and Hispanic students at Research Universities.

Respondents were asked to cite the source of the funding that supported their research. Fifty-five percent of the respondents reported being funded by HHMI programs. Local university or college funding accounted for 12% of the respondent support. Other sources of funding were each less than 10% of the total. Non-HHMI-funded students were eligible to take part in the survey because institutions with HHMI grant programs often fund other students for similar activities in the same time period, so it was deemed sensible to invite these students to participate.

Approximately 59% of the respondents are women. Table 2 shows the research fields of the respondents crossed with the sex of the respondent. As might be expected from national trends, women outnumber men in biology, chemistry, and biochemistry, but men outnumber women in physics, mathematics, computer science, and engineering.

Table 2
The research fields of the respondents classified by sex of respondent

About 48% of the respondents reported their graduation year as 2003 or 2004. Juniors graduating in 2005 made up 32.6% of the total, whereas sophomores made up 16.3%. The remaining 2.7% reported their graduation year as 2007, implying that they were entering first-year students. Specific ethnic categories conformed to this pattern, with the exception of Hispanic respondents, who had a lower proportion of rising seniors (34.5%) and a higher proportion of rising juniors (40%). Only about one-third (35%) of the entire group reported no prior experience in undergraduate research. Older students tended to have more prior experience than younger students.

The Survey

The Survey of Undergraduate Research Experiences (SURE) consisted of 44 items, including demographic variables, learning gains, and evaluation of aspects of summer programs. Items regarding learning gains were suggested by previous survey research. Before the main survey was conducted, a brief checklist survey was sent to HHMI program directors. A list of learning gains concepts was presented, and program directors were asked to report whether each was primarily taught explicitly (e.g., in a seminar or workshop) or implicitly (e.g., a behavior modeled by the student's supervisor).

The SURE survey was located online on server at Washington University in St. Louis, MO.

Procedure

Notices of survey availability were sent to each program director. Participating program directors (PDs) were asked to specify the number of students from their school eligible to take the survey and the date on which they would be asked to do so.

The target date was immediately after the end-of-program symposium or other “summing up” activity. Two weeks after that date, PDs were informed how many students had participated, giving the PDs the option to contact their students to remind them to participate in the survey. Students were provided with a name and password for access to the survey. Within the survey, students identified their school and provided demographic information, but anonymity was maintained. Student names were collected for a raffle that awarded gift certificates to the winners, but the names were separated from the survey material. Students answered items on the survey by either selecting from a pull-down menu or choosing a number on a rating scale. A “no answer” option was available. At the end of the survey, students were provided with a text box for written comments, which were directed to the PD for that institution by e-mail. After the site closed, all PDs received the aggregate results for their school.

RESULTS

Interesting Science Careers

Students reported their plans for postsecondary education. Their responses are categorized in Table 3. Most of the respondents had some plan for further education; the leading categories are medical school (22%) and biology related (20%). Respondents were asked how their research experience influenced their plans for postgraduate education. These responses are presented in Table 4. Of those who responded, almost 91% reported that their research experience sustained or increased their interest in postgraduate education. Only 4.7% reacted to their undergraduate research experience by changing their plans away from postgraduate education.

Table 3
Respondents' reported plans for science education beyond the undergraduate degree
Table 4
Responses to the question about how the research experience influenced a student's plan for postgraduate education

Reported Learning Gains and Overall Evaluation of the Undergraduate Research Experience

To evaluate the educational experience of undergraduate researchers, 20 evaluative questions on specific learning gains, drawn from previous research, were presented to the respondents. Table 5 summarizes the results by listing the means for each set of responses. These means are listed in descending order. The highest rated item is “Understanding of the research process in your field” (X = 4.13), followed by “Readiness for more demanding research” (X = 4.03), “Understanding how scientists work on real problems” (X = 4.0), and “Learning laboratory techniques” (X = 4.0). The lowest rated item is “Learning ethical conduct in your field” (X = 3.15), followed by “Skill in science writing” (X = 3.32), “Skill in how to give an effective oral presentation” (X = 3.42), and “Clarification of a career path” (X = 3.42). A measure of interitem consistency, Cronbach's alpha, on these 20 items is .92, indicating a high degree of consistency. Students who reported being funded by HHMI also reported higher means on all 20 items compared with the overall survey means (Table 5).

Table 5
Mean responses to 20 gains from the undergraduate research experience

In addition to the 20 specific learning gains, respondents were asked to evaluate five general questions about their experience. Student attitudes toward their research supervisor, peers, and overall sense of summer research as a learning experience were decisively positive. The frequency distributions of responses to these questions are found in Table 6. The responses reveal a high degree of satisfaction with the undergraduate research experience. Eighty-seven percent of the respondents rated their experience as good or better than they expected. Seventy-eight percent of the respondents rated their supervisors as above average or outstanding, an impressive finding. The supervisors included faculty, postdoctoral students, and graduate students. Sixty-three percent of the students rated their experience with other students in a positive way; 19% of the respondents did not answer this question, presumably because they did not work with other undergraduates. Eighty-four percent of the respondents indicated they were likely or very likely to choose another research experience if they could, and a similar percentage gave a positive overall evaluation of their summer research as a learning experience. These five overall evaluative questions are correlated with each other (Table 7). For example, the overall sense of summer research, perhaps the most global of these questions, is significantly related to prior expectations (rs = .47), to supervisor ratings (rs = .39), to student ratings (rs = .20), and to choosing another research experience (rs = .36). Thus, the responses are generally consistent.

Table 6
Student responses to five questions concerning the overall undergraduate research experience
Table 7
Spearman correlation coefficients for the five global evaluative questions about the research experience

To investigate the question regarding the attraction of talented undergraduates to science, the data were analyzed for students who reported positive or negative experiences. One variable for such exploration concerns the influence of the research experience on plans for postgraduate education given that one of the goals of undergraduate research is to promote graduate education in science. As described earlier, the undergraduate research experience sustained or confirmed the student's plans for graduate education. Over 83% of the students continued to plan for postgraduate education in the sciences following their undergraduate research experience. Two groups of students did change their plans (see Table 4). Forty students initially had no plans for postgraduate education, but the research experience changed their minds. Fifty-one students initially had plans for postgraduate education, but their research experience convinced them otherwise. In what ways did their experiences differ? To find out, the 20 learning gains questions were analyzed. When the mean scores for these two subgroups are inspected, the students who now plan to further their education have the highest means of any group displayed in Table 5 on 18 of the 20 items. In contrast, the respondents who decided not to pursue further educations have the nominally lowest means of any group displayed in Table 5 on 15 of the 20 items. When the two groups are directly compared to each other via independent group t-tests, 13 significant differences emerge at the α = .005 level, all in the direction of higher learning gains by the group that is now planning graduate education.

The two groups that were influenced by summer research to change their plans for postgraduate education also differed on some of the overall evaluation questions. An inferential statistic, the Mann-Whitney U-test (evaluated at α = .01), was employed to analyze the data. Students who changed their plans in favor of graduate education had higher ratings about summer research meeting or exceeding their expectations, evaluated their supervisors more favorably, were more likely to choose another experience, and had a more positive overall sense of summer research than students who changed their plans to forego graduate education. The two groups did not differ in their evaluation of working with other students. The two groups did not differ in the type of institution at which they had their experience. Although the results can be interpreted in various ways, the SURE survey nevertheless appears to capture the different experiences of students who become attracted to, compared with those who turn away from, science careers.

Differences between Women and Men

Women constituted almost 60% of the sample of respondents. As noted in Table 2, women are the majority in the biological sciences, whereas men are the majority in the physical sciences, math, computer science, and engineering. This distinction is repeated when students were asked about their postgraduate education plans. A higher percentage of women than of men plan to continue in biology (21.9% of all women vs. 19.3% of all men) and medicine (24.7% of all women vs. 21.1% of all men), whereas a lower percentage of women than of men plan to continue with a Ph.D. in the physical sciences (9.5% of all women vs. 21.6% of all men). Women and men did not differ overall in prior experience, in the influence of the research experience on their future plans, in their evaluation of their supervisor, or in whether they would choose another research experience. Women reported higher gains than men on 14 of the 20 learning items. These results, however, are entangled with institution (there are two women's colleges in the group), research area (biological vs. physical science), and previous experience indicating that women have a positive response bias when compared with men.

On overall evaluation questions, women reported higher ratings than men on two items. When asked to compare their experience to their prior expectations about the experience, more women than men rated the research experience as “Much better than I expected” (35.5% of women vs. 26.2% of men; χ2 =13.7; p < .05). Women also rated their experience with other students more favorably than men did (45.1% of women vs. 37.6% of men rated working with other students as “One of the best parts of the summer research experience” (χ2 = 14.0; p < .05).

Differences among Ethnic Groups

To address the issue of retention of minority students in a pathway to a scientific career, the reported experience of minority students was analyzed. Nearly half of the respondents in the sample were not members of the Caucasian majority. The better represented minority groups include Asian-American (14.9%) and African-American (10.0%). Hispanic students made up 4.9% of the total, whereas foreign national students constituted 6.1%. The memberships of some ethnic groups are so small (e.g., 1 Native American) that statistical comparisons are not meaningful.

Ethnic groups did not differ in their distribution of women and men, with women constituting the majority in every ethnic group. The research fields chosen by each group were dominated by the biological sciences (Table 8), as might be expected because the major source of student funding in this study was from the HHMI. Ethnic groups did not differ in prior experience. Asian-Americans showed relatively less interest in postgraduate education in biology-related Ph.D. programs (15%) and relatively more interest in medical school (35%) than comparison groups (see Table 9). There were no statistically significant differences among ethnic groups in the influence their summer research experience had on their postgraduate plans (Table 10), an indication that the undergraduate research experiences are retaining minority students in the pathway to a science career as well as they retain Caucasian students.

Table 8
Research fields categorized by ethnic group
Table 9
Plans for continuing education categorized by ethnic group
Table 10
Influence of research experience on postgraduate plans categorized by ethnic group

An analysis of the five general satisfaction questions revealed no differences among ethnic groups in their expectations of summer research being met, their evaluation of their supervisors, their evaluation of their student coworkers, their inclination to have another research experience, or their overall sense of research as a learning experience. Analysis of the 20 learning gains questions indicates that ethnic groups statistically differed on three items (“Learning ethical conduct”, “Skill in oral presentations”, and “Becoming part of a learning community”), however, with Hispanic students reporting scores that were higher than at least one other group.

Institutional Type

The means and standard deviations for each institutional type for the 20 learning gains items are presented in Table 11. Although some ratings of the institutions differ significantly on some items, the pattern of the differences is difficult to determine. Analyzing differences in student learning with reference to Carnegie Classification categories does not reveal clear differences among institutions.

Table 11
Mean gains classified by institution type

DISCUSSION

This study provides some answers to the three research questions raised initially. First, the educational experience of undergraduates is enhanced, both in terms of their general satisfaction and in terms of learning gains on 20 specific items. Of these items, those related to the research process, scientific problems, and lab techniques are rated as the highest gains. These are followed by gains in personal development (such as tolerance for obstacles and working independently). Institutional type does not determine the level of enhancement. Second, the 41 institutions represented in the data are attracting and supporting students interested in a career in science, as represented by their plans for postgraduate education in science. The majority of the student respondents planned to continue in the sciences, and health-related education, including medical school, was also a popular choice. The undergraduate research experience sustained the science career plans of a large majority of the students. A few students (about 4%) no longer planned for further science education; about the same number were inspired to change their plans toward further science education. Finally, the patterns of postgraduate plans were similar across gender and ethnic groups. Over half the students responding to the survey are women; almost half the students are from minority groups. The undergraduate research programs are providing a pathway to a scientific career for minority students, and the data indicate that most of these students intend to continue on this path.

The present findings replicate pilot work on assessing undergraduate research (Lopatto, 2003b blue right-pointing triangle; Seymour et al., 2004 blue right-pointing triangle). Lopatto surveyed 384 science undergraduate researchers working on summer research programs at four liberal arts colleges. As in this study, students rated their gains on potential benefits on a scale of 1 (no gain) to 5 (very large gain). The results showed that “Learning laboratory techniques” and “Understanding the research process” were among the highest rated benefits. Personal development items, including “Readiness for more demanding research” and “Tolerance for obstacles,” were also rated highly. The overall pattern of rated benefits was similar to that seen in this study. Seymour and her colleagues (2004) blue right-pointing triangle analyzed the results of 76 interviews with students participating in undergraduate research experiences at the same four sites examined by Lopatto (2003b) blue right-pointing triangle. A qualitative analysis, analyzing the text of the transcripted interviews, revealed the same pattern of benefits reported by students as are seen in the results presented here.

This study does not include a control group and does not sort out all the influences, including precollege career intentions, general academic achievement, and family influence, that might account for the choices students make to pursue careers in science. A control group is selected with the goal of creating a comparison group that is like the treatment group in all characteristics except the treatment. This goal is elusive in the in situ study of undergraduate students. Undergraduate research programs are affected by both institutional selection procedures, which filter the best applicants for research positions, and self-selection by students motivated to explore science. An attempt to create a quasi-control group, such as a group composed of students who applied for research positions but were turned down, does not escape the criticism that the control subjects were not identical to the treatment subjects before the undergraduate research experience commenced. Differential academic credentials or interpersonal skills might have sorted the undergraduate researchers and unselected students into nonequivalent groups. An attempt to create a true control group by traditional means, such as random assignment, would be unlikely to succeed in the face of ethical concerns regarding withholding education from talented students and regarding traditional student autonomy in choosing courses of study.

Should a reasonable control group be provided for the study of the undergraduate research experience, a second obstacle renders comparisons between treatment and control groups difficult. Undergraduate research experiences contain features that are not exclusive; laboratory courses, “researchlike” experiences and other pedagogies can share some of the features of undergraduate research. The presence of nonexclusive features means that treatment and control subjects are likely to differ in amount of treatment rather than to differ by the presence or absence of treatment. Furthermore, the undergraduate research experience is a molar treatment (Campbell, 1986 blue right-pointing triangle), consisting of an array of treatment components, including the mentoring skill of the supervisor, the social interactions of the working group, the sophistication of the instrumentation related to the research experience, and so on. Therefore, this study cannot meaningfully compare the learning gains of students who experienced undergraduate research relative to those who did not, except for the small group that left the science career pathway as a result of their experience. The results do show, however, that most respondents report an enhanced educational experience through a variety of learning gains. The experience sustains the interest of many students already planning science careers and attracts some students who did not previously plan a science career. These outcomes are consistent across gender and ethnic group.

Further information is needed to support these findings. The current research includes a 9-month follow-up survey, currently in progress, that will contribute more information to the robustness of student plans and to the specific enhancement of later educational experiences following the summer research experience. Ideally, a further research effort should include tracking the cohort of student respondents to determine how many continued on pathways to science careers.

Acknowledgments

This research was supported by HHMI grant 52003953 to Professor Sarah Elgin of Washington University, whose guidance made the project successful. The author is grateful to Marti Schafer for her organizational skills and to Frances Thuet for her expertise with online surveys and data management. I am grateful to the HHMI project directors who facilitated the use of the survey at their institutions and to the students who completed the survey. The pilot research referred to in the text was supported by NSF/ROLE grant REC0087611.

REFERENCES

  • Bauer K.W., Bennett J.S. Alumni perceptions used to assess undergraduate research experience. J. Higher Educ. 2003;74:210–230.
  • Campbell D.T. Relabeling internal and external validity for applied social scientists. In: Trochim W.M.K., editor. In: Advances in Quasi-Experimental Design and Analysis. New Directions for Program Evaluation, No. 31. San Francisco: Jossey-Bass; 1986.
  • Fitzsimmons S.J., Carlson K., Kerpelman L.C., Stoner D. A Preliminary Evaluation of the Research Experiences for Undergraduates (REU) Program of the National Science Foundation. Washington, D.C: National Science Foundation; 1990.
  • Hathaway R.S., Nagda B.A., Gregerman S.R. The relationship of undergraduate research participation to graduate and professional education pursuit: an empirical study. J. Coll. Stud. Dev. 2002;43:614–631.
  • Kremer J.F., Bringle R.G. The effects of an intensive research experience on the careers of talented undergraduates. J. Res. Dev. Educ. 1990;24:1–5.
  • Lopatto D. The essential features of undergraduate research. Counc. Undergrad. Res. Q. 2003a;24:139–142.
  • Lopatto D. What undergraduate research can tell us about research on learning. 2003b Invited address presented at the meeting of Project Kaleidoscope, University of Richmond, VA. Text posted online by Project Kaleidoscope. http://www.pkal.org/template2.cfm?c_id=1002 (accessed 15 June 2004)
  • Mogk D.W. Undergraduate research experiences as preparation for graduate study in geology. J. Geol. Educ. 1993;41:126–128.
  • Nagda B.A., Gregerman S.R., Jonides J., von Hippel W., Lerner J.S. Undergraduate student-faculty partnerships affect student retention. Rev. Higher Educ. 1998;22:55–72.
  • Seymour E., Hunter A.-B., Laursen S., DeAntoni T. Establishing the benefits of research experiences for undergraduates: first findings from a three-year study. Sci. Educ. 2004;88:493–594.
  • Tomovic M.M. Undergraduate research—prerequisite for successful lifelong learning. ASEE Annu. Conf. Proc. 1994;1:1469–1470.
  • Zydney A.L., Bennett J.S., Shahid A., Bauer K.W. Impact of undergraduate research experience in engineering. J. Eng. Educ. 2002;91:151–157.

Articles from Cell Biology Education are provided here courtesy of American Society for Cell Biology
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • Cited in Books
    Cited in Books
    PubMed Central articles cited in books
  • PubMed
    PubMed
    PubMed citations for these articles