HHMI Grants to Kenyon

Survey of Summer Science Research Alumni
at Kenyon College

Conducted by Sarah Murnen, Dana Krieg, and Joan L. Slonczewski
November 7, 2003

The purpose of the Alumni Survey was to assess the contributions of undergraduate summer science research to science education, and to compare the value of summer research experience with regular academic course work. We hoped to address questions such as:

• What are students getting out of summer research that they cannot get from academic year courses?
• Kenyon 's regular laboratory courses contain strong investigative components. Is it possible to provide the benefits equivalent to summer research through our regular lab courses, instead of the relatively expensive summer programs?
• What do alumni feel they have gained from summer research? Have they pursued employment or further study in science?
• What possible areas for improvement of our summer research program may we identify?

Surveys were completed in February, 2003 by 174 alumni of the summer science scholarship program, who graduated from Kenyon College between 1985 and 2002. Fifty-seven percent of the respondents were women and 12.8% identified themselves as members of an ethnic minority group.

The survey form was designed by Associate Provost Sarah Murnen at Kenyon College. The form avoided communicating to alumni that summer research assessment was its main purpose; instead, the questions were designed to address various academic experiences in an equivalent way. Respondents rated the contributions of their lecture and seminar courses, laboratory or science methods courses, and the summer science experience to various academic goals.

For each type of science experience:

• Lecture and Seminar Courses
• Laboratory or Science Methods Courses
• Kenyon Summer Science Scholars
• Other Science Experience

Respondents were asked the following set of questions:

Regarding your experience in lecture and seminar courses at Kenyon, please indicate how much you agree with each statement. My lecture and seminar courses…

1. _____ increased my scientific knowledge
2. _____ helped me improve my scientific writing
3. _____ encouraged me to share ideas with Kenyon faculty
4. _____ encouraged me to critically examine research findings
5. _____ helped me generate scientific research questions of my own
6. _____ helped me test scientific research questions of my own
7. _____ helped me develop technical research skills
8. _____ helped me feel like a scientist
9. _____ encouraged me to contemplate a career in science

Respondents were asked to rate their responses on the following scale:

1 = disagree strongly
2 = disagree
3 = disagree to a slight extent
4 = neither disagree nor agree
5 = agree to a slight extent
6 = agree
7 = agree strongly

Results:

For each question, the mean difference was calculated between the scores for the summer research and for either lecture/seminar or laboratory courses. The score differences are plotted below, with error bars representing SEM (N=174). For questions 3 through 8, a clear advantage was reported for the summer research as compared with academic year courses. Even in comparison with laboratory courses, students felt that summer research better "encouraged me to critically examine research findings," "helped me generate scientific research questions of my own," and "helped me develop technical research skills."

Two areas where the summer research experience showed no significant advantage over academic courses were "increased my scientific knowledge" in comparison with lecture/seminar courses, and "helped me improve my scientific writing" in comparison with laboratory courses.

Statistical analysis:

Paired samples t-tests were used to compare responses to questions about the benefits of lecture and seminar courses to the summer science experience, as well as to compare the benefits of laboratory and methods courses to summer science. Because of the number of tests conducted, the Holm’s Sequential Bonferonni method was used to control for Type I error. We found that respondents felt that the summer science experience encouraged sharing of ideas with faculty (question 3) and critical examination of research findings (question 4), helped students generate (question 5) and test (question 6) research questions, developed technical research skills (question 7) and helped students feel like scientists (question 8) significantly more than either lecture or laboratory courses. In addition, students reported that summer science helped them to improve scientific writing (question 2) significantly more than lecture classes did, although not more than laboratory courses. Summer science also encouraged students to contemplate a career in science (question 9) significantly more than did the laboratory courses.

Employment history and further study:

After participation in summer science, 40.5% presented their research at a conference, 18.5% presented their research in a research abstract, and 11.6% presented their research in a research article. The majority of the summer science alumni continue to work in a scientific field: 22.5% attended (or are attending) medical school, 50.3% attended (or are attending) graduate school, 3.5% attended veterinary school, and 4% are working as research assistants or laboratory technician.

Conclusions:

From our alumni survey we conclude that alumni believe our undergraduate summer research program provides substantial benefits to their education that are not received from academic year courses, even Kenyon's investigative laboratory course. These results are limited however by the fact that no distinctions are made within the major categories of science experience; some courses are of course more investigative than others. Also, the survey covered the past twenty year, a period over which Kenyon's course offerings have developed considerably.

The survey identified scientific writing as a possible area for future development of our summer research program. We are investigating this possibility further; we have already conducted a small program of workshops on scientific writing for our summer research students, and we are considering future expansion of this program.

Our summer research alumni report substantial continued involvement in science-related careers. Fully half the respondents are attending, or have attended, graduate school in a science field.