1. How Effective Are Interactive Biology Tutorials as Learning Enhancement Tools? Jean Heitz, E. Michelle Capes, Robert Jeanne and Jan Cheetham University of Wisconsin, Madison, WI 53706 Abstract Supported by a grant from the UW Madison Transforming Teaching Through Technology (T4) program, we developed "Connecting Concepts” a set of nine interactive tutorials designed for our two-semester introductory- level biology course for majors. These are all available at http://ats.doit.wisc.edu/biology/ and through the Merlot web site (http://www.merlot.org)..http://ats.doit.wisc.edu/biology/ In this study we tested one of these tutorials, Evolution: Species and Speciation, to determine its effectiveness as a learning enhancement tool. Our results indicate the tutorials were most effective for students whose prior grades in the course were less than 80%. The average exam grade for this group increased by 11% over their averages on the previous two exams. We did not see any significant change for other students in the course. Why did we develop the Connecting Concepts interactive tutorials? Increasing class size in recent years has resulted in decreased student-instructor interaction and a large demand for additional learning tools accessible to students outside the classroom. Such tools must also accommodate diverse learning styles and backgrounds. Our goals in developing the tutorials were to: provide students with a type of interactive learning that cannot generally be provided in the classroom. stimulate critical thinking skills, reinforce concepts learned in lecture, and promote the application of both thinking skills and concepts — all in an interactive on-line environment that enhances students’ motivation. In addition, the instantaneous feedback students get on line allows them to assess for themselves how well they are able to understand and apply important concepts What is the Evolution: Species and Speciation tutorial designed to do? In this tutorial students decide whether organisms should be considered separate species using the criteria of three well-known species concepts. They interactively explore each species concept, identifying their strengths and weaknesses. Students become familiar with speciation patterns, and integrate understandings of continental drift with speciation. At the end of the tutorial, students analyze two case studies to determine whether the organisms described are unique species. In doing the case studies they consider morphological traits, haplotypes, population histories, ecology, molecular phylogenies, hybridization, and geographic distributions. How did we analyze the data? Refining the groups In Group 1 only students who verified that they actually completed the tutorial as well as the pre- and post-tests and the final exam were included in the analysis (N=51). In Groups 2 and 3 only students who verified that they did not access the tutorial on-line and who completed pre- and post-tests and the final exam were included (N = 42 and 53 respectively). Each test group was broken down into two approximately equal subgroups: students whose averages on the previous two exams were < 80% and students whose averages were >80%. Analyzing the data One way ANOVA was used to determine whether differences among test groups were significant. Scores on pre-tests were compared between test groups to determine whether there was any significant difference inherent in the groups. Scores on pre- versus post-tests were compared within each test group as a whole and then for 80% subgroups. Total final exam scores were compared to averages on the previous two exams. Total final exam scores were also compared to the scores students would have received for the evolution questions alone. Evolution questions made up 41% of the final exam. What did we discover? Comparison of pretest results indicated no significant difference between Group 1(mean = 51.0) and either Group 2 (mean = 48.6) or 3 (mean = 52.3). Comparing scores on pre- versus post-tests within each test group showed significant differences for all groups (p 80% subgroups. Only the <80% subgroup in Group 1 showed a significant difference when comparing total final exam scores to averages on the previous two exams. Credits: Principle Investigator: Robert Jeanne Departments of Zoology and Entomology UW – Madison Jeanne@entomology.wisc.edu Project Manager: Jan Cheetham Learning Solutions UW-Madison cheetham@wisc.edu Instructional Designers and Consultants: Lee Clippard Learning Solutions lhclippard@wisc.edu Alan Wolf Learning Technology and Distance Education & Center for Biology Education ajwolf@doit.wisc.edu Les Howles Learning Solutions howles@doit.wisc.edu Figure 1. Comparison of prior exam means to final exam scores in Group 1 Subgroup <80 Prior Mean Compared to Final Exam Score Anova Summary Groups Count Sum Ave Variance Column 1 (Prior) 26 1897 72.73 26.804 Column 2 (Final) 26 2162.72 83.18 36.349 Source of Variation SS df MS P-Value Between Grps 1419.842 1 1419.842 1.73E-08 Within Grps 1578.846 50 31.576 Total 2998.687 51 Subgroup >80 Prior Mean Compared to Final Exam Score Anova Summary Groups Count Sum Ave Variance Column 1 (Prior) 25 2137.5 85.5 17.854 Column 2 (Final) 25 2134.31 85.37 35.333 Source of Variation SS df MS P-Value Between Grps 0.204 1 0.204 0.931 Within Grps 1276.482 48 26.593 Total 1276.686 50 How did we test the effectiveness of the interactive Speciation tutorial as a learning enhancement tool? To test the effectiveness of the tutorial on student learning, we randomly divided an introductory biology course (N = 283) into three groups. Group 1 completed the speciation tutorial on-line. Group 2 was given the information from the tutorial in non-interactive pdf format. Group 3 was assigned a general homework question on speciation and was assigned the pdfs for only one of the case studies from the speciation tutorial. All students who gave written consent (N = 276) were asked to complete an evolution/speciation bioinventory test 1 week before and 1 week after the homework assignment. All students received the standard lectures on speciation between the pre- and post test and before the homework was due. The bioinventory questions used for pre- and post- tests were not used on the final. Instead, a different but related questions developed for the final section exam. What do the results tell us about the speciation tutorial as a learning enhancement tool? While some improvement in post test scores was noted across all groups, the overall scores on the unannounced pre- and post-tests were low with means ranging from the high 40s to the low 60s. Scores on the final exam were much higher with means ranging from the high 60s to mid 80s. This confirms what we already know, i.e. while classroom and tutorial experiences can help students learn how to learn, these alone are not sufficient for full development of understanding. In addition, most students require both an impetus (e.g.an exam) and individual study time to more fully develop their understanding. Both Groups 1 and 2 were given the same information for their homework assignment. The only difference was that Group 1 did the tutorial interactively on-line and Group 2 read the information in non-interactive pdf format. The Group 1 students with grades <80% prior to the final exam benefited most from doing the tutorial. This group did better not only on the evolution questions alone (mean = 81.0%) but on the final exam as a whole (mean = 83.2%). These results imply that: the interactivity of the tutorial provides clear learning benefits for this group of students. the interactive tutorial experience provides some learning enhancement not directly related to the specific subject material. Discovering exactly why this occurs warrants further investigation. Acknowledgements – Connecting Concepts Connecting Concepts: Interactive Lessons in Biology (http://ats.doit.wisc.edu/biology/), was produced collaboratively at the University of Wisconsin – Madison by: Transforming Teaching Through Technology (T4), Learning Solutions, Division of Information and Technology Instructors of Introductory Biology 151/152 Project Assistants: Edna Francisco Steven Grunder Sainath Suyanarayanan Ben Schulte Olaf Olson Progammers: Michelle Glenetski Learning Solutions maglenetski@doit.wisc.edu Bahman Zakeri Learning Solutions bahman.zakeri@doit.wisc.edu Cidney Frietag Learning Solutions cfreitag@wisc.edu Various Instructors and Instructional Support Staff, Introductory Biology 151/152 Photo credits: Paul Berry, Botany Department, UW-Madison and http://photos.news.wisc.edu/results.php References: Fraenkel, Jack R. & Wallen, Norman E., 2000. How to Design & Evaluate Research in Education (4th ed.), San Francisco: McGraw-Hill. What did we discover? Comparison of pretest results indicated no significant difference among Groups. Comparing scores on pre- versus post-tests within each test group showed significant differences for all groups. In Group 1, both 80% subgroups showed significant differences. In the other groups, only the >80% subgroup showed a significant difference. Comparing total final exam scores to averages on the previous two exams, only the <80% subgroup in Group 1 showed a significant difference.