1. Diane Ebert-May Michigan State University ebertmay@msu.edu http://first2.org Scientific Teaching and Interactive Learning: Strategies to Maximize Student Understanding

3. Engage

4. Question 1 Active learning strategies enable students to learn science better than passive lectures. Please respond on a scale of 1-5: 1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly disagree

6. Large Class Meeting

7. Class Meeting

9. Question 2 Students learn science best by “doing” science. Please respond on a scale of 1-5: 1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly disagree

14. Question 3 How important is it to use multiple kinds of assessments to determine student learning? Please respond on a scale of 0-100 in increments of 10:

15. Question 4 The proportion of assessments I use in my course that demonstrate students’ critical thinking abilities is.... Please respond on a scale of 0-100 (%) in increments of 10:

16. Connections among concepts Organization of concepts Visual representations Model-based reasoning Test models What is critical thinking?

17. Question 5 In my department, excellence in teaching is rewarded at a level comparable to excellence in research. Please respond on a scale of 1-5: 1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly disagree

19. Articles derived from journal papers

20. How People Learn Bransford et al 1999, 2004

21. Explore: Out of Thin Air

22. What is going on? Brainstorm: talk to your neighbor 1. Diagnose situation - what is the learning challenge 2. Where is the missing link?...misconception?

23. How and when do you identify student learning difficulties? Don’t have to grade Pre-test (e.g., diagnostic questions - identify misconceptions) Engagement activity - brain teaser, discussion starter, ‘need to know’ questions Surveys or polls (clickers?) Others May use pretest or diagnostic (clicker) question

24. Photosynthesis as Energy Biomass from Soil Energy as Biomass All Green Plant Altruism Thin Air Respiration as ‘breathing’ Misconceptions about Photosynthesis, Respiration, and the Carbon Cycle

25. Radish Problem in Ebert-May D, Batzli J, Lim H. 2003. Bioscience 53:1221-1228. Experimental setup: Weighed out 3 batches of radish seeds each weighing 1.5 g. Experimental treatments: 1. Seeds placed on DRY paper towels in LIGHT 2. Seeds placed on WET paper towels in LIGHT 3. Seeds placed on WET paper towels in DARK

26. Problem (cont) After 1 week, all plant material was dried in an oven overnight (no water left) and plant biomass was measured in grams. Predict the biomass of the plant material in the various treatments. No Water, light [yellow] Water, light [pink] Water dark [green] No idea [orange]

27. Results Mass of Radish Seeds/Seedlings 1.46 g 1.63 g 1.20 g Write an explanation about the results. Explain the results. Write individually on carbonless paper.

28. Midterm Assessment Hypothetical scenario: Grandma Johnson had very sentimental feelings toward Johnson Canyon, Utah, where she and her late husband had honeymooned long ago. Her feelings toward this spot were such that upon her death she requested to be buried under a creosote bush overlooking the canyon. Trace the path of a carbon atom from Grandma Johnson’s remains to where it could become part of a coyote. NOTE: the coyote will not dig up Grandma Johnson and consume any of her remains.

29. Coding Scheme

30. Correct Student Responses (%) Cellular Respiration by Decomposers Bio1/Bio2Other/Bio2 Friedmans, p<0.01

31. Pathway of Carbon in Photosynthesis Bio1/Bio2 Correct Student Responses (%) Other/Bio2 Friedmans, p<0.05

32. Deep within a remote forest of Guatemala, the remains of a spider monkey were buried under an enormous mahogany tree. Although rare, jaguars (big cats - carnivores) were spotted in this forest by local farmers. Explain how a carbon atom in carbohydrates contained within the muscle cells of the spider monkey could become part of a cell within the stomach lining of a jaguar. Note: the jaguar does not dig up the monkey and eat the remains! Final Exam Question

33. Final Exam Q Deep within a remote forest of Guatemala, the remains of a spider monkey were buried under an enormous mahogany tree. Although rare, jaguars (big cats - carnivores) were spotted in this forest by local farmers. Explain how a carbon atom in carbohydrates contained within the muscle cells of the spider monkey could become part of a cell within the stomach lining of a jaguar. Note: the jaguar does not dig up the monkey and eat the remains!

34. Make a clearly labeled box model of the system. Use the template on the Answer Sheet. You will not need all of the boxes. In the model, clearly label the processes (next to arrows), organisms or places, and forms (carbon pools) the carbon atom must go through to cycle within the ecosystem. Use the organisms/places and pools from the lists below. To do so... Place or Organism [not listed in a specific order] Carbon Pools [form] Atmosphere Jaguar (carnivore) Mahogany tree (producer) Bacteria (decomposer) Spider Monkey (herbivore) Tapir (similar to a pig) (herbivore) CO 2 gas Carbohydrate

35. Process Pool Place Pool Place Pool Place Process Reminder: format for a box model:

37. Circles identify key portions of box model. Orange circles identify difficult portions for students.

40. Explain

41. What is assessment? Data collection with the purpose of answering questions about… students’ understanding students’ attitudes students’ skills instructional design and implementation curricular reform (at multiple grain sizes) Informing BOTH instructors and students about learning.

42. Assessment Gradient High Ease of Assessment Low Multiple Choice, T/F Diagrams, Concept maps, Quantitative response Short answer Essay, Research papers/ reports Oral Interview Low Potential for Assessment of Learning High Theoretical Framework Ausubel 1968; meaningful learning Novak 1998; visual representations King and Kitchner 1994; reflective judgment National Research Council 1999; theoretical frameworks for assessment

43. What level of learning do we ask of our students? Bloom (1956) Cognitive Domain of Educational Objectives 6 categories - Knowledge Comprehension Application Analysis Synthesis Evaluation

44. How am I going to grade all this stuff??

45. Subsample= You don’t need to grade everything!! Classroom Assessment Techniques (Angelo & Cross 1993); Muddiest Point, Minute papers etc.. Pyramid Exams- Individual 75% + Group 25% Diagnostic Questions & Clickers Rubrics Assessment and Feedback Approaches

46. Learning Objective Identify desired results Learning Outcome Determine acceptable evidence Assessments Data collected & Feedback given Instructional Design & Activities Planned learning experiences and instruction Like This? Backward Design Adapted from Wiggins and McTighe 1998, 2005

47. Backward Design

48. Learning Outcome Students will demonstrate understanding of evolution and natural selection by designing and testing a research question, explaining the results. Learning Outcome

49. Changes in a population occur through a gradual change in individual members of a population. New traits in species are developed in response to need. All members of a population are genetically equivalent, variation and fitness are not considered. Traits acquired during an individual’s lifetime will be inherited by offspring. Alternative Conceptions: Natural Selection

50. Write a scenario that explains the phenotypic changes in the trees and animals. Use your understanding of evolution by natural selection.

51. Scoring Rubric for Quizzes and Homework Ebert-May http://www.flaguide.org/cat/rubrics/rubrics1.php

52. Coding Student Responses MisconceptionsCorrect Change in the individualChange in the population Need to Change/ Must Change/ Choice Change due to genes All members of a population are equally fit Individuals within a population have varying fitness levels Traits acquired during a lifetime are passed on Genetic traits help the individual to survive and reproduce

55. If faculty change....

56. Do students learn better?

57. FIRST/SI Study of Faculty Change 75 Faculty from Research and Comprehensive Universities, Liberal Arts and Community Colleges FIRST -- multiple workshops, over 2-4 years Summer Institute at U Wisconsin - one week

58. RTOP Assess Path Model of Faculty Professional Development Course Size Self Report Assessment Department Teaching: Research Pedagogy Professional Develop Self Report Practice

59. Does self report of teaching practice correlate with observed practice? RTOP Assess Course Size Self Report Assessment Department Teaching: Research Pedagogy Professional Develop Self Report Practice

60. Does self report of teaching practice correlate with observed practice? Standard Regression used to Test Hypothesis RTOP Self Report Practice Does self report of teaching practice correlate with observed practice?

61. Standard Regression used to Test Hypothesis Self Report of Practice RTOP low high Strong relationship: use self report data to predict classroom practice. No relationship: Must observe in classroom.

62. Path Model of Faculty Professional Development RTOP Assess Course Size Self Report Assessment Department Teaching: Research Pedagogy Professional Develop Self Report Practice

63. Does the number of forms (types) of assessment influence the Bloom's level of items on exams? RTOP Assess Course Size Self Report Assessment Department Teaching: Research Pedagogy Professional Develop Self Report Practice

64. Does the number of forms (types) of assessment influence the Bloom's level of items on exams? Assess Course Size Self Report Assessment

65. Does the number of forms (types) of assessment influence the Bloom's level of items on exams? Standard Regression used to Test Hypothesis Course Size Mean Bloom low high Class Logistics: Lower Bloom’s level questions are quicker to grade. No relationship: Type and frequency of assessment are not related to Blooms. Self Report Assessment low high

66. What is the impact of course size on pedagogy? RTOP Assess Course Size Self Report Assessment Department Teaching: Research Pedagogy Professional Develop Self Report Practice

67. What is the impact of course size on pedagogy? RTOP Course Size

68. What is the impact of course size on pedagogy? Standard Regression used to Test Hypothesis Number of Enrolled Students RTOP low high Strong negative relationship: Active learning methods are easier to use lower enrollment courses. No relationship: There is no difference based upon course enrollment.

69. Sub-models for analysis... RTOP Assess Course Size Self Report Assessment Department Teaching: Research Pedagogy Professional Develop Self Report Practice

70. Assessment Database: Structure and Function De-identification Software Data Upload SpreadsheetPseudonym Spreadsheet Sample Spreadsheet for a Course Secure Database Data Upload Software Public Database Data Verification Verified de-identified data Faculty Computer

71. -Find assessment instruments -Prepare metadata and upload student assessment data -Analyze and download data -Archive records of analyses How can I use the database?

72. Based on Ecological Metadata Standards (Michener 1997) Describe what collected, who collected, where collected, when collected, how collected, why collected “Educational Metadata Standard” EdMS

73. What is in the Educational Metadata Standard? Where –Institution, class size How –Experimental and sampling design –Administration of assessments –Instructional design Who –Project personnel What - Assessment instruments -Rubrics and assessment concepts Why - Classroom study

74. Metadata also include... Taxonomies –Bloom’s Taxonomy - cognitive Concept Categories of Biology –Evolution –Carbon Cycling Identification of Published Instruments –Concept Inventory of Natural Selection (CINS)

75. Metadata - questions can be linked to more than one concept In guppy populations, what are the primary changes that occur gradually over time? a. The traits of each individual guppy within a population gradually change. b. The proportions of guppies having different traits within a population change. c. Successful behaviors learned by certain guppies are passed on to offspring. d. Mutations occur to meet the needs of the guppies as the environment changes. Question on Assessment Concept category - Evolution Anderson’s 10 concepts - change in population - variation heritable Instrument CINS Concept category - Evolution Dino/Plant rubric - P - change in population - G - genetic traits help individual survive and reproduce

76. Using the Database: Faculty Questions Question: An instructor wants to determine student learning gains about evolution. What evolution assessment tools are available?

77. What evolution assessment tools are available? Search database by concept category ✓ Evolution Narrow questions by more specific classification criteria ✓ Change occurs in populations ✓ Change due to change in genes Narrow questions by availability of student results ✓ Large class size (>100 students) ✓ Majors Biology class ✓ R1 Institution

78. Finding Assessments: Search Database by Concept Category Concept category - Evolution 1. In guppy populations, what are the primary changes that occur gradually over time? 2. What would happen if a breeding pair of finches was placed on an island under ideal conditions? 3. What type of variation is passed on to finches over time? 1. In guppy populations, what are the primary changes that occur gradually over time? 2. What would happen if a breeding pair of finches was placed on an island under ideal conditions? 3. What type of variation is passed on to finches over time? 1. In guppy populations, what are the primary changes that occur gradually over time? 2. What would happen if a breeding pair of finches was placed on an island under ideal conditions? 3. What type of variation is passed on to finches over time? 1. In guppy populations, what are the primary changes that occur gradually over time? 2. What would happen if a breeding pair of finches was placed on an island under ideal conditions? 3. What type of variation is passed on to finches over time? 2000 questions from 1500 assessments

79. Narrow 2000 questions by more specific classification criteria Concept category - Evolution a.Anderson’s 10 Concepts/Misconceptions of Evolution b. Dino/Plant Rubric (unpublished rubric by Ebert-May and Linton) c. Sexual Selection i.P – change in population ii.C – change due to change in genes iii.V – individuals within a population have varying levels of fitness iv.G – genetic traits help the individual to survive and reproduce 1. In guppy populations, what are the primary changes that occur gradually over time? 2. What would happen if a breeding pair of finches was placed on an island under ideal conditions? 3. What type of variation is passed on to finches over time? 200 questions from 150 assessments

80. Narrow 200 questions further by course criteria Concept category - Evolution Dino/Plant rubric - P - change in population - G - genetic traits help individual survive and reproduce Criteria Class Size:>100 students Subject:Biology Target Students:Majors Institution Type:Research 1 1. In guppy populations, what are the primary changes that occur gradually over time? 2. What would happen if a breeding pair of finches was placed on an island under ideal conditions? 3. What type of variation is passed on to finches over time? 50 questions from 15 assessments

81. 1. In guppy populations, what are the primary changes that occur gradually over time? a. The traits of each individual guppy within a b. The proportions of guppies having different. c. Successful behaviors learned by. d. Mutations occur to meet the needs. 2. In guppy populations, what are the primary changes that occur gradually over time? a. The traits of each individual. b. The proportions of guppies. c. Successful behaviors. d. Mutations occur to meet. 3. In guppy populations, what are the primary changes that occur gradually over time? a. The traits of each b. The proportions of guppies having. c. Successful behaviors learned. d. Mutations occur to meet the. Link Questions with Metadata 1. In guppy populations, what are the primary changes that occur gradually over time? a. The traits of each individual guppy within a b. The proportions of guppies having different. c. Successful behaviors learned by. d. Mutations occur to meet the needs. 2. In guppy populations, what are the primary changes that occur gradually over time? a. The traits of each individual. b. The proportions of guppies. c. Successful behaviors. d. Mutations occur to meet. 3. In guppy populations, what are the primary changes that occur gradually over time? a. The traits of each b. The proportions of guppies having. c. Successful behaviors learned. d. Mutations occur to meet the. Assessment #1 Assessment #2 Concept Categories of Biology Evolution Concepts Weber, 6/1/2006

82. Metadata - Rating Quesitons In guppy populations, what are the primary changes that occur gradually over time? a. The traits of each individual guppy within a population gradually change. b. The proportions of guppies having different traits within a population change. c. Successful behaviors learned by certain guppies are passed on to offspring. d. Mutations occur to meet the needs of the guppies as the environment changes. Question on Assessment Taxonomy Bloom 1956 2 - Comprehension Ebert-May, 2/27/2007 Taxonomy Bloom 1956 3 - Application Urban-Lurain, 1/20/2007 Taxonomy Bloom 1956 3 - Application Weber, 2/27/2007

83. Finding assessment instruments Administer the instrument, teach, and prepare student data Prepare metadata and upload student assessment data Analysis and download Results of analyses How do I use the database?

84. Team at MSU Rett Weber - Plant Biology (postdoctoral researcher) Deb Linton - Plant Biology [Central Michigan U] Duncan Sibley - Geology Doug Luckie - Physiology Scott Harrison - Microbiology [postdoctoral fellow] Tammy Long - Plant Biology Heejun Lim - Chemistry Education Rob Pennock - Philosophy Charles Ofria - Engineering Rich Lenski - Microbiolgy Janet Batzli - Plant Biology [U of Wisconsin]

85. “...we note that successful people are the ones who take advantage of those around them to ultimately benefit students.” Ebert-May D, Weber R, Hodder J, Batzli J (2006) Finally...