2. Diane Ebert-May Department of Plant Biology Michigan State University www.first2.org Engaging Students: Techniques and Strategies to Enhance Student Learning HHMI

3. Goals for Workshop As a result of your participation, you will... l Think about large courses as environments where you teach less and students learn more. l Experience a learning cycle model of instruction l Analyze strategies and instructional designs for learner-centered, large courses l Articulate significant learning goals for your course/curriculum l Determine the ‘why’ and practical use of multiple forms of assessment l Use assessment data to drive instructional decisions

4. Learning Cycle: Models for Instruction Karplus et al: BSCS »Exploration Engage »Concept IntroductionExplore »Concept ApplicationExplain » Elaborate

6. Engage Questions are designed to: »Find out more about students’ thinking »Level the playing field (everyone involved) »Remind students they HAVE a role in this course »Unveil alternative/misconceptions

7. “Consensogram” Directions 1. Take one color-coded post-it for each question, write the question # in the corner. 2. Write a number between 0-100 on each post-it in increments of 10. 3. Please do not share responses.

8. “Consensogram” Questions Please respond on a scale of 0 -100 in increments of 10: 1.Teaching is a human endeavor that does not and cannot improve over time. (100 agree - 0 disagree) 2.Human beings are fantastic learners. (100 agree - 0 disagree) 3.Humans don’t learn well in a teaching-centered classroom. (100 agree - 0 disagree) 4.What percent of your students appear to understand concepts in your course “very well” during class time, but perform disappointingly “less well” on the exams?

9. Consesogram Qs (2) 5.To what degree do the assessments you use in your courses provide convincing data about student learning? 6.How important is it to use multiple kinds of data to assess your students? 7.How often do you use data to make instructional decisions? 8.In my department, teaching is as important as research and is rewarded accordingly (100 agree - 0 disagree)

10. Cooperative Groups A type of formal structure for in class activities. 4 students per group Person A, B, C, D in each group (assign/select) First - read problem/think about task individually Discuss: A with B C with D Form group consensus

11. Explore Introduce concepts, ideas Ask more questions Find out more about what students know, misunderstand

12. Assessment in Teaching Parallels Assessment in Research â We ask questions and develop hypotheses and to solve problems and make predictions about student learning. â Our questions are based on current knowledge and theories, are creative, original and relevant to the investigator. â Research designs and methods we use to collect data are logical arguments to answer questions. â Instruments/techniques we use are valid, repeatable measures of learning. â Assessment (results) help us understand student thinking. â Results drive our next questions or decisions about a course. â Our ideas are peer reviewed - informally or formally

13. What is assessment? Data collection with the purpose of answering questions about… »student understanding »students’ attitudes »students’ skills »instructional design and implementation

14. Graduate Education Often excellent at preparing individuals to design and carry out disciplinary research.

15. Graduate Education Often inadequate and haphazard in preparing future faculty/professionals to take on the increasingly complex demands of the professoriate. Teaching is not mentored, peer reviewed, or based on accumulated knowledge.

16. Solution: IRD model Intergenerational research teams (IRDs) in cooperative academic environments »Who: senior faculty, junior faculty, postdoctoral and graduate students. »What: scholarship of science teaching and learning is fully integrated into the professional culture along with discipline-based activities. Assessment is critical to both practices.

17. Collaborators Janet Batzli - Plant Biology (University of Wisconsin) Doug Luckie - Physiology Scott Harrison - Microbiology (graduate student) Tammy Long - Plant Biology Jim Smith - Zoology Deb Linton - Plant Biology (postdoc) Heejun Lim - Chemistry Education (postdoc) Duncan Sibley - Geology

18. Recognizing and Rewarding Evaluating and Improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics (2003) »National Research Council »www.nap.edu/catalog/10024.htmlwww.nap.edu/catalog/10024.html

19. What are central questions about learning? 1.What do we want our students to know and be able to do? 2.What knowledge or misconceptions do our students bring to the course? 3.What evidence will we accept that students know and can do? 4.How does our instruction help learning?

24. Cognitive Theory “Learners are not simply passive recipients of information; they actively construct their own understanding.” Svinicki 1991

25. What Type of Learning? Bloom (1956) 6 major categories in the Cognitive Domain of Educational Objectives

26. Cognitive Levels Knowledge - remember Understanding and Application - grasp meaning, use, interpret Critical Analysis - original thinking, open-ended answers, whole to parts, parts to whole, evaluation

27. What is assessment? Data collection with a purpose »gather data about students’ learning. »use tools like Bloom’s taxonomy to ‘calibrate’ data

28. What type of data do we gather? Depends on the evidence we will accept that students have learned what we want them to learn. Data must be aligned with the course goals. Measures of knowledge, attitudes, and skills. »tests, extended responses, concept maps, »research papers, teamwork, communication

30. Assessment -> Inseparable from Instruction What kind of data do you want from the assessment? (non-trivial?) How is data collection embedded in context of learning over time? Is assessment of student learning direct, rather than indirect? How will the data influence your instructional design?

31. False Hopes of Grading (Evaluation) Total objectivity Total agreement Hope for one-dimensional student motivation for learning. From Walvoord and Anderson (1998)

32. Managing Grading 1.Use to enhance learning (socially constructed/context dependent process). 2.Substitute judgment for objectivity. 3.Distribute time effectively. 4.Be open to change - grades/grading systems. From Walvoord and Anderson (1998)

33. Managing Grades (2) 5.Listen and observe student. 6.Communicate and collaborate with students. 7.Integrate grading with other key processes - planning, teaching, interacting. 8.Seize teachable moment - emotional process. 9.Make student learning primary goal - involve them with high expectations, assessment, feedback.

34. Managing Grades (3) 10. Be a teacher first, gatekeeper last. 11. Encourage learning-centered motivation. 12. Emphasize student involvement.

35. Explain Now: Let us work through a detailed example of assessment of student understanding…

36. Learning Goal Students will be able to demonstrate their understanding of photosynthesis and cellular respiration. Tools: multiple forms of assessment Feedback loop to instructional design

37. Some Common Misconceptions about Photosynthesis & Respiration Concept 1: Matter disappears during decomposition of organisms in the soil. Concept 2: Photosynthesis as Energy: Photosynthesis provides energy for uptake of nutrients through roots which builds biomass. No biomass built through photosynthesis alone. Concept 3: Thin Air: CO 2 and O 2 are gases therefore, do not have mass and therefore, can not add or take away mass from an organism. Concept 4: Plant Altruism: CO 2 is converted to O 2 in plant leaves so that all organisms can ‘breathe’. Concept 5: All Green: Plants have chloroplasts instead of mitochondria so they can not respire.

38. Instructional Design Two class meetings on carbon cycle (160 minutes) Active, inquiry-based learning »Cooperative groups »Questions, group processing, large lecture sections, small discussion sections, multi-week laboratory investigation »Homework problems including web-based modules Different faculty for each course »One graduate/8-10 undergraduate TAs per course

39. Experimental Design Two introductory courses for majors: »Bio 1 - organismal/population biology (faculty A) »Bio 2 - cell and molecular biology (faculty B) Three cohorts: »Cohort 1 Bio 1 »Cohort 2 Bio1/Bio2 »Cohort 3 Other/Bio2

41. Assessment Design Multiple iterations/versions of the carbon cycle problem Pretest, midterm, final with additional formative assessments during class Administered during instruction »Semester 1 - pretest, midterm, final exam »Semester 2 - final exam

42. Multiple choice question (pre-post) The majority of actual weight (dry biomass) gained by plants as they progress from seed to adult plant comes from which one of the following substances? a. Particle substances in soil that are take up by plant roots. (15%). b. Molecules in the air that enter through holes in the plant leaves (4%). c. Substances dissolved in water taken up directly by plant roots. (28%). d. Energy from the sun (29%). N=138

43. Radish Problem (formative) Experimental Setup: Weighed out 3 batches of radish seeds each weighing 1.5 g. Experimental treatments: »1. Seeds placed on moistened paper towels in LIGHT »2. Seeds placed on moistened paper towels in DARK »3. Seeds not moistened (left DRY) placed in light

44. Radish problem (2) After 1 week, all plant material was dried in an oven overnight (no water left) and plant biomass was measured in grams. Predict the biomass of the plant material in the various treatments. »Water, light »Water, dark »No water, light

45. Results: Weight of Radish Plants 1.46 g1.63 g 1.20 g Write an explanation about the results.

46. Assessment - depends on purpose Reports from groups, formative Peer evaluation Individual evaluation by instructor Score - 5 points

47. Whale Problem (midterm Bio 1) Two fundamental concepts in ecology are “energy flows” and “matter cycles”. In an Antarctic ecosystem with the food web given above, how could a carbon atom in the blubber of the Minke whale become part of a crabeater seal? Note: crabeater seals do not eat Minke whales. In your response include a drawing with arrows showing the movement of the C atom. In addition to your drawing, provide a written description of the steps the carbon atom must take through each component of the ecosystem Describe which biological processes are involved in the carbon cycle.

48. Grandma Johnson Problem (final, Bio 1) Hypothetical scenario: Grandma Johnson had very sentimental feelings toward Johnson Canyon, Utah, where she and her late husband had honeymooned long ago. Her feelings toward this spot were such that upon her death she requested to be buried under a creosote bush overlooking the canyon. Trace the path of a carbon atom from Grandma Johnson’s remains to where it could become part of a coyote. NOTE: the coyote will not dig up Grandma Johnson and consume any of her remains.

49. Jaguar Problem (final, Bio 2) Deep within a remote forest of Guatemala, the remains of a spider monkey have been buried under an enormous mahogany tree. Although rare, jaguars have been spotted in this forest by local farmers. Use coherently written sentences and clearly labeled drawings to explain how a carbon atom in glucose contained within muscle cells of the spider monkey might become part of a cell within the stomach lining of a jaguar. (Note:The jaguar does not dig up the monkey and eat the remains!) Include in your answer descriptions of the key features (not complete biochemical pathways!) of the organismal and cellular processes that explain how the carbon atom of the monkey’s corpse could become a part of the jaguar’s body.

50. Analysis of Responses Used same scoring rubric for all three problems - calibrated by adding additional criteria when necessary, rescoring: Examined two major concepts: Concept 1: Decomposers respire CO 2 Concept 2: Plants uptake of CO 2 Explanations categorized into two groups: Organisms (trophic levels) Processes (metabolic)

51. Trace Carbon from Whale to Seal (Bio1 students, n=141) Organism Process Concept 1 Decomposers respire CO 2 Concept 2 Plants uptake CO 2 Respiration Release CO 2 Primary produces Through AirThrough RootGlucose Photosynthesis 0 20 40 60 80 100 % Decomposers

52. Cellular Respiration by Decomposers (Bio1/Bio2 students, n=63)  2 (2) = 20.16, p < 0.01 Q 1 WhaleQ 2 Grandma JQ 3 Jaguar 0 20 40 60 80 100 % Concept 1: Decomposers respire CO 2

53. Pathway of Carbon into Primary Producer (Bio1/Bio2 students, n=63)  2 (2) = 4.778, p =.092 0 20 40 60 80 100 Q 1 WhaleQ 2 Grandma JQ 3 Jaguar % Air Root Concept 2: Plants uptake CO 2

54. Trace Carbon from Spider Monkey to Jaguar Respiration NA  2 (1) = 14.59, p <.01 0 20 40 60 80 100 % Bio1/Bio2 (n=63) 0ther + Bio2 (n=40) Concept 1: Decomposers respire CO 2

55. Pathway of Carbon into Primary Producer Concept 2: Plants uptake CO 2 0 20 40 60 80 100 Bio1/Bio2 (n=63) 0ther + Bio2 (n=40) % Air Root NA  2 (1) = 8.89, p < 0.05

56. So What? Problem sets about major concepts: »Diagnostic re: what students understand/misconceptions »Methods; parallel to process in disciplinary research »Predict what prior knowledge students brought to course »Learned what knowledge students’ gained »Unveil new misconceptions »Influenced what we taught, how we taught it

57. So What? (2) Curricular changes: »Bacteria/Archaea metabolism - often omitted »Primary production - models in lab »Source/Sink and carbon flux »‘Spiral’ major concepts - over/over/over »Use of CTOOLS (concept mapping java applet ctools.msu.edu)

58. So Who? Faculty in the disciplines -- use the research approach they know best to gather data about student learning to guide the direction of courses and curriculum.

59. Gene-DNA-Chromosome l Students could explain transcription & translation but not the relation... “Gene-DNA-Chromosome.” l Concept mapping forces students to “Think different” and confront their (mis) understanding.

60. Concept Maps

61. Visual Diagrams or Models are

62. Concept Maps Visual Diagrams or Models are Reflection & Learning promotes Assessment Used for Organization

63. Concept Maps Visual Diagrams or Models are Concepts display connected with Linking Words Reflection & Learning promotes Assessment Used for Organization

64. Concept Maps Visual Diagrams or Models are Knowledge or Understanding represent Concepts display connected with Linking Words Hierarchy has Structure has Reflection & Learning promotes Assessment Used for Organization

65. Concept Maps Visual Diagrams Or Models are Knowledge or Understanding represent Concepts display connected with Linking Words Hierarchy has Structure has Reflection & Learning promotes Assessment Used for Organization Context is constructed with New Information Prior Knowledge

66. Multiple Choice … … Concept Maps … … Essay … … Interview high Ease of Assessment low low Potential for Assessment of Learning high Theoretical Framework Ausubel 1968; meaningful learning Novak 1998; visual representations King and Kitchner 1994; reflective judgement National Research Council 1999; theoretical frameworks for assessment Assessment Gradient

67. Make a draft of a concept map Take those three chapters, concepts from your class, and some post-it notes and make a concept map.

68. Steps to making a concept map 1.List the concepts: brain, genome, dog, plant 2.Arrange them - rank-order in terms of the top, most general, to at the bottom, most specific. 3.Add linking lines that connect the subordinate concepts under the broader ones. 4.Add linking words that indicate the relationship between two linked concepts e.g., connect to, are found in, build proteins inside.

69. Concepts PhotosynthesisGlucose RespirationEnergy Carbon cycleWater DecomposersOxygen Primary producers Consumers Carbon dioxide

70. Make a draft of a concept map Now form a group of three people, merge your post- its and make a concept map of all the science concepts.

71. Open-ended questions Align with learning goals Align with learning goals What thinking skills do you wish to assess, choose one questioning format What thinking skills do you wish to assess, choose one questioning format »interpret data? »write conclusions from previous work? »describe? »solve a problem?

72. Writing Open-ended Questions Write a description of the situation. Write a description of the situation. Write the directions for writing. Write the directions for writing. Develop a simple rubric Develop a simple rubric »Conceptual understanding »Content knowledge »Critical-thinking processes »Communication skills

73. C-TOOLS –Concept Connector  MSU http://hobbes.lite.msu.edu/concept/review http://hobbes.lite.msu.edu/concept/review – username: guest – password: concept –Ebertmay@msu.edu

74. Goals for Today 1.Participate in another learning cycle of instruction. 2.Practice develop a scoring rubric 3.Develop action plans for courses and/or curricula.

75. Goal: explain evolution by natural selection

76. Individual Problem Explain the phenotypic changes in the tree and the animal. Use your understanding of evolution by natural selection.

77. How do we develop rubrics? Describe the goals for the activity, problem, task Select the assessment tasks aligned with goals Develop performance standards Differentiate levels of responses based on clearly described criteria Rate (assign value) the categories

78. Scoring Rubric for Quizzes and Homework

79. Advantages of Scoring Rubrics Improve the reliability of scoring written assignments and oral presentations Convey goals and performance expectations of students in an unambiguous way Convey “grading standards” or “point values” and relate them to performance goals Engage students in critical evaluation of their own performance Save time but spend it well

80. Limitations of Scoring Rubrics Problem of criteria Problem of practice and regular use Scoring Rubric website: »http://www.wcer.wisc.edu/nise/cl1/flag/ Sample Rubrics for Organismal Biology http://www.msu.edu/course/lbs/144/f01

81. Learning Cycle: Models for Instruction Karplus et al: BSCS »Exploration Engage »Concept IntroductionExplore »Concept ApplicationExplain » Elaborate

82. Engage Form groups according to discipline or related interests. 1 min: think about a difficult topic you teach well. 3 min: describe to your peers »Learning goal » your topic » how you teach it » how you assess it Collectively, make a defining features matrix’ on postits »Vertical - teaching strategy characteristics » - assessment strategy characteristics »Horizontal - a column for each person to place (+) or (-)

83. Defining Features Matrix Requires students to categorize concepts according to the presence (+) or absence (-) of important defining features. Provides data on analytic reading and thinking skills.

84. DFM - Procedure Focus on 2-3 concepts that are similar enough to confuse your student Determine features of concepts most critical for the students to recognize List features each concept clearly does or does not possess Matrix - features on left side; concepts on top Check - if each cell can be (+) or (-) Give copies, or use postits with students Explain purpose of matrix and directions for filling in.

85. Data Analysis Scan/compare/tally Ask: »Are students paying more attention to certain features than others? »Are students failing to notice defining differences of specific kinds that would be obvious to an expert?

86. Explore Adaptations of this classroom assessment: Students develop their own defining features matrix on an important topic Design matrices to use more than binary response (e.g., always present, often present, rarely present, never present) Students write statements explaining configuration of data

87. Scoring Rubric for Quizzes and Homework

88. Advantages of Scoring Rubrics Improve the reliability of scoring written assignments and oral presentations Convey goals and performance expectations of students in an unambiguous way Convey “grading standards” or “point values” and relate them to performance goals Engage students in critical evaluation of their own performance Save time but spend it well

89. Limitations of Scoring Rubrics 1.Problem of criteria 2.Problem of practice and regular use Scoring Rubric website: »http://www.wcer.wisc.edu/nise/cl1/flag/ Sample Rubrics for Organismal Biology http://www.msu.edu/course/lbs/144/f01