1. Supported by NSF grant DUE-1043443. Conceptual Assessment: What should your students learn? And how do you know if they have learned it? Joel Michael, PhD Dept of Molecular Biophysics and Physiology Rush Medical College,Chicago, IL jmichael40@gmail.com Rosalind Franklin University Oct. 1, 2014

2. Members of the team (in alphabetical order) Bill Cliff, Niagara University Jenny McFarland, Edmonds Community College Joel Michael, Rush Medical College Harold Modell, Bastyr University Mary Pat Wenderoth, Univ. of Washington Ann Wright, Canisius College Rosalind Franklin University Oct. 1, 2014

3. Mary Pat Harold Ann BillJenny Jeannie Members of the team

4. A long standing problem Rosalind Franklin University Oct. 1, 2014

5. A long standing problem I have been teaching physiology to 1 st year medical students for more years than I want to think about. Rosalind Franklin University Oct. 1, 2014

6. A long standing problem I have been teaching physiology to 1 st year medical students for more years than I want to think about. It has always been the case that... Rosalind Franklin University Oct. 1, 2014

7. A long standing problem I have been teaching physiology to 1 st year medical students for more years than I want to think about. It has always been the case that... WE KNOW MORE THAN WE CAN EXPECT STUDENTS TO LEARN! Rosalind Franklin University Oct. 1, 2014

8. To illustrate the problem Rosalind Franklin University Oct. 1, 2014

9. To illustrate the problem How many of you assign, require, or recommend textbooks for use in your course? Rosalind Franklin University Oct. 1, 2014

10. To illustrate the problem How many of you assign, require, or recommend textbooks for use in your course? How big are these books? How many pages? Rosalind Franklin University Oct. 1, 2014

11. “SIZE” of some PHYSIOLOGY textbooks# of pages Human Anatomy & Physiology Saladin1,248 Marieb and Hoehn1,296 Martini1,110 Undergraduate Physiology Sherwood801 Widmaier et al738 Medical Physiology Berne et al.978 Boron and Boulpaep1,267 Guyton and Hall1,066 Michael et al., 2009

12. To illustrate the problem How many of you expect the students to learn EVERYTHING in the textbooks you assign? Rosalind Franklin University Oct. 1, 2014

13. The problem Too much is known and the amount known is growing every day (the knowledge explosion) Rosalind Franklin University Oct. 1, 2014

14. The problem Too much is known and the amount known is growing every day (the knowledge explosion) We all recognize that students cannot, and should not, be expected to know everything Rosalind Franklin University Oct. 1, 2014

15. The problem Too much is known and the amount known is growing every day (the knowledge explosion) We all recognize that students cannot, and should not, be expected to know everything So, what should your students know? And how should you go about determining that? Rosalind Franklin University Oct. 1, 2014

16. The problem Too much is known and the amount known is growing every day (the knowledge explosion) We all recognize that students cannot, and should not, be expected to know everything So, what should your students know? And how should you go about determining that? And how do you determine whether they do know it? Rosalind Franklin University Oct. 1, 2014

17. Today’s agenda Describe an approach to defining what students should know Describe an approach to determining if they do know it Provide you with a brief example of what has been accomplished in physiology Rosalind Franklin University Oct. 1, 2014

18. How do you determine what your students should know? Rosalind Franklin University Oct. 1, 2014

19. How do you determine what your students should know? Ask yourself what’s important Rosalind Franklin University Oct. 1, 2014

20. How do you determine what your students should know? Ask yourself what’s important Ask your local colleagues Rosalind Franklin University Oct. 1, 2014

21. How do you determine what your students should know? Ask yourself what’s important Ask your local colleagues Ask a broader sampling of your colleagues Rosalind Franklin University Oct. 1, 2014

22. How do you determine what your students should know? Ask yourself what’s important Ask your local colleagues Ask a broader sampling of your colleagues Collectively reach consensus on what are the important things Rosalind Franklin University Oct. 1, 2014

23. “I want my students to learn...” Rosalind Franklin University Oct. 1, 2014

24. “I want my students to learn...” the “BIG IDEAS” that they will be able to use to help understand physiology wherever they encounter it, even if they don’t remember all the details. Rosalind Franklin University Oct. 1, 2014

25. What are “big ideas?” Rosalind Franklin University Oct. 1, 2014

26. What are “big ideas?” (1) By definition, big ideas are important and enduring. Big ideas are transferable beyond the scope of a particular unit... Big ideas are the building material of understanding. They can be thought of as the meaningful patterns that enable one to connect the dots of otherwise fragmented knowledge. Wiggins and McTighe, Understanding by design, expanded 2 nd edition, Arlington, VA: Association for Supervision and Curriculum Design., 2005, pp. 338-339 Rosalind Franklin University Oct. 1, 2014

27. What are “big ideas?” (2) “Each [BIG IDEA] is well tested, validated, and absolutely central to the discipline. Each integrates many different findings and has exceptionally broad explanatory scope. Each is the source of coherence for many key concepts, principles and even other theories in the discipline.” Duschl, R. A., Schweingruber, H. A. and Shouse, A. W. (Editors). (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academies Press. Rosalind Franklin University Oct. 1, 2014

28. Conceptual Assessment in Biology (sponsored by NSF) Three meetings over a three year period First meeting brought together 24 biologist from a WIDE variety of disciplines (A-Z almost) We immediately agreed that before talking about how to do assessment we had to know what we wanted assessed! So we decided to take as much time as needed to generate a list of core concepts in biology. Rosalind Franklin University Oct. 1, 2014

29. Conceptual Assessment in Biology (sponsored by NSF) To the surprise and amazement of all of us it took us not more than 1 hour to reach agreement on a list of 8 core concepts! We did acknowledge that different disciplines emphasize different core concepts to a varying extent. – Physiology emphasizes a slightly different set of concepts than does biochemistry or anatomy. Rosalind Franklin University Oct. 1, 2014

30. “Big ideas” in biology (edited) 1.Living organisms are causal mechanisms... 2. The cell is the basic unit of life. 3. Information flow.... 4.Transformations of matter and energy 5.Homeostasis (and “stability”) 6.... structure and function... 7. Evolution.... 8. All life exists [in] an ecosystem.... From 2 nd CAB workshop (Michael and McFarland, 2008) Rosalind Franklin University Oct. 1, 2014

31. “Big ideas” in biology (edited) 1.Living organisms are causal mechanisms... 2. The cell is the basic unit of life. 3. Information flow.... 4.Transformations of matter and energyBIOCHEM 5.Homeostasis (and “stability”) PHYSIOLOGY 6.... structure and function... ANATOMY 7. Evolution.... 8. All life exists [in] an ecosystem.... From 2 nd CAB workshop (Michael and McFarland, 2008) Rosalind Franklin University Oct. 1, 2014

32. What are “core concepts?” Rosalind Franklin University Oct. 1, 2014

33. What are “core concepts?” Operationally, Core Concepts = Big Ideas Rosalind Franklin University Oct. 1, 2014

34. What are “core concepts?” Operationally, Core Concepts = Big Ideas For me a core concept is what I want students to remember in five years, even if they have forgotten some or all of the details. Rosalind Franklin University Oct. 1, 2014

35. How do you determine the core concepts of your discipline? Rosalind Franklin University Oct. 1, 2014

36. CAB mtgs CAP mtgs Asked colleagues via a survey Compiled a list of 15 core concepts Asked colleagues to rank order items on our list AND designate the top 3 Selected set for the CAP group to further explore Rosalind Franklin University Oct. 1, 2014

37. The core concepts of physiology identified by the CAP group CausalityHomeostasis Cell-cell communicationsInterdependence Cell membraneLevels of organization Cell theoryMass balance EnergyPhysics/chemistry EvolutionScientific reasoning Flow down gradientsStructure/function Genes to proteins Michael and McFarland, 2011) Rosalind Franklin University Oct. 1, 2014

38. The core concepts of physiology ranking by diverse faculty Causality (14)Homeostasis (1) Cell-cell communications (3)Interdependence (4) Cell membrane(1)Levels of organization (12) Cell theory (9)Mass balance (13) Energy (6)Physics/chemistry (10) Evolution (15)Scientific reasoning (8) Flow down gradients (5)Structure/function (7) Genes to proteins (11) Michael and McFarland, 2011) Rosalind Franklin University Oct. 1, 2014

39. The core concepts of physiology ranking by diverse faculty Causality (14)Homeostasis (1) Cell-cell communications (3)Interdependence (4) Cell membrane(1)Levels of organization (12) Cell theory (9)Mass balance (13) Energy (6)Physics/chemistry (10) Evolution (15)Scientific reasoning (8) Flow down gradients (5)Structure/function (7) Genes to proteins (11) Michael and McFarland, 2011) Rosalind Franklin University Oct. 1, 2014

40. The core concepts of physiology to be expanded by CAP Causality (14) Homeostasis (1) Cell-cell communications (3) Interdependence (4) Cell membrane(1)Levels of organization (12) Cell theory (9)Mass balance (13) Energy (6)Physics/chemistry (10) Evolution (15)Scientific reasoning (8) Flow down gradients (5) Structure/function (7) Genes to proteins (11) Rosalind Franklin University Oct. 1, 2014 Michael and McFarland, 2011)

41. Unpacking core concepts Homeostasis, cell-cell communications, and flow down gradients are all, literally, BIG IDEAS, comprised of many smaller ideas. “Unpacking” a core concept is the process of determining in a systematic way what those subsidiary ideas are and how they are organized. Rosalind Franklin University Oct. 1, 2014

42. Unpacking the core concept of homeostasis It is too large for a PowerPoint slide, so I’ll ask you to look at the handout I have provided as I proceed. Rosalind Franklin University Oct. 1, 2014

43. Top level unpacking of the concept of homeostasis H1.The organism maintains a more or less stable internal environment. H2.A substantial change to a regulated variable will result in a physiological response to restore it toward to its normal range. H3.Homeostatic processes require a sensor (“what can’t be measured can’t be regulated”). H4.Homeostatic processes require a control center. H5.Homeostatic processes require effectors. Rosalind Franklin University Oct. 1, 2014

44. Unpacking of component concept of homeostasis H3.Homeostatic processes require a sensor (“what can’t be measured can’t be regulated”). H3.1. Sensors detect the regulated variable and respond by transducing that stimulus into a different signal. H3.2. Sensors respond within a limited range of stimulus values. H3.3. Sensors are constantly active (not just active when the regulated variable is not at the set point value or outside of a ‘normal’ range). H3.4. An organ system may employ a variety of types of sensors (e.g. chemoreceptors, baroreceptors, mechanoreceptors, etc) to regulate variables associated with that organ system Rosalind Franklin University Oct. 1, 2014

45. We are working on unpacking core concepts of: Cell-cell communications Flow down gradients Rosalind Franklin University Oct. 1, 2014

46. PLEASE NOTE... The set of unpacked core concepts DOES NOT constitute the content of a course. Nor is it a physiology curriculum. It is a framework around which one can construct a course or a curriculum. I’m sure this true in every discipline. Rosalind Franklin University Oct. 1, 2014

47. Developing a conceptual assessment instrument on homeostasis Rosalind Franklin University Oct. 1, 2014

48. Developing a conceptual assessment instrument on homeostasis (1) One cannot directly assess student understanding of homeostasis Must assess student understanding of the unpacked, smaller ideas that make up the core concept We made a decision to build a concept inventory targeting undergraduate students taking introductory physiology or A&P courses Rosalind Franklin University Oct. 1, 2014

49. Developing a conceptual assessment instrument on homeostasis (2) We developed questions reflecting our own students’ difficulties with the concept of homeostasis We also solicited observed misconceptions from colleagues at national meetings We took great care to keep the language accessible to our assumed audience The result was a 20 MCQ inventory Rosalind Franklin University Oct. 1, 2014

50. Validating the inventory We had a cohort of physiology teachers tells us about the importance and degree of difficulty of the questions and modified them accordingly We have had 250 students at 7 institutions take the inventory and have concluded a preliminary analysis of their responses Rosalind Franklin University Oct. 1, 2014

51. Question types and results General model Application Rosalind Franklin University Oct. 1, 2014

52. Student responses to HCI Students did better on general model questions (no physiological system identified) than they did on application questions (which reference a specific system). At one institution students in a 400 level course did significant better than students in the required 300 level course AND the 400 level students did not exhibit the difference described above McFarland et al., 2014 Rosalind Franklin University Oct. 1, 2014

53. Conclusions (1) What are the core concepts you want your students to understand? – You will have to unpack these core concepts into their important pieces. This is NOT to suggest that there are not important “facts” they need to know. Your list of core concepts is not a course description or curriculum! Rosalind Franklin University Oct. 1, 2014

54. Conclusions (2) If you want to know if your student understand these “big ideas” you have to write questions that test “big ideas” (their component pieces) and not just facts. – This is at least as difficult, if not more so, than writing your usual exam questions! Rosalind Franklin University Oct. 1, 2014

55. References Michael, J. (2007). Conceptual assessment in the biological sciences: A National Science Foundation-sponsored workshop. Advances in Physiology Education, 31, 389-391. Michael, J., McFarland, J., and Wright, A. (2008). The second Conceptual Assessment in the Biological Sciences workshop. Advances in Physiology Education, 32, 248-251. Michael, J., Modell, H., McFarland, J., and Cliff, W. (2009). The “core principles” of physiology: What should students understand? Advances in Physiology Education, 33, 10-16. Michael, J. and McFarland, J. (2011). The core principles (“big ideas”) of physiology: results of faculty surveys. Advances in Physiology Education, 35, 336-341. Rosalind Franklin University Oct. 1, 2014

56. Michael, J., McFarland, J., Cliff, W., Modell, H., Wenderoth, M. P., and Wright, A. (2013). Homeostasis in undergraduate physiology textbooks. Experimental Biology 2013, Boston, MA. Wright, A., McFarland, J., Cliff, W., Michael, J., Modell, H., and Wenderoth, M. P. (2013). Preliminary results on the prevalence of physiology students’ homeostatic misconceptions. Experimental Biology 2013, Boston, MA. McFarland, J., Michael, J., Wenderoth, M. P., Modell, H., Wright, A., and Cliff, W. (2014). Conceptual assessment of physiology (CAP): Development of a concept inventory for homeostasis. Experimental Biology 2014, an Diego, CA. Wright, A., Michael, J., McFarland, J., Modell, H., Cliff, W., and Wenderoth, M. P. (2014). What visual representations of homeostasis do faculty use? Experimental Biology 2014, San Diego, CA. Rosalind Franklin University Oct. 1, 2014

57. Wright, A., McFarland, J., Cliff, W., Michael, J., Modell, H., and Wenderoth, M. P. (2013). Preliminary results on the prevalence of physiology students’ homeostatic misconceptions. Experimental Biology 2013, Boston, MA. McFarland, J., Michael, J., Wenderoth, M. P., Modell, H., Wright, A., and Cliff, W. (2014). Conceptual assessment of physiology (CAP): Development of a concept inventory for homeostasis. Experimental Biology 2014, an Diego, CA. Wright, A., Michael, J., McFarland, J., Modell, H., Cliff, W., and Wenderoth, M. P. (2014). What visual representations of homeostasis do faculty use? Experimental Biology 2014, San Diego, CA. Rosalind Franklin University Oct. 1, 2014

58. If I can be of any help to you or your colleagues in thinking about or implementing a core concepts approach, don’t hesitate to contact me (jmichael40@gmail.com).jmichael40@gmail.com Rosalind Franklin University Oct. 1, 2014

59. If I can be of any help to you or your colleagues in thinking about or implementing a core concepts approach, don’t hesitate to contact me (jmichael40@gmail.com).jmichael40@gmail.com Thank you for your attention! Rosalind Franklin University Oct. 1, 2014