1. Identifying Core Concepts and Constructing Concept Inventories Jenny L. McFarland, Ph.D. Biology Department Edmonds Community College Collaborators: Joel Michael (Rush Medical College), Mary Pat Wenderoth (Univ of Washington), Bill Cliff (Niagara University), Ann Wright (Canisius College), Harold Modell (Physiology Education Research Consortium, PERC APS Intersociety Meeting: Comparative Approaches to Grand Challenges in Physiology 7 October 2014, San Diego, CA Supported by NSF grant DUE-1043443

2. Backwards Design & Core Concepts 1.What is “Backwards Design”? 2.What are the core concepts for  undergraduate biology?  physiology education?  comparative physiology education? 3.What is a concept inventory? 4.Development of a homeostasis conceptual assessment for undergraduate physiology.

3. Backwards Design & Core Concepts 1.I dentify desired goals or outcomes  Core Concepts 2.Determine acceptable evidence (design conceptual assessment)  Concept Inventories 3.Design learning experiences/ instruction  Student-centered active learning 4.Evaluate alignment of instruction, outcomes and assessment  instructor metacognition: explicit assessment of teaching & learning Wiggins and McTighe 1998; Dirks, Wenderoth and Withers, 2014

4. Backwards Design in this workshop 1.I dentify desired goals or outcomes  Cynthia Bauerle, Jenny McFarland 2.Determine acceptable evidence (design conceptual assessment)  Douglas Luckie, Jenny McFarland 3.Design learning experiences/ instruction  Barbara Goodman, Douglas Luckie 4.Evaluate alignment of instruction, outcomes and assessment  Miranda Byse – finding resources for doing this!

5. What are the Core Concepts? Step 1 in Backwards Design: Identify desired goals or outcomes for undergraduates  What are the Core Concepts in Biology?  What are the Core Concepts in Physiology?  What are the Core Concepts in Comparative Physiology? – contact Kerry Hull (Khull@UBishops.ca)Khull@UBishops.ca

6. Biology Core Concepts: Vision & Change The Vision & Change report identified 5 core concepts for undergraduate biology  Evolution  Structure and Function  Energy and matter  Information flow  Systems AAAS 2011

7. Physiology Core Concepts Physiology faculty identified 15 core concepts Some physiology core concepts were explicitly identified in the Vision and Change Report, others were not. AAAS 2011; Michael and McFarland 2011  Homeostasis (systems)  Cell Membrane  Cell-Cell Communication  Interdependence  Flow Down Gradients  Energy  Structure/Function  Scientific Reasoning (V&C core competency)  Cell Theory  Physics/Chemistry  Genes to Proteins (info)  Levels of Organization  Mass Balance  Causality  Evolution

8. Comparative Physiology Core Concepts A group of physiology faculty at the APS Institute on Teaching & Learning (APS- ITL, June 2014) have begun to identify Core Concepts in Comparative Physiology. This group has asked me to share their core concepts & competencies with you and request your help. The next five slides will not be discussed in this talk (see handout), but are included here to encourage comparative physiologists here to participate in this effort.

9. Comparative Physiology Core Concepts AP1. Animals use diverse physiological mechanisms/strategies to solve similar environmental challenges. AP2. Animals inhabit diverse habitats and some possess unique physiology (adaptations) allowing for survival in so-called extreme environments. AP3. The evolutionary and developmental origin and history of an animal constrains anatomical structures and physiological processes. AP4. The capabilities of interacting physiological systems represent dynamic trade-offs in function and efficiency. AP5. Body size influences the behavior of physiological systems. AP6. Physiological phenotype is a product of genotype and environment. AP7. Life cycles and life history traits influence physiological processes. AP8. Comparative physiology informs the medical physiology of humans. AP9. Comparative physiology brings to light how body systems interact to meet environmental challenges and promotes an understanding of integrative physiology AP10. Physiological phenomena can be explained in multiple, compatible ways that involve different levels of functional complexity and different scales of time (molecular, cellular, developmental, organismal, environmental/ecological, evolutionary)

10. Concept of “Trade-Offs” Unpacked AP4. The capabilities of interacting physiological systems represent dynamic trade-offs in function and efficiency. 1. Optimal operation of a body system may adversely impact the homeostasis achieved by other body systems and/or the optimal operations of these systems. 2. Body systems may work below optimal capacity in order to reduce or avoid disturbances in homeostasis and/or adverse changes in other body systems. The activities of individual body systems are adjusted to achieve a trade-off in the combined actions of different body systems. This trade-off favors overall homeostasis and results in the most efficient operation of the body under the existing system constraints. 3. Trade-offs in body system activities are prioritized according to the importance of each activity to the immediate survival of the animal. Less commonly, the animal will compromise immediate survival in order to increase reproductive success. 4. A number of factors interact to determine the balance point for physiological tradeoffs. These include: Body size of the animal Life history of the animal Phenotypic plasticity of the animal Environmental conditions

11. Comparative Physiology Core Competencies AP1. Demonstrate knowledge that animals use diverse physiological mechanisms/strategies to solve common environmental challenges. AP2. Demonstrate understanding that animals inhabit diverse habitats and some possess unique physiology (adaptations) allowing for survival in so-called extreme environments. AP3. Recognize that the evolutionary and developmental origin and history of an animal constrains anatomical structures and physiological processes. AP4. Analyze the dynamic trade-offs in function and efficiency seen in interacting physiological systems. AP5. Demonstrate understanding of how body size influences the behavior of physiological systems. AP6. Demonstrate understanding that physiological phenotype is a product of genotype and environment. AP7. Explain how life cycles and life history traits influence physiological processes. AP8. Demonstrate knowledge of how comparative physiology informs the medical physiology of humans. AP9. Recognize how comparative physiology brings to light how body systems interact to meet environmental challenges and promotes an understanding of integrative physiology AP10. Explain physiological phenomena in multiple, compatible ways that involve different levels of functional complexity and different scales of time (molecular, cellular, developmental, organismal, environmental/ecological, evolutionary)

12. “Trade-Offs” Competency with Learning Objectives AP4. Analyze the dynamic trade-offs in function and efficiency seen in interacting physiological systems.  Explain the checks and balances in resource allocation between somatic growth and reproduction.  Explain, using examples, the reasons that physiological systems may be locally, but not 100 %, optimized.  Explain the reason that you would not expect a gill breathing animal to be completely (or fully) homeothermic.  Explain how the competing needs to exchange gases and retain water are met in terrestrial animals.

13. This group would appreciate your feedback! 1.Have we missed major concepts/competencies that faculty expect from their comparative physiology students? 2.How can we use these concepts/competencies to promote the teaching and learning of comparative physiology? Please send comments/suggestions to : Kerry Hull (Khull@UBishops.ca)Khull@UBishops.ca APS ITL - Animal Physiology Group Beth Beason-Abmayr (Rice University)Patricia Halpin (Univ. New Hampshire) Jason Blank (Cal Poly San Luis Obispo)Kerry Hull (Bishop’s University) Sydella Blatch (Stevenson University)Patricia Schulte (Univ. British Columbia) Bill Cliff (Niagara University) Alice Villalobos (Texas A&M)

14. How can we assess Core Concepts? Step 2 in Backwards Design: Determine acceptable evidence (design conceptual assessment)  What are Concept Inventories?  How are Concept Inventories used?  How are Core Inventories developed?  How are Core Inventories validated and assessed?

15. What are Concept Inventories?  A set of questions (inventory) designed “to probe student understanding” of fundamental concepts.  Reliable and validated through statistical analysis. Concept inventories can be used to  assess student understanding and application of concepts.  reveal common, persistent misconceptions that interfere with progression to expert-level understanding.  provide formative assessment during teaching & learning.  assess conceptual learning gains in a course when used as a pre-test and a post-test. Bailey, EB 2011; Smith and Tanner 2010

16. Concept Inventory Development 1.Identify core concepts. (faculty) 2.Develop a conceptual framework for each core concept. (faculty) 3.Understand student thinking & identify “misconceptions”. (students & faculty) 4.Create open-ended questions. (student responses) 5.Create “multiple-choice” questions to assess student thinking. (faculty) 6.Validate questions. (student interviews & faculty surveys) 7.Administer to classes (students) and do statistics (classical test theory and item response theory). Adams & Weiman, 2011

17. Conceptual Assessment for Undergraduate Physiology: a Homeostasis Concept Inventory 1.Developed a conceptual framework for homeostasis (“unpack”) and validate this framework with faculty. (paper in progress) 2.Identified common student “misconceptions”. (paper in progress) 3.Created 20 multiple-choice questions (MCQs)  Created multiple-choice questions (MCQs),  based on the conceptual framework  used common misconceptions as distractors 4.Validated questions (MCQs) 5.Administer to classes to and analyze data to validate the concept inventory as concept inventory. (2014) Michael, McFarland et al. EB abstracts, 2012, 2013, 2014

18. Developing a Homeostasis Concept Inventory 1.conceptual framework: importance & difficulty  Faculty agree that some elements in the framework are more important for students to understand.  Some elements in the framework are more difficult for students to understand. 2.student “misconceptions”: some are “sticky”  Some misconceptions are easy to “correct” or displace with expert- level understanding.  Some misconceptions “stickier”, they are more likely to persist after instruction exposure to expert-level understanding.

19. Homeostasis: Conceptual Framework I.The organism maintains a relatively stable internal environment in the face of fluctuating external environment. II.A substantial change to a regulated variable (a perturbation) will result in a physiological response to restore it toward to its normal range. III.Homeostatic processes require a sensor inside the body (“what can’t be measured can’t be regulated”) IV.Homeostatic processes require a control center (which includes an integrator). V.Homeostatic processes require target organs or tissues, i.e. “effectors”.

20. Conceptual Framework: sensors III.Homeostatic processes require a sensor inside the body (“what can’t be measured can’t be regulated”) i.Sensors detect the regulated variable and respond by transducing that stimulus into a different signal. ii.Sensors respond within a limited range of stimulus values. iii.Sensors generate an output whose value is proportional to the magnitude of the input to the sensor (i.e. the stimulus). iv.Sensors are constantly active (not just active when the regulated variable is not at the set point value). v.An organ system may employ a variety of types of sensors (e.g. chemoreceptors, baroreceptors, mechanoreceptors, etc) to regulate variables associated with that organ system.

21. Framework: Difficulty & Importance  Physiology faculty assessed the difficulty and importance of elements of the framework.  How should we focus our instructional activities? The regulated variable is held stable by a negative feedback system Homeostatic processes require a sensor inside the body

22. Negative Feedback Question “The regulated variable is held stable by a negative feedback system.” This idea was ranked essential (5/5) and easier to understand (3.29/5). The following question addresses this important component of the framework (IIA) and includes student misconceptions as distractors: In organisms, like humans, negative feedback mechanisms results in A. an unfavorable, or damaging effect on the body. B. a constant decrease in the regulated variable. C. equilibrium amongst body cells and fluids. D. maintaining an internal variable within a ‘normal’ range of values.

23. Are these misconceptions “sticky”? In organisms, like humans, negative feedback mechanisms results in

24. Sensor Question Homeostatic processes require a sensor inside the body …” This idea was ranked important (4.67/5) and difficult to understand (2.86/5). The body has a sensor that measures blood pressure, but does not have a sensor that can measure heart rate. Which of the following are held more or less constant even when the internal or external environment changes? A.heart rate B.blood pressure C.Both D.Neither

25. Is there a “sticky” misconception? The body has a sensor that measures blood pressure, but does not have a sensor that can measure heart rate. Which of the following are held more or less constant even when the internal or external environment changes?

26. General Model vs. Application Our Homeostasis questions are either  “general model” questions with no specific physiological system mentioned or  “application” questions, situated in a specific physiological system.  Predict which type students will answer more accurately. Think – Pair - Share

27. General Model vs. Application 4. A homeostatic control mechanism functions to maintain the concentration of X at a relatively constant level. This mechanism is functioning A.when the concentration of X gets too high. [0.8%] B.when the concentration of X gets too low. [0.8%] C.when the concentration of X gets too high or too low. [46.5%] D.at all concentrations of X. [51%] 9. Baroreceptors detect blood pressure. Blood pressure is maintained relatively constant even when the internal or external environment changes. Under what conditions do the baroreceptors send signals to the brain? A.when blood pressure is not at its normal value. [34.2%] B.when blood pressure is increasing. [5.3%] C.when blood pressure is constant. [1.2%] D.at all levels of blood pressure. [58%]

28. General Model vs. Application Students performed better on the general model questions (74.3% correct) than on the application questions (58% correct). Question # 1-20 (x-axis) and % correct (y-axis) from 244 undergraduate student responses.

29. Homeostasis Concept Inventory for Undergraduate Physiology 1.We are writing up a homeostasis conceptual framework for homeostasis paper now. (paper in progress, to be submitted this fall) 2.We are in the first drafts of a homeostasis “misconceptions” paper. (paper in progress) 3.We are doing item analysis of the data from students from ~ 6 pilot institutions to validate the questions. 4.In 2014-2015 we are administering the current draft to students in broad range of courses to validate the concept inventory as a whole.. Michael, McFarland et al. EB abstracts, 2012, 2013, 2014

30. Thank you! Supported by NSF grant DUE-1043443 Collaborators  Joel Michael, Rush Medical College, Chicago IL  Mary Pat Wenderoth, University of Washington, Seattle WA  Bill (William) Cliff, Niagara University, Niagara NY  Harold Modell, Physiology Education Research Consortium (PERC), Seattle WA (http://physiologyeducation.org)http://physiologyeducation.org  Ann Wright, Canisius College, Buffalo NY  more than 200 physiology faculty who have responded to surveys &/or responded to our data-gathering questions at workshops.

31. References Adams, W.K. and Wieman, C.E. 2011. Development and validation of instruments to measure learning of expert-like thinking. International Journal of Science Education 33:1289-1312. American Association for the Advancement of Science (AAAS). 2011. Vision and Change in Undergraduate Biology Education: A Call to Action, Washington, DC: American Association for the Advancement of Science. Association of American Medical Colleges (AAMC). 2009. Scientific Foundations for Future Physicians. Washington, DC: AAMC. http://services.aamc.org/publications/http://services.aamc.org/publications/ Bailey,C. 2011. Department of Biochemistry, University of Nebraska, Lincoln, Concept Inventories, presentation at EB session "Promoting Concept Driven Teaching Strategies in BMB through Concept Assessments" Brownell, S.E., Freeman, S., Wenderoth, M.P., and Crowe, A.J. 2013. BioCore Guide: A tool for interpreting the core concepts of vision and change for biology majors. CBE–Life Sciences Education. 13:200-211. D’Avanzo, C. 2008. Biology concept inventories: overview, status, and next steps. Bioscience 58:1079-1085. Dirks, C., Wenderoth, M.P. and Withers, M. 2014. Assessment in the College Science Classroom. New York NY: WH Freeman

32. References – continued Fisher, K.M and Williams, K.S. Concept Inventories/Conceptual Assessments in Biology (CABs): An annotated list. 2012 http://www.sci.sdsu.edu/CRMSE/files/Concept_Inventories_in_Biology_20110325.pdf http://www.sci.sdsu.edu/CRMSE/files/Concept_Inventories_in_Biology_20110325.pdf Hestenes, D., M. Wells, and G. Swackhamer. 1992. Force concept inventory. Physics Teacher 30(3):141–158. Michael, J. (2007). Conceptual Assessment in the Biological Sciences: a National Science Foundation-sponsored workshop. Advances in Physiology Education, 31: 389-391 Michael, J. and McFarland, J. (2011. The core principles (“big ideas”) of physiology: results of faculty surveys. Advances in Physiology Education. 25:336-341. Michael, J., Modell, H., McFarland, J., and Cliff, W. (2009). The “core principles” of physiology: what should students understand? Advances in Physiology Education, 33: 10- Smith A.C., 2008, Department of Cell Biology and Molecular Genetics University of Maryland, ASM- CUE. HPI Concept Inventory Smith, J.I. and Tanner, K. 2010. The problem of revealing how students think: concept inventories and beyond. CBE–Life Sciences Education. 9:1-5. Wiggins, G. and McTighe, J. 1998. Understanding by Design. Alexandria, VA: Association for Supervision and Curriculum Development.

33. Disclosure: I am a PULSE leadership fellow & PULSE is dedicated to the implementation of V&C