1. Promise and Problems of Learning Progression-guided Interventions Hui Jin, Hyo Jeong Shin, Michele Johnson, Jinho Kim

2. Overview  Carbon Cycle Teaching Experiment: Using knowledge of photosynthesis & cellular respiration to explain plant growth  Tracing matter  Tracing energy  Connecting scales  Motivations  Validation LPs vs. Evolutionary LPs (Duschl, Maeng, & Sezen, 2011). Evolutionary LPs can be aimed at either knowledge enrichment or conceptual change depending on “different conditions of prior knowledge” (Chi, 2009).  Conceptual change-oriented learning progression  Very few learning progression studies explore teacher’s role in interventions (e.g. Furtak, 2012; Thompson, Braaten, & Windschitl, 2012). Teachers are agents who enact curriculum in class.  Explore teachers’ role in LP-guided interventions.

3. Research Questions  Student Outcomes: What are students’ learning outcomes in an intervention guided by a conceptual change- oriented Learning Progression Framework (LPF)? What learning difficulties do students have?  Teacher Knowledge : How do we develop LPF-based measures of teachers’ CK (content knowledge) and PCK (Pedagogical content knowledge)? What is teachers’ achievement in the CK and PCK assessments?  Teachers’ Impact on Student Outcomes : Is teachers’ CK and PCK linked to students’ outcomes? How? What are some other factors affecting student outcomes?

4. Development Process  The LPF served as a guide in the design of a coordinated set of resources: 1. Student Assessments 2. Teacher Assessment 3. Teaching unit 4.Professional Development programs and resources

5. Level 1. Force-dynamic Reasoning Level 2. Hidden Mechanisms Reasoning Level 4. Tracing Matter; Tracing Energy; Connecting Scales Level 3. Reasoning about Matter and Energy Unsuccessfully Phenomena Explanation Practice: Learning Progression Framework (LPF) 1. Student Assessment 7 Tracing Matter Items 3 Tracing Energy Items 4 Connecting Scales Items 2. Teacher Assessment CK Items: 6 PCK Items: Analyze responses at Levels 1, 2, and 3; 3 Knowledge of student thinking items; 3 Next instructional move items

6. To engage students in scientific practices Level 1. Force-dynamic Reasoning Level 2. Hidden Mechanisms Reasoning Level 4. Tracing Matter; Tracing Energy; Connecting Scales Level 3. Reasoning about Matter and Energy Unsuccessfully Explanation Practice Learning Progression Framework (LPF) To promote conceptual change Tracing Matter; Tracing Energy Connecting Scales 3. TEACHING UNIT 4. PROFESSIONAL DEVELOPMENT http://www.pathwaysproje ct.kbs.msu.edu/?page_id= 59

7. Data Sources & Analysis Research Components Student Outcomes Teacher Knowledge Teachers’ Impact on Student Outcomes DataPre- and post- assessments from students (2011-12: 605 students; 2012- 13: 380 students) CK & PCK assessments from teachers (2011-12: 120 teachers; 2012-13: 74 teachers) Pre- and post-student assessments (2011-12: 598 students; 2012-13: 380 students) CK & PCK assessments from teachers who taught these students (2011-12: 15 teachers; 2012-13: 10 teachers) Feedback forms from 11 teachers who were identified as low- performing/high-performing teachers AnalysesUse LPF to code data Apply IRT Analyses to the coding results Use a LPF-based PCK Rubric to code data Apply IRT Analyses to the coding results Qualitative & Quantitative Analyses

8. LPF-based PCK Rubrics LevelsTeacher ResponsesAlignment with the student LPF 4. Targeting the transition from naïve ideas to scientific big ideas A student responds, “Along with soil, plants use carbon dioxide, sunlight, and water to help them make food.” Which of the following question would you ask next? b. Where does carbon dioxide go? b. By asking where the CO2 goes during the process, I am looking to see if students understand that carbon is the backbone of organic molecules. I am a little uncomfortable asking where plants get food as that might lead students to think about fertilizer as food.  Level 4 understanding of the LPF & understanding of students’ ideas at Levels 1, 2, & 3 of the LPF. 3. Targeting scientific big ideas a. How is your explanation related to photosynthesis? My looking at photosynthesis one will need to answer the other questions. The formula for photosynthesis is a central theme to the question.  Level 4 understanding at the LPF 2. Content- specific approach In a lesson on food, students debate whether or not water is food for plants. Which one of the following would be the best next instructional step? b. Cut open a cactus to show students that water is stored inside. This would show the storage of water, thus using it for food.  Level 3 understanding of the LPF. 1. Content general approach D. Have students observe two plants, only one of which is watered, over a period of a week. It is an experiment and has results

9. Findings  Student Outcomes  Teacher Knowledge  Teachers’ Impact on student outcomes

10. Student Learning Gains IRT Analyses PrePostGain 2011-12-0.380.400.72** 2012-13-0.420.751.17** Distribution of Students’ Responses ** p<0.001

11. Teacher Knowledge: General Pattern

12. Teacher Knowledge: The Most Difficult Items CKPCK

13. Teacher Knowledge: The most difficult PCK items  The easiest item ( Identifying incorrect description of content ): During a discussion about how plants and animals get energy, one student says, “I know animals break down food to get energy, but I don’t think plants break down food for energy because they get light energy from the Sun.” What, if anything, is wrong with this student’s statement?  The most difficult item ( Identifying naïve ideas of students ): A teacher asks students where plants get their food. A student responds, “Along with soil, plants use carbon dioxide, sunlight, and water to help them make food.” In order to find out more fully how the student’s ideas of how matter is transformed when plants grow, which of the following question would you ask next?

14. Teacher Knowledge and Student Learning Gains  The association of teacher knowledge and student learning gains is statistically significant  Teachers with average knowledge level (combined CK & PCK scores) produced a significant learning gain in their students:  0.59 logits (p<0.001) for 2011-12  1.13 logits (p<0.001) for 2012-13  Teachers who had one logit higher knowledge produced an additional increase in student learning gain:  0.36 logits (p<0.001) for 2011-12  0.34 logits (p<0.001) for 2012-13.

15. Identify High/Low performing Teachers Zero learning gain Average learning gain 2011-12 Data

16. Identify High/Low performing Teachers Zero learning gain Average learning gain 2012-13 Data

17. Compare High-performing teachers with low-performing teachers  Feedback forms from 7 High performing teachers & 4 Low performing teachers  Low-performing teachers taught 6, 7, or 8 lessons  High-performing teachers taught 10 or 11 lessons  Teacher H1 (highest learning gain) taught only 5 activities, but produced the highest learning gain in 2012-13. Feedback from suggests that the teacher used similar activities to replace activities in the curriculum  The coverage of curriculum is positively associated with student learning gains.

18. Implications  The LPF was the basis for developing an intervention that enabled students to learn significant knowledge and practices in an important domain.  We developed LPF-based measures of teachers’ CK and PCK, and those independently contributed to students’ learning.  Teachers’ classroom practices also made a difference, at least in terms of the coverage of curriculum. We are currently analyzing teachers’ classroom teaching videos to examine how teaching practice affect learning outcomes.

19. Questions?  Find manuscript, presentations, curriculum and PD resources at:  www.pathwaysproject.kbs.msu.edu www.pathwaysproject.kbs.msu.edu  Further questions, contact:  Hui Jin: jin.249@osu.edu; Michele Johnson: mkiss@ucsb.edujin.249@osu.edumkiss@ucsb.edu  This grant was funded by the National Science Foundation (NSF) under grant number DUE-0832173. The views expressed here are those of the authors and do not necessarily reflect those of NSF.