1. Copyright © 2013 Stevens Institute of Technology NSTA April 5, 2014 A Mathematics and Science Partnership Partnership to Improve Student Achievement in Physical Science: Integrating STEM Approaches (PISA 2 )

2. o Arthur Camins, CIESE Director, Project Director, Co-PI o Kathy Kennedy, Professional Development Specialist o Tom Smith, Professional Development Specialist PISA 2 Leadership Team

3. CIESE develops and supports effective innovative curricula and professional development and conducts research in order to inspire, catalyze and strengthen scientific, technological, engineering and mathematics literacy.

4. Stevens Institute of TechnologyStevens Institute of Technology The Innovation University®, is a premier, private research university situated in Hoboken, N.J. overlooking the Manhattan skyline. Founded in 1870, technological innovation has been the hallmark and legacy of Stevens’ education and research programs for more than 140 years.

5. Develop new innovations in K-12 STEM education, linked to a strong educational research agenda. Critical Features Partnership Driven Teacher Quality, Quantity and Diversity Challenging Courses and Curriculum Evidence-based Design and Outcomes Institutional Change Mathematics and Science Targeted Partnerships

6. Teachers’ success in implementing new strategies is mediated by school and district-based constraints. o Ongoing Pressure to focus on language arts and math; o Current evaluation pressures constrain risk taking; o Teachers are Isolated. While teachers may be motivated to try new approaches, science and engineering may not be their district’s priority; o Limited by textbook-based curricula; o Administrative changes create instability. Context: Constraining Factors

7. How can we increase the likelihood of knowledge transfer from professional development exemplars to teacher practice across diverse curricula? Which professional development and one-one-one coaching practices are most effective to target individual teacher needs and support improved instruction? How should we calibrate evidence of change in teachers’ content knowledge, PCK and practices given their initial state and our external role? Today’s Discussion Questions

8. 1.Bayonne Board of Education 2.Brick Township Public Schools 3.Camden City Public Schools 4.Cherry Hill Public Schools 5.Hoboken Public Schools 6.Howell Township Public Schools 7.Jersey City Public Schools 8.Lakewood School District 9.Margate City School District 10.Morris School District 11.Mustard Seed School (private) 12.Princeton Regional Schools 13.Red Bank Borough Public Schools 14.Toms River Regional Schools 15.West New York School District PISA 2 School District Partners

9. IHE & Other Partners  Stevens Institute of Technology  Columbia University/Teachers College  National Science Resources Center  Education Development Center (Evaluator)  St. Peter’s University

10.  To enhance teachers’ content knowledge in science & engineering (S&E) and cultivate positive attitudes & beliefs towards teaching S&E  To increase students’ content knowledge and experiences in S&E  To promote students’ 21 st century skills (Critical Thinking and Creativity)  To institutionalize new graduate programs in STEM education and impact undergraduate teaching & learning  To increase the number of teachers with elementary endorsement in science  To build leadership and capacity in partner school districts PISA 2 Goals TEACHER LEARNING STUDENT ACHIEVEMENT 21 st CENTURY SKILL (Content/Pedagogy) (Science & Engineering) ACQUISITION

11. Graduate Courses/ PD Institutes Full-Day PD Workshops Classroom Support Visits Science and Engineering Practices PCK Core Ideas Increase students’ science content knowledge Increase # teachers with science endorsement Increase students’ 21 st Century skills PISA 2 Improvement Model

12. PISA 2 Participant Enrollment YearScholar Participants Leader Participants Total 141 2 3281644 4224769 520102 (69 new)89 6 (projected)120 (70 new)70

13. Courses are designed to builds PISA 2 teachers’ capacity and confidence to master and teach more complex concepts. Teacher Scholars Program Five Graduate Courses Fundamental Principles of Physical Science Fundamental Principles of Earth Science Energy Production & Consumption Understanding Global Climate Change Engineering Solutions to the Challenges of Energy & Global Climate Change

14. Teacher Leader Program

15. What Students Do: Ask questions and define problems Develop and use models Plan and carry out investigations Analyze and interpret data Use mathematics and computational thinking Construct explanations and design solutions Engage in argument from evidence Obtain, evaluate, and communicate information Professional Development Strategy: Focus on Two High Leverage Practices + EDP

16. What Students Do:What Teachers Do: Make sketches, concept maps, graphs, and/or charts or use words to illustrate concepts and represent their models of the natural world or a design solution. Provide a context or questions for students to represent and articulate their models. Create and use models to predict an outcome; explain their ideas; express their understandings of the natural world (Science). Provide opportunities for students to represent and articulate their models. Create and use a model to explain their design and why the design will be successful (EDP). Engage students in productive discourse to clarify their models; to explain and defend their models. Refine models based on evidence (Science). Guide students to use their evidence; facilitate discussion through questioning. Test and revise models based on test results (EDP). Provide productive feedback to students to clarify their models. PD Strategy: Focus on Two High Leverage Practices + EDP Making Models

17. What Students Do:What Teachers Do: Make an accurate and relevant claim. Provide opportunities for students to construct explanations and design solutions. Provide appropriate and sufficient evidence to support claim. Engage students to clarify and strengthen their explanations by connecting evidence to the model (Science). Provide reasoning that connects evidence to the claim. Engage students to review their solution, identify points of and reasons for failure during the test; guide how to improve the design (EDP). Include appropriate and sufficient scientific principles to explain why the evidence supports the claim. Provide productive feedback to students to clarify and strengthen their explanations. Recognize alternative explanations and provide appropriate and sufficient counter evidence and reasoning when making rebuttals. PD Strategy: Focus on High Leverage Practices + EDP Claims, Evidence and Argumentation

18. Coaching goals for each visit Planning Observation Co-Teaching/ model lesson Instructional Support Follow up or reflection meeting Components of the Coach Site Visits

19. CIESE PISA 2 staff model and call attention to Practices for Selected Science Topics…..

20. Teachers elicit students’ initial models about heat transfer (e.g. metals are inherently colder than plastic). Students investigate, collect evidence and make revised claims. … that teachers try out in their classrooms

21. Teacher Directed Models Activity: The teacher led an activity with play dough in which the students created models of planets to compare their sizes. Teacher: Take the play dough out of the bag and mush it into one big ball. Now we have to make our play dough into like a bread loaf. You want to roll it out so the play dough goes from 0-10 (teacher handed out sheets of paper with 10 lines marked 0-10). Projector: Step 2: Combine 6 parts together and put them in the Jupiter section – put 3 parts into the Saturn section.

22. Teacher Modified GEMS lesson on models of Earth-Sun- Moon systems with the help of coach Students analyzed data from table to determine what models were supported Modification – students explained why some models did not match the data. Teacher needed to devote more time to activity Modifying Lessons with Coach

23. E1- Does a project which uses scientific inquiry and the engineering design process (EDP) contribute to an increase in teachers' content knowledge of science and engineering? E2- Does a project which uses scientific inquiry and EDP contribute to an increase in students' content knowledge of science and engineering? E3- Do students improve their 21st Century Skills as a result of the program? E4- To what extent does participating in the program impact use of teaching practices and and the amount of time spent on science instruction? E5- To what extent did the program promote an increase in collaboration and shared vision among partners? Partners include University Faculty, District and Schools, Administrators, Teachers, Students, and Parents. Evaluation Questions

24. Teachers Science and Engineering Design Process (Pre- Post) Content Practices and Time Survey (Pre-Post) Practices Observations Interviews (including limiting factors) PISA 2 Outcome Measures Students Science and Engineering Design Process (Pre- Post) Content Critical Thinking and Creativity in Science and Engineering

25. Evidence from end-of-course assessments shows increased content knowledge for Cohort 2 teachers in each course. Pre-testPost-test Change (in percentage points) Physical Science, selected response (Course 1) 45% (SD=11) 69% (SD=12) 24*** Earth Science, selected response (Course 2) 63% (SD=14) 78% (SD=7) 14*** Understanding Global Climate Change (Course 3) 40.7% (SD=147.2) 62.1% (SD=17) 21.4*** Energy Production and Consumption (Course 4) 29.1% (SD=13.4) 52.2% (SD=16.7) 23*** ***p<.001

26. Student Score Increases on PISA2 Content Test % Correct (Pre-Test) % Correct (Post-Test) Change Elementary student overall performance (N=143) 27.6% (SD=8.4) 47.8% (SD=14.3) 20.1*** Middle school student overall performance (N=187) 44.1% (SD=14.1) 49.9% (SD=14.5) 5.9*** ***p<.001 Topics tested are: Properties and Changes in Matter, Forces and Motion (including mechanical energy), Weather and Climate, and EDP.

27. Traditional" Teacher Practices Q5A: Teacher explains to whole class Q5B: Sts read science/engineering text in class Q5D: Sts work individually on assignment Hands-on Science & Engineering Q5E: Sts work in pairs/groups on assignment Q5F: Sts do lab activity/investigation/experiment Q5i: Sts collect and analyze data Q5J: Sts use spreadsheets, databases, charts, etc. to summarize and display data Explanation-Driven Science Q5K: Sts maintain science/engineering notebook Q5L: Sts explain understanding to teacher verbally Q5M: Sts construct written explanation supported by evidence Q5N: Sts engage in debate about how well claims are supported by evidence Q5O: Sts reflect on strengths and weaknesses of their own thinking Q5P: Sts reflect on strengths and weaknesses of others Composite Teacher Observation Measures

28. Explanation-Driven Science School LevelTime 1Time 2Change Elementar y (n=10) 2.132.800.67 Middle (n=5) 2.832.37-0.47 Total 2.372.660.29 Changes in Targeted Teaching Practices (After 1 Year) Hands-on Science & Engineering School LevelTime 1Time 2Change Elementar y (n=10) 2.502.680.18 Middle (n=5) 2.60 0.00 Total 2.532.650.12 Traditional or Lecture & textbook School LevelTime 1Time 2Change Elementar y (n=10) 3.372.83-.53* Middle (n=5) 3.402.60-0.80 Total 3.382.76-0.62* * p<.05

29. The analysis (repeated measures ANCOVA) on Cohort 2, Year 1 data did not find an association between teachers’ content knowledge, time on topic nor changes in teacher practice and student test gains. The model includes: NJ ASK, grade as covariates; and teacher content knowledge, changes in practices, and time on topic as independent variables Results from preliminary analysis

30. Elementary School Model ( grades 4 & 5) F*p val.Partial Eta Squared Change in Scores 2.576.111.021 Change in Scores * Grade.118.732.001 Change in Scores * Teacher Content Knowledge.620.433.005 Change in Scores * Time on Topic.078.780.001 Change in Scores * NJ ASK Math 3.143.079.025 Change in Scores * Change in explanation driven science practices 4.743.031.038 Change in Scores * Change in hands-on science & engineering practices 1.666.199.014 * Wilks’ Lambda Repeated Measures ANCOVA Middle School Model (Grade 5,6 & 7) F p val. Partial Eta Squared Change in Score.@.@.. Change in Scores * Grade 6.@.@.. Change in Scores * Grade 8.@.@.. Change in Scores * Teacher Content Knowledge.957.330.010 Change in Scores * Time on Topic 1.395.241.015 Change in Scores * NJ ASK Math.004.947.000 Change in Scores * Change in explanation driven science practices 1.454.231.015 Change in Scores * Change in hands-on science & engineering practices 1.413.238.015 * Wilks’ Lambda; @ negative values or too near 0, reported as missing NOTE: The model treats grade differently because of the number of grades at each level

31. “The instructors are outstanding. This workshop has provided me more options and resources to use engineering design problems in my classroom. My kids love EDPs. I have encouraged all eligible educators in my building to participate in summer workshops.” Institutes and PD

32. [In] our full-day inclusive classroom […] struggles with an extraordinarily short attention span and has extreme difficulty with retention. Her memory is among the worst we've ever seen. It seems like the movie Groundhog Day with her, we teach and reteach skills to her, only to have her come in the next day like it is all new material. A traditional classroom would be a terrible fit for her. She came to us with very low self-esteem after struggling for years with her academics. Thanks to the ideology Jaime and I share, coupled with our STEM classroom and the training we received from PISA2, this student has made considerable progress this year. She shines during science experiments and engineering activities. While it is a challenge for her to retain the step-by-step methodology needed to solve long division problems, she can brainstorm ways to problem-solve within a real- world context. Her strengths are her creativity and her ingenuity and doing these projects affords her the opportunities to excel when might would otherwise flounder. Impact on Struggling Learners

33. Students created an “amusement park ride” and were asked to explain/defend where the three laws of motion were present in their ride. In the following dialogue, students presented and explained their ride to their classmates, defended where the targeted concept was present and answered questions posed by the teacher and also other students. Teachers, most for the first time, are incorporating modeling into instruction, but at a superficial level.

34. Dialogue: T: Explain the game to me. S: We have 3 bottles in a pyramid form and you have to knock them down with a ball. T: Ok, so what’s the first law acting on it? S: When you pick a ball and throw it the ball moves through the air. T: What force is making the ball move through the air? S: The person throwing it. T: If the ball is heavier would it move faster through the air? S: No, because it has more mass, so you would have to put more force on the ball to make it go faster. S: The third law is in out project because when you’re standing here and the bottles over there are still the same shape and size. T: That’s the second law, right? S: Yes. T: Can you tell me about the third law? S: We can put more weight in the cans; enough mass would counteract the force of the ball being thrown. T: What would happen if the cans were knocked down? S: Gravity would take its course and they would fall down to the table, or the ground if the table wasn’t there. T: Any questions for this group (teacher asked the rest of the class)? T: How would we change the model for this game to make it harder? S: We can change the form the ball by changing the mass of the ball, and use like a baseball. T: What if we used a Ping-Pong ball? S: It wouldn’t work because there is more mass in the milk containers than in the ball. T: What if I used a cannon to shoot the Ping-Pong ball out? Would it knock down the bottles? S: Depends on the distance. T: So you’re telling me distance matters? S: Yes.

35. Guiding Question: How can we use our knowledge of the properties of play dough materials and the Engineering Design Process to improve an existing play dough process? First, students investigate properties of flour, salt and water and how they interact. Then they are given a batch that is too grainy and sticky and asked to improve it by altering proportions or the steps in the process. Teachers, most for the first time, are incorporating the EDP into instruction with explicit reference to “steps.”

36. T: What’s the first stage? S: Ask. T: What are we asking? What’s the challenge? S: Play dough. T: What’s the next stage? S: Imagine. What does that mean? S: We imagine what it’s going to look like. T: Next step? S: Plan. T: What does that mean? S: Draw a picture and identify. T: That’s right, we’re going to plan what we’re going to do and need. Then the most fun part? S: Create. T: And then the last step? S: Improve. T: We’re going to create and improve. We’re going to work as chemical engineers.

37. T: Ok, like if you think it’s too wet what could you do? G: Add more salt. T: What else? S: Maybe squish it more and add more flour. T: And [step] number 4, how will you test it? B: we can play with it. T: I guess that’s why they call it play dough, because we’re going to play with it. Do you think the steps we’re going to follow are important? Whole Class: Yes. T: You want to be accurate with your steps and make sure you know what you’re adding. You need to keep track of everything you put in so you know what you can do to improve it. T: Talk softly with your partner and figure out what you will do first, second, third etc, but make sure you write it down. One of you can mix while the other keeps track of what you’re putting in. More Dialogue

38. Did they reference prior investigation of the ingredients (flour, water, salt)? She did not refer to prior activity to the students during the observation – but in the post-lesson interview with Jaime she said she had covered the materials previously. The only reference to prior knowledge she made to the students was about states of matter. Did they write and/or discuss and improvement plan? They discussed it (in the brainstorm) but there was no moment of reporting out their plan to the teacher or anyone. Did you observe the actual improvement testing? Yes. Some groups created whole new batches of playdough to test. Other groups just kept adding ingredients to their "failed" batch. There was no clarification about this from the teacher. Questions to Observer

39. PISA 2 Staff meet with teachers to plan and apply what they have learned to their own curriculum, model lessons, share teaching, coach and provide feedback. Teachers are translating aspects of what they learned in PISA 2 in their classrooms. Applying improved science knowledge; Using the activities and resources from the courses, institutes and PD workshops; Implementing pedagogical approaches to enrich textbooks’ content (i.e. use of modeling and explanation-driven science) Response to Constraints

40. Teacher Learning Progression Teachers have limited understanding of practices and EDP Teachers gain comfort with and understanding of practices and EDP during institutes and coaching Teachers struggle to implement practices and EDP effectively in the classroom In Class Support: Planning Observation/ Co- teaching Reflection In Class Support: Planning Observation/ Co- teaching Reflection

41. Model: Consistently use the targeted practices across all courses, PD and coaching; – explanation driven science; – representing student thinking (models); – Representing data and its use of data (lab notebooks, data representations) – Connections between EDP and science content Be Explicit: about practices, EDP process and the phases and pedagogical purpose of each phase; Slow Down: Give teachers time to reflect and discuss new practices; Clarify: the distinction between EDP and Inquiry Science; Help Plan: Discuss how to connect each EDP activity to science content. Lessons Learned About Supporting Practice

42. How can we increase the likelihood of knowledge transfer from professional development exemplars to teacher practice across diverse curricula? Which one-one-one coaching practices are most effective to target individual teacher needs and support improved instruction? How should we calibrate evidence of change in teachers’ content knowledge, PCK and practices given their initial state and our external role? Today’s Discussion Questions

43. Greg Benedis-Grab – gbenedis@stevens.edugbenedis@stevens.edu Arthur H. Camins – acamins@stevens.eduacamins@stevens.edu PISA2- http://ciese.org/pisa2/http://ciese.org/pisa2/ CIESE- http://ciese.org/http://ciese.org/ CIESE in the News- http://ciese.org/pub_news.htmlhttp://ciese.org/pub_news.html Contact Us