1. © 2006 Teachers Development Group 1 Building K-12 Math Leaders: “Walking the Walk” MEANS “Talking the Talk” Tom Dick, Oregon State University Linda Foreman, Teachers Development Group NCSM Conference April 2008, Salt Lake City

2. © 2006 Teachers Development Group 2 RMC Research Corporation2 OMLI: Oregon Mathematics Leadership Institute Partners Oregon State University Portland State University Teachers Development Group (TDG) 10 Oregon School Districts RMC Research Corporation OMLI is in Year 4

3. © 2006 Teachers Development Group 3 RMC Research Corporation3 OMLI Objective Build a cadre of school- and district-based intellectual leaders and master mathematics teachers through: 3 summer institutes (2007 was the last institute) 2 of 6 content courses each summer 1 collegial leadership course each summer  Pedagogy and leadership Follow-up academic year professional development 4 site visits by TDG staff

4. © 2006 Teachers Development Group 4 RMC Research Corporation4 School Leadership Teams (SLT) 82 K-12 schools from 10 school districts Each school has a SLT composed of: 2 teacher leaders Attend 3 weeks of all 3 summer institutes Participates in school year activities 1 Administrator Attends 1 week of all 3 summer institutes Participates in school year activities

5. © 2006 Teachers Development Group 5 RMC Research Corporation5 OMLI Research Logic Model Summer Institute Math Content Leadership TDG Site Visits 4 Each School Year Action Plan School Leadership Teams Follow-up PD School-Based Implementation of Action Plan Increase Student Discourse Increased Student Discourse Classes of Teacher Leaders And Later in Classes of the Other Math Teachers Improved Teaching and Learning in Mathematics Improved Student Achievement

6. © 2006 Teachers Development Group 6 RMC Research Corporation6 Data Sources Classroom observation data Student achievement data Professional development participation data

7. © 2006 Teachers Development Group 7 RMC Research Corporation7 Random Sampling School Sampling 25 school out of 86 Stratified by grade level Sample demographically representative Teacher Sampling 1 of the 2 teacher leader 1 other teacher of mathematics at similar grade level

8. © 2006 Teachers Development Group 8 RMC Research Corporation8 Classroom Observation Protocols Must be about mathematics Among students not the teacher Lesson episodes Attributes of discourse Mode Type Tools

9. © 2006 Teachers Development Group 9 RMC Research Corporation9 Discourse Mode Who the Student Addresses Teacher (even if public) Student Group (small group or the class) Individual

10. © 2006 Teachers Development Group 10 RMC Research Corporation10 Discourse Type Represents continuum of cognitive demand Answering Stating/ sharing Explaining Questioning Challenging Relating Predicting/Conjecturing Justifying Generalizing Low Cognitive Demand High Cognitive Demand

11. © 2006 Teachers Development Group 11 RMC Research Corporation11 Discourse Tools Verbal Gesturing/Acting Written Graphs, Charts, Sketches Manipulative Symbolization Notation Computers/Calculators Others

12. A Typical Recording Sheet

13. © 2006 Teachers Development Group 13 RMC Research Corporation13 Classroom Parallel: Student Discourse Observation Scribing verbatim student discourse Characterizing discourse types: Procedures and Facts Justification Generalization

14. © 2006 Teachers Development Group 14 RMC Research Corporation14 Procedures and Facts Short answer to a direct question Restating facts/statements made by others Showing work/methods to others Explaining what and how Questioning to clarify Making observations/connections

15. © 2006 Teachers Development Group 15 RMC Research Corporation15 Justification Explaining why by providing mathematical reasoning Challenging the validity of an idea by providing mathematical reasoning Giving mathematical defense for an idea that was challenged

16. © 2006 Teachers Development Group 16 RMC Research Corporation16 Generalization Using mathematical relationships as the basis for: Making conjectures/predictions about what might happen in the general case or different contexts Explaining and justifying what will happen in the general case.

17. Student Discourse Observation Tool

18. © 2006 Teachers Development Group 18 RMC Research Corporation18 OMLI Work in Schools To accomplish the OMLI objectives … All OMLI School Leadership Teams collaborate to create sustainable and effective school-based professional learning communities whose work centers on improving mathematics instruction. All OMLI schools institutionalize ongoing, high- quality, practice-based professional learning characterized by protocol-based dialogue and inquiry about mathematics content, students’ mathematical thinking, and effective instruction.

19. © 2006 Teachers Development Group 19 RMC Research Corporation19 School Leadership Team Obligations Design and implement a School Action Plan to improve mathematics learning and teaching school-wide Facilitate a minimum of 2 hours of mathematics PD per month for building colleagues Utilize OMLI-defined Professional Learning Tasks (PLTs) during their building-based PD

20. © 2006 Teachers Development Group 20 RMC Research Corporation20 Professional Learning Tasks (PLT) Student Discourse Observations Case Discussions* Mathematics Tasks Framework & Task Analysis Guide* Data Snaps* Examining Student Work* Analyzing Trends Artifacts of Student Thinking Lesson Planning Framework* PLT Planning Framework*

21. © 2006 Teachers Development Group 21 RMC Research Corporation21 *Keeping Our Eye on the Prize: Student Thinking/Discourse Developing generative learners (teachers, administrators, and students) through a relentless focus on students’ mathematical thinking, cognitive demand, and sociomathematical norms Explicit focus in each PLT on sensemaking about students’ mathematical thinking through analyses of student discourse

22. © 2006 Teachers Development Group 22 RMC Research Corporation22 Training the Professional Eye: Student Discourse Observation Protocol Phase 1 Predictions – Framing the Observations Phase 2 Observations – Collecting and Classifying the Data Phase 3 Inferences – Inquiry Dialogue and Action Steps

23. © 2006 Teachers Development Group 23 RMC Research Corporation23 Why the Protocol? Besides supporting sensemaking about the discourse types, this process supports deprivatization of practice professional community sensemaking about mathematical ideas and their trajectory inquiry-based stance towards one’s practice

24. © 2006 Teachers Development Group 24 RMC Research Corporation24 Professional Learning Tasks (PLT) Student Discourse Observations Case Discussions* Mathematics Tasks Framework & Task Analysis Guide* Data Snaps* Examining Student Work* Analyzing Trends Artifacts of Student Thinking Lesson Planning Framework* PLT Planning Framework*

25. © 2006 Teachers Development Group 25 RMC Research Corporation25 One of the Research Questions Has the OMLI project increased the percentage of students who demonstrate proficiency on the Oregon State Mathematics Assessments for Grades 3, 5, 8, and 10 in all participating K – 12 schools?

26. © 2006 Teachers Development Group 26 RMC Research Corporation26 About the Research Data The school is the unit of change for the OMLI project Data available form the Oregon Department of Education website School-level aggregate data School demographic data

27. © 2006 Teachers Development Group 27 RMC Research Corporation27 Initial Results Were Discouraging and Inconclusive

28. © 2006 Teachers Development Group 28 RMC Research Corporation28 Taking Note of Implementation Fidelity Project staff reported that some schools are not implementing their action plans as well as others Re-examining project impact on student achievement Collect level of implementation data through the Teachers Development Group (TDG) site team staff Using a standard rubric developed in partnership with the TDG staff Explore relationship between student achievement and level of implementation.

29. OMLI 13 Trait Implementation Rubric

30. © 2006 Teachers Development Group 30 RMC Research Corporation30 Revising the Evaluation Question Do students in schools that do a better job of implementing the practices promoted through the OMLI project show higher performance on the state assessments compared to students in school that do a poorer job of implementing those practices?

31. © 2006 Teachers Development Group 31 RMC Research Corporation31 Traits Measured Quality of the Action Plan Implementation of the Action Plan Teacher Leadership Teacher 1 & 2 School Administrator Leadership & Engagement District Leadership Team (DLT) School/District Policies and Practices Stability of the School Leadership Team (SLT) School Priority of Mathematics Professional Development (PD) Responsibilities Scope of Professional Development (PD) Use of Professional Learning Tasks (PLTs) & Protocols Evidence of Impact

32. © 2006 Teachers Development Group 32 RMC Research Corporation32 Secondary School Results 5 implementation traits were positively correlated with student achievement in secondary schools Quality of the school action plan Implementation of the action plan Regular school-based professional development Scope of school-based professional development Use of well-defined professional learning tasks and protocols during school-based professional development Secondary Implementation Scale (SIS)

33. © 2006 Teachers Development Group 33 RMC Research Corporation33 Analysis of Grade 10 Student Achievement

34. © 2006 Teachers Development Group 34 RMC Research Corporation34 Means Weighted By Number of Students Assessed

39. © 2006 Teachers Development Group 39 RMC Research Corporation39 Conclusion for Secondary Schools The degree to which schools implement the practices promoted by the OMLI project is a significant positive predictor of student performance above and beyond what can be explained by the socioeconomic factor as indicated by the percent of students who qualify for free and reduced lunch program. This relationship is particularly acute at the Grade 10 (R2=.738, Beta=.320) and Grade 8 (R2=.524, Beta=.197).

40. © 2006 Teachers Development Group 40 RMC Research Corporation40 Secondary Implementation Scale Quality of the school action plan Implementation of the action plan Regular school-based professional development Scope of school-based professional development Use of well-defined professional learning tasks and protocols during school-based professional development (discourse)

41. © 2006 Teachers Development Group 41 RMC Research Corporation41 Elementary School Results 5 different traits were positively correlated with student achievement Leadership qualities of the teachers on the School Leadership Team Whether the School Leadership Team had a second teacher participating Supportive school and district policies and practices The degree to which mathematics is a priority for the school Use of well-defined professional learning tasks and protocols during school-based professional development (discourse) Elementary Implementation Scale (EIS)

44. © 2006 Teachers Development Group 44 RMC Research Corporation44 Conclusion for Elementary Schools The degree to which schools implement the practices promoted by the OMLI project and socioeconomic factors are predictors of student performance, but The regression model does not account for enough of the variance in student achievement There are other factors at play in elementary schools that are not accounted for by the traits measured by the implementation rubrics and socioeconomics

45. © 2006 Teachers Development Group 45 RMC Research Corporation45 Back to the Evaluation Question Question: Has the OMLI project increased the percentage of students who demonstrate proficiency on the Oregon State Mathematics Assessments for Grades 3, 5, 8, and 10 in all participating K–12 schools? Answer: Yes for those schools that effectively implemented the project as intended.

46. © 2006 Teachers Development Group 46 RMC Research Corporation46 Evaluation Plans for 2008 School level analysis Add traits to implementation rubrics that may explain other factors influencing student achievement at the elementary level Rate school again in June 2008 Repeat the analysis using 2008 student assessment data Obtain student level data from state ??? Analyze student achievement of student of SLT teachers compared to that of non-SLT teachers

47. Questions and Thank You! Tom Dick, OMLI Principal Investigator Mathematics Department, Oregon State University tpdick@math.oregonstate.edu Linda Foreman, OMLI Co-PI & Project Director Teachers Development Group linda.foreman@teachersdg.org www.teachersdg.org Dave Weaver, OMLI Evaluator RMC Research Dave.Weaver@rmccorp.com http://omli.org