1. 1 Milwaukee Mathematics Partnership Program Evaluation Year 5 Results Carl Hanssen Hanssen Consulting, LLC Cindy Walker University of Wisconsin-Milwaukee October 2008 MTL Meeting The Milwaukee Mathematics Partnership (MMP) is supported with funding from the National Science Foundation.

2. 2 Evaluation Goals  Help the MMP better serve its constituents and improve its effectiveness  Serve the broader mathematics education community through documentation and dissemination of MMP activities

3. 3 MMP Evaluation Logic Model Student Achievement Teacher Content & Pedagogical Knowledge Math Faculty Involvement Learning Team Effort School Buy-in Teacher Involvement New Courses District Buy-in MPA Ownership MATC Buy-In UWM Buy-In Classroom Practice MMP Activities Proximal Outcomes Distal Outcomes

4. 4 Agenda 1. Student Achievement 2. Learning Teams 3. Distributed Leadership 4. Conclusions 5. Next Steps

5. 5 1. MMP Impact on Student Achievement Student Achievement Learning Team Effort School Buy-in Teacher Involvement Classroom Practice

6. 6 MMP Impact on 2006 Student Achievement Are student achievement gains greater in schools that have more fully embraced MMP principles?

7. 7 Sep 04 Sep 05 Sep 06 Sep 07 Sep 08 MMP Online Survey Spring 2005 WKCE Fall 2005 WKCE Fall 2006 WKCE Fall 2007 MMP Online Survey Spring 2006 MMP Online Survey Spring 2007 2004-2005 School Year 2005-2006 School Year 2006-2007 School Year 2007-2008 School Year Data Collection Timeline MMP Online Survey Spring 2008

8. 8 Two cohorts of students  Cohort A Approximately 3,000 students Same school in grades 3-5 from 2005-2007  Cohort B Approximately 2,800 students Same school in grades 6-8 from 2005-2007

9. 9 HLM Analysis Consistent curriculum + Teachers working together + PD perceived as valuable Predicts Math Focus Math Focus Predicts % of students proficient Math Focus Used as school-level predictor for HLM analysis

10. 10 HLM Results  Math focus was a predictor of initial math achievement scores AND of student learning rates  If your school scores 1 point higher on math focus, you can expect Cohort A students to start 15 points higher and grow at a rate 7 points higher Cohort B students to start 18 points higher and grow at a rate 7 points higher

11. 11 HLM Results Time Score 15-18 pt gap to start Expected growth may be 30 pts Growth would be 30 + 14 points School X: math focus score 3.5 School Y: math focus score 2.5 Yr 1Yr 2Yr 3

12. 12 2. Learning Teams Student Achievement Learning Team Effort School Buy-in Teacher Involvement Classroom Practice Distributed Leadership

13. 13 Twelve Case Study Schools  Diverse set of schools  Grade levels 9 schools with K-5 students 6 schools with 6-8 students 2 high schools 10-12  Diverse demographics Minority students (30-90%) Special education (11-32%) Free/Reduced lunch (41-95%)

14. 14 Case Study Data Collection  22 learning team or math department meeting observations  42 classroom observations  MKT Assessment for math teachers  SNA Survey for math teachers and administrators

15. 15 Key Trends from Learning Teams  Overall, meetings tended to focus more on ‘administration’ than ‘learning’ though higher performing teams retain more of an emphasis on learning  Contrast between LT meetings and math department meetings Math meetings more focused Math departments less team oriented  Transition to released MTL model Seamless transitions Identification of replacement teacher No release

16. 16 Characteristics of High &Low Rated Learning Teams—Team Functioning  Less focus on administration  Positional authority is less important  Multiple views are represented and heard  Multiple segments of the school are represented  Written agenda, note taker, facilitator  Explicit action items  Participants have high knowledge and skill levels  Focus on administration  Principal does all the talking  A few individuals dominate the discussion  No agenda or team is easily distracted from the agenda  Little follow-through on assignments  No clear actions High Low

17. 17 Characteristics of High & Low Rated Learning Teams—MMP Issues  Consistent curriculum  Math is addressed alongside and in combination with other subjects  Coherent within grades and across grades  Math is discussed irrespective of presence of MTL  Reference to MMP work courses including formative assessment, descriptive feedback, benchmarks  Teachers operate autonomously  Math not addressed at the meeting  No clear math leader— MTL may be unsure of role  Confusion about the MMP and CMF High Low

18. 18 3. Distributed Leadership  Teachers and administrators in each school were asked to name individuals with whom they communicated about mathematics  This is a key indicator of distributed leadership

19. 19 Mathematics Distributed Leadership Continuum HighLow Tight Network MTL Central Many Links to MTL MTS Inside Many Links to MTS Leadership is shared among many Loose Network MTL Not Central Few Links to MTL MTS Outside Few Links to MTS Leadership responsibility of few

20. 20 Low Student Achievement: 2007: 66% Proficient 3-year trend: -9%

21. 21 Medium Student Achievement: 2007: 47% Proficient 3-year trend: +15%

22. 22 High Student Achievement: 2007: 53% Proficient 3-year trend: +3%

23. 23 Evolution of Distributed Leadership 1. MTL is active within the school 2. Teachers begin extensive collaboration 3. MTL & Teacher collaboration extends outside school (MTS may become heavily involved in the school) 4. MTL is used primarily as a resource 5. Teachers assume math leadership

24. 24 4. Overall Conclusions  There is support for the argument that schools that have more fully adopted MMP principles are demonstrating stronger results.  There is tremendous variability in MMP adoption and progress across the district—though MMP impact appears more pervasive.

25. 25 Overall Conclusions  Important considerations for sustaining MMP work High levels of math focus have been shown to be related to higher student achievement. What are the indicators of math focus? Creating Distributed Leadership in a school takes time—and communication is critical but helps promote math focus

26. 26 Overall Conclusions  Important considerations for sustaining MMP work MTL role may be shifting from focal point to facilitator—we see a shift in the perception of who is responsible for helping the school focus on improving mathematics teaching and learning MTL release model presents benefits and challenges for sustaining MMP work

27. 27 5. Evaluation 2008-09  MMP Online survey in May 2009  Continue HLM analysis for student achievement  25-30 case study schools to participate over 3 years—Sign up now! LT/Math meeting Observations SNA  Focus on different MTL release models  Goal to implement SNA in most schools across the district

28. 28 Focus Question  What message will you be taking back about… Your ongoing work with teachers to improve math in your school? How your learning team can be most effective?