Presentation is loading. Please wait.

Presentation is loading. Please wait.

OUSD New Small Schools External Evaluation - 0 - New Small Schools Evaluation Board Presentation September 26, 2007.

Similar presentations


Presentation on theme: "OUSD New Small Schools External Evaluation - 0 - New Small Schools Evaluation Board Presentation September 26, 2007."— Presentation transcript:

1 OUSD New Small Schools External Evaluation - 0 - New Small Schools Evaluation Board Presentation September 26, 2007

2 OUSD New Small Schools External Evaluation - 1 - 2007-2008 School Portfolio Management Timeline DatesActivityDescription 9/26/07Small Schools External Evaluation (Phase I) The Small Schools Movement has been a critical component of OUSDs School Portfolio Management work over the past six years as we work towards our vision of providing families with two quality school options in each neighborhood 10/10/07 Launch of Tiered Accountability & Support model Presentation of 2007-2008 School Portfolio Management process and Area of Focus Schools OUSD has developed a robust and differentiated accountability & support model to ensure that all schools are receiving the supports required to ensure acceleration for all students Staff have identified a list of Area of Focus schools with whom we will be engaging in a community decision making process about the appropriate interventions. This analysis takes into account enrollment trends, as well as academic performance 12/12/07 Intervention recommendations for Area of Focus schools Following a thorough community engagement process, staff will make recommendations regarding the appropriate interventions 12/19/07 Intervention decisions for Area of Focus schools State Administrator will make decisions regarding the appropriate interventions Fall 2007Small Schools External Evaluation (Phase II) School Portfolio Management will provide support to Research & Assessment for the completion of Phase II of the Small Schools External Evaluation Fall 2007 Collaboration with Fiscal regarding Multi-year Fiscal Recovery Plan School Portfolio Management will provide support to the Fiscal Team regarding the Districts fiscal recovery plan and the long-term impact on the optimal number of schools for OUSD

3 OUSD New Small Schools External Evaluation - 2 - I. Small Schools Movement in Oakland Historical Context Incubation Process II. External Evaluation Methodology Results III. New Small Schools Investment IV. Reflections and Lessons Learned Small Schools Evaluation Agenda A New Small School is a school that is created around a common vision; has an explicit theory of action around closing the achievement gap; and develops a clear plan for a culture of high, non-negotiable expectations for student achievement. Small size is a facilitating condition that enables personalized learning structures for students; personalized involvement and leadership opportunities for families; and personalized learning structures for staff. PHASE 1 OF 2

4 OUSD New Small Schools External Evaluation - 3 - Part 1: Small Schools Overview Graphic: Courtesy of Oakland Community Organizations (OCO) HISTORICAL CONTEXT Oaklands Small Schools Movement was community-based and led by parents organized primarily by OCO. Parents wanted an alternative to some of the most overcrowded and lowest performing public schools in California. Parents and teachers were inspired by visits to successful small schools in Chicago & New York. OUSD School Board unanimously passed the New Small Autonomous Schools (NSAS) policy in May 2000, and 9 schools were opened as a result. In 2003 OUSD moved to full implementation and has opened 48 new small schools to date. New small school creation is one of six foundational strategies for Expect Success.

5 OUSD New Small Schools External Evaluation - 4 - Fall 2000 ACORN Woodland Fall 2001 Urban Promise Life Academy Melrose Leadership Academy Fall 2002 KIPP Fall 2003 Fall 2005 Fall 2004 Think College Now YES Mandela Media College Prep Robeson School of Visual & Performing Arts College Prep & Architecture Explore CBITS EnCompass Academy * East Oakland Community High School East Oakland School of the Arts Reach Academy Sankofa Academy RISE Community School * Kizmet Academy Manzanita SEED EXCEL HS BEST HS 48 New Small Schools in Oakland: 2000 - 2007 ASCEND MetWest Fall 2006 United for Success Academy Peralta Creek MS Alliance Academy Bridges Academy Coliseum College Prep Esperanza Elementary New Highland Fred T. Korematsu Discovery Academy Manzanita Community School Elmhurst Community Prep PLACE @ Prescott * Closed * Opening 2007, Incubating 2007-08 * New York model, no OUSD incubation Fall 2007 West Oakland MS Greenleaf Elementary East Oakland PRIDE Futures Elementary Learning Without Limits * Community United Elementary Global Family School Graphic: Courtesy of OUSD New School Development Group * International HS * School of Social Justice & Community Development Leadership Prep Roots Int l International Community School Part 1: Small Schools Overview

6 OUSD New Small Schools External Evaluation - 5 - New School Incubator Curriculum – 1 st Semester Primary focus = to help design teams build background knowledge, conduct research and plan toward developing a thoughtful School Design Proposal that has a very explicit vision and a clearly articulated theory of action that addresses the needs of and ensures equitable outcomes for all children. StrandStrand Name Strand 1The New School Incubator: The Year, The Work, The Movement Strand 2Know Thyself: Understanding who you are as a leader in the context of this work Strand 3Getting Started: Forming and working with a design team SCHOOL VISITS Strand 4Visioning: Who is the child walking out of your doors? Strand 5Know Thy Community: Who is the child walking into your doors? Strand 6Theory of Action: Bridging the gap between the child walking in the door and the child walking out of the door SCHOOL VISITS Strand 7The School Program: Laying the foundation for the development of your new school Graphic: Courtesy of OUSD New School Development Group Part 1: Small Schools Overview Incubation Process: 2004-2007

7 OUSD New Small Schools External Evaluation - 6 - OUSD INCUBATION PROCESS: 2004-2007 Recruiting Students Fiscal Realities Recruiting & Selecting Teachers School Culture Supporting & Training Teachers Supporting Students Community and Family Involvement Governance District Operations New School Incubator Curriculum – 2 nd Semester Jan - Feb Mar - Apr May - Jun Graphic: Courtesy of OUSD New School Development Group Instructional Program Part 1: Small Schools Overview SCHOOL DESIGN PROPOSAL = THE FOUNDATION Community AnalysisTheory of ActionVision Instructional Program

8 OUSD New Small Schools External Evaluation - 7 - Part 2: External Evaluation of Small Schools OUSD contracted with Strategic Measurement and Evaluation, Inc. to answer the following questions: 1. What is the demographic and academic profile of students attending New Small Schools? 2.Despite the lower average CST scores of students entering New Small Schools, is there evidence that the New Small Schools are accelerating student achievement? 3.How do the outcomes achieved by the New Small Schools compare to the outcomes produced by the schools they replaced? 4.Have the New Small Schools been successful at implementing the approved design models? Do the New Small Schools function effectively as organizations? 5.How does the number of years a New Small School has been open relate to measures of model implementation and effective school functioning? How do these measures relate to measures of stakeholder satisfaction? RESEARCH QUESTIONS

9 OUSD New Small Schools External Evaluation - 8 - Methodology Part 2: External Evaluation of Small Schools

10 OUSD New Small Schools External Evaluation - 9 - QUESTION #1: What is the demographic and academic profile of students attending New Small Schools? RELEVANCE: To identify the intended population at the New Small Schools METHODOLOGY: Analysis of the OUSD student data files focusing on gender, ethnicity, free/reduced lunch status, English language development status, disability status, and previous CST scores Part 2: External Evaluation of Small Schools

11 OUSD New Small Schools External Evaluation - 10 - Compared to the District overall, New Small Schools currently serve: a greater percentage of Latino students (53% @ New Small Schools vs. 34% @ district) a greater percentage of English Learner students (38% vs. 29%) a greater percentage of students receiving free/reduced lunch than the district as a whole (78% vs. 69%). Part 2: External Evaluation of Small Schools SMALL SCHOOLS DEMOGRAPHICS in 2006-07

12 OUSD New Small Schools External Evaluation - 11 - Compared to the District overall performance in 2005-06, New Small Schools serve: a lower percentage of students designated as Proficient or Advanced based on state test scores (e.g. CST), and a higher percentage of students designated as Below Basic or Far Below Basic. Part 2: External Evaluation of Small Schools SMALL SCHOOLS STUDENT ACADEMIC PERFORMANCE *CHASEE results are based on 2005-06 10 th grade performance.

13 OUSD New Small Schools External Evaluation - 12 - For the 2006-2007 school year, the New Small Schools had fewer students who performed at Proficient or Advanced on the CST and CAHSEE exams. Compared to Comparison schools, the New Small Schools had a lower average number of days absent per student and lower suspension rate. The differences are not statistically significant. Part 2: External Evaluation of Small Schools SMALL SCHOOLS ACADEMIC PERFORMANCE

14 OUSD New Small Schools External Evaluation - 13 - FINDINGS: The New Small Schools had significantly higher Use Your Voice ratings from students, parents, and teachers than did the comparisons schools. The New Schools compared favorably to the District ratings. Part 2: External Evaluation of Small Schools STAKEHOLDER SURVEY RESPONSES N=6,796 N=10,735 N=11,929 N=1,256

15 OUSD New Small Schools External Evaluation - 14 - Part 2: External Evaluation of Small Schools STAKEHOLDER SURVEY RESPONSES FINDINGS: The New Small Schools had significantly higher Use Your Voice ratings from students and teachers than did the comparisons schools in the categories of Clean Learning Environment, Safety, Caring and Support, and College Readiness.

16 OUSD New Small Schools External Evaluation - 15 - QUESTION #2: Is there evidence that the New Small Schools are accelerating student achievement? RELEVANCE: On average, students who enter the Small Schools during 2006-07 generally demonstrate academic performance that are below the District average. How can we show that small schools truly accelerate student achievement in a way that factors in student initial academic achievement? METHODOLOGY: To show students performance relative to their starting point, a statistical model was created to link student demographic characteristics and previous years CST test performance to current year CST scores. Use the model to calculate projected CST ELA and math scores for each student in each school year and grade level. Compare average projected CST scores of students within each school to the average CST scores actually obtained by the same students. If the average CST score obtained by a group of students is significantly higher than projected, there is evidence that the school may have accelerated student achievement. Part 2: External Evaluation of Small Schools

17 OUSD New Small Schools External Evaluation - 16 - SMALL SCHOOLS STUDENT PERFORMANCE Student groups at the New Small Schools demonstrated accelerated CST ELA achievement 22% of the time and underperformed only 7% of the time. Student groups at the Comparison Schools demonstrated accelerated CST ELA achievement only 16% of the time and significantly underperformed projected performance 25% of the time. A similar pattern was obtained on the CST math tests. Both groups of schools tended to perform as projected on the CAHSEE. Part 2: External Evaluation of Small Schools

18 OUSD New Small Schools External Evaluation - 17 - QUESTIONS #3: How do the outcomes achieved by the New Small Schools compare to the outcomes produced by the schools they replaced? RELEVANCE: By comparing outcomes for similar groups of students before and after they enrolled in the New Small Schools it is possible to track changes that can be attributed to the New Small Schools. METHODOLOGY: Sufficient data on attendance (days absent) and CST/CAHSEE test scores (percent of students proficient or higher) existed to make comparisons based on these variables. The family tree charts were used to identify the relevant transitions from a traditional school to New Small School(s). Part 2: External Evaluation of Small Schools

19 OUSD New Small Schools External Evaluation - 18 - FINDINGS: The percent of students labeled proficient or higher on the CST was compared as students moved from traditional schools to the New Small Schools. The chart below provides CST ELA data related to four transitions. Over time, these transitions generally show an increase in the percent of students performing Proficient or higher on both ELA and Math CST when compared to the larger school that they came from. Part 2: External Evaluation of Small Schools SMALL SCHOOLS PERFORMANCE *The n is based on the number of students with valid CST scores.

20 OUSD New Small Schools External Evaluation - 19 - Part 2: External Evaluation of Small Schools FINDINGS: The chart below provides the percent of students labeled Proficient or higher on the CST ELA at the 10 comparison schools. The pattern of performance change over the course of four years between the Comparison schools varied by as much as 23 percentage points in the percent of students performing Proficient or higher. The range was between +10% points to -13% points. Overall, the 10 Comparison schools made a 3.1 point gain in the percent of Proficient and Advanced students in ELA over the course of 4 years. COMPARISON SCHOOL PERFORMANCE

21 OUSD New Small Schools External Evaluation Graduation Rates: New Small Schools vs. Comparison Schools The percent of 2005-06 graduates (based on NCES calculations) in small high schools are higher than Comparison Schools, district, county, and state. The percent of small high school graduates who meet A-G requirements show more variability than Comparison Schools. Part 2: External Evaluation of Small Schools - 20 -

22 OUSD New Small Schools External Evaluation - 21 - New Small Schools vs. Comparison * Schools * Comparison School = School that students would have attended if the new school had not opened. Part 2: External Evaluation of Small Schools Academic Performance Index (API)

23 OUSD New Small Schools External Evaluation - 22 - Model Implementation & School Functioning QUESTION #4: Have the New Small Schools been successful at implementing the district-approved design models? Do the New Small Schools function effectively as organizations? RELEVANCE: To understand the degree to which the district-approved design model has been implemented at each site. Evidence of effective school functioning provides additional context from which to judge the success of design model implementation. METHODOLOGY: Content analysis of each original design proposal identified specific organizational and instructional elements associated with each New Small School. A customized survey was designed for teachers to assess the degree to which each element was present at a particular school. A second survey was developed to assess the degree to which the New Small Schools exhibited characteristics associated with effective organizations (e.g., clear communication, strong leadership, shared purpose, etc.). Part 2: External Evaluation of Small Schools

24 OUSD New Small Schools External Evaluation - 23 - Model Implementation: Teachers at the New Small Schools tended to agree that the schools attempted to implement the organizational and instructional elements outlined in the approved design model. Their average model implementation rating was 3.85 on a scale of 1 (no implementation) to 5 (full implementation). School Functioning: Teachers at New Small Schools tended to agree that their schools demonstrated characteristics associated with effective organizational functioning. Their average school functioning rating was 3.71, significantly higher than the functioning rating obtained from teachers at a sample of comparison schools (3.71 vs. 3.44). Part 2: External Evaluation of Small Schools School Functioning n=322 n=125 n=322 Model Implementation

25 OUSD New Small Schools External Evaluation - 24 - School Performance Over Time QUESTION #5: How do the number of years a New Small School has been open relate to measures of model implementation and effective school functioning? RELEVANCE: The time needed to demonstrate sufficient implementation and functioning informs strategies for incubation and district support. METHODOLOGY: The model implementation ratings and effective school functioning ratings were plotted by the number of years a New Small School had been open. Part 2: External Evaluation of Small Schools

26 OUSD New Small Schools External Evaluation - 25 - Model Implementation & School Functioning FINDINGS: The average model implementation ratings were not significantly different based on the number of years a New Small School had been open. The 12 New Small Schools opened in 2006-2007 compared favorably to older New Small Schools. Part 2: External Evaluation of Small Schools Model Implementation School Functioning The average school functioning ratings were not significantly different based on the number of years a New Small School had been open. There was significant variation at the individual school level. The district may benefit from reviewing organizational and personnel factors associated with higher functioning schools.

27 OUSD New Small Schools External Evaluation - 26 - Part 3: External Evaluator Summary of Key Findings Most of the 39 new small schools open in 2006-07 appear headed in a positive direction: 1.The New Small Schools serve students, families, and communities in large, overcrowded, and low performing schools. 2.New Small Schools accelerate both student CST ELA and Math achievement more frequently than do the comparison schools (22% vs. 16% for ELA and 33% vs. 21% for Math). 3.Most New Small Schools are achieving CST and CAHSEE scores that meet or exceed projected average levels. Compared to the traditional schools that students moved from, the graduation rates at the New Small High Schools were higher, ranging from 69% to 97% compared to 61% to 80% at existing comparison high schools. 4.There are examples of increases in the percentage of students being designated proficient or higher on the CSTs as students move from existing traditional schools to the New Small schools. 5.The average student, parent, and teacher satisfaction ratings achieved by New Small Schools were higher than the student, parent, and teacher satisfaction ratings achieved by a sample of comparison schools and compared favorably to overall district levels. 6.The survey data indicate New Small Schools are generally implementing proposed designs that address school culture, instructional program, professional learning community, parent, and community engagement. 7.New Small Schools can achieve high levels of organizational functioning during the first year of operation.

28 OUSD New Small Schools External Evaluation One-Time Investment in New Small Schools OUSDs one-time financial investment in the creation of new small schools has relied on Expect Success funding. From the 2005-06 to the 2007-08 school year OUSD has invested $5.64M or, on average $201K per new small school. These dollars were invested in a number of activities including: Incubator services, Support for principals in training, School improvement coaching services, and Subsidies for the opening and closing of schools. * - Assumes that 28 new small schools have been opened between September 2005 and September 2007. Part 4: Small Schools Investment - 27 -

29 OUSD New Small Schools External Evaluation In addition to the financial investment made by OUSD, there are also other intangible costs to new small school development. These costs primarily have an impact on the capacity of district staff to serve schools. These include: New principal recruitment, New teacher recruitment, High impact on Service Org to support new staff (e.g. FS), High impact on Administration, and Facilities. One-Time Investment in New Small Schools Part 4: Small Schools Investment - 28 -

30 OUSD New Small Schools External Evaluation Part 4: Spending Per Student, Small vs. Other Schools Overall spending per student is higher among 1 st year, new small schools due to categorical funding allocations that are driven by federal and state formulas based on the type of students in the school. NOTE: Existing Small Schools + 2 nd Year and Above New Small Schools are identified as those schools that fell below the small school cut-off as defined by OUSD. 1 st Year, New Small Schools are those schools that are opening in the 2007-08 school year. OUSD Small School Cutoff Elementary – 360 students Middle – 300 students High – 400 students

31 OUSD New Small Schools External Evaluation Part 4: Subsidizing 1 st Year and Existing Small Schools OUSD provided additional subsidies starting in 2006-07 through the Targeted Instructional Improvement Grant (TIIG). There are two types of subsidies that new, small schools received: (1) For being in their 1 st or 2 nd year of operation and (2) For being classified as a small school. 2006-072007-08 1 st Year New, Small Schools Subsidy for opening/ closing schools $1.01M ($59K avg. per school) $805K ($45K avg. per school) Small school subsidy $853K ($71K avg. per school) $845K ($94K avg. per school) Existing small schools + 2 nd Year and Above, New Small Schools Small school subsidy $3.36M ($59K avg. per school) $3.70M ($60K avg. per school) TOTAL TIIG INVESTMENT $5.22M$5.35M Footnote: New, small schools and existing small schools received subsidies because their total enrollment fell below the small school cutoff established by OUSD.

32 OUSD New Small Schools External Evaluation Part 4: Comparison of allocation methods # of students enrolled Total subsidy allocation at a school 360 cutoff 200 100 0 Gap student allocation method Binary allocation method Illustrative The proposed gap student allocation method reverses which schools get the most subsidy

33 OUSD New Small Schools External Evaluation - 32 - New Small Schools bring stakeholders together in new ways to create the conditions for student success, which is a major but not singular step in raising student achievement. There are a number of new small schools that have leveraged these conditions to accelerate student achievement. Having established these preconditions in almost all the new small schools, it will now be critical to press forward with academic rigor and quality of instruction while holding school accountable for performance with adequate levels of support. OUSD has shifted our small schools strategy over time, based on lessons learned, to focus on providing more structure and support for the incubation process and ongoing monitoring and intervention. We believe that this approach will improve the performance of all of our small schools. In order to continue to improve the effectiveness of new small schools we need to do a more in-depth analysis about how years of implementation, type of incubation, district support, and differences at elementary, middle and high school impact student achievement results. Part 5: District Reflections

34 OUSD New Small Schools External Evaluation Phase II Evaluation Focus Areas Analysis of performance at elementary, middle and high school level in alignment with District academic goals especially college readiness and post- secondary experiences/ choices of students who attend new small schools. Analysis of student growth for students who participate in new small school for 2 or more years, and movement between performance bands on CST, especially out of far below and below basic. Analysis of teacher and leader recruitment, quality, and retention as both a measure of success and as a variable that impacts student achievement. Further analysis of how and to what degree new small schools have increased family and community engagement and ownership at a school level. Further analysis of how new small schools have addressed family and community concerns around equity issues, especially related to the overcrowding and school quality. Measures of Success

35 OUSD New Small Schools External Evaluation Phase II Evaluation Focus Areas (continued) School Quality and Functioning Analysis of how decision making to incubate, incubation approach, level of community engagement, options/ choice, age of school, and specific program design elements impact measures of success. Increase depth of analysis of school fidelity to implementation models through on-site observations and evaluation using school quality rubrics. Financial Analysis of the resources required for full initial implementation and on- going sustainability of new school models. Study and illustrate the intended and unintended intersections and impacts of new small schools with Results Based Budgeting and service economy/ central office services. Analysis of the investment per graduate of new small schools including the impact of increased graduates in our community.

36 OUSD New Small Schools External Evaluation - 35 - New Small Schools Evaluation Board Presentation September 26, 2007

37 OUSD New Small Schools External Evaluation - 36 - Small Schools Evaluation Appendix

38 OUSD New Small Schools External Evaluation - 37 - CONCLUSIONS: Before Small Schools Initiative and Expect Success Performance Before and After Small Schools Academic Performance Index (API) <500 500-599 600-699 700-799 >800 This map illustrates OUSD schools in 1999 and their API performance range. Notice the number of red schools (n=42). OUSD Schools: 1999 API Performance

39 OUSD New Small Schools External Evaluation - 38 - CONCLUSIONS: After Small Schools initiative and Expect Success Performance Before and After Small Schools Academic Performance Index (API) <500 500-599 600-699 700-799 >800 OUSD Schools: 2006 API Performance This map illustrates OUSD schools in 2006 and their API performance range. Notice the number of red schools (n=3). Eight new Small Schools were opened in 2005-06. *First-year schools in 2006-07

40 OUSD New Small Schools External Evaluation Academic Performance Index (API) <500 500-599 600-699 700-799 >800 *First-year schools in 2006-07 OUSD Schools: 2007 API Performance CONCLUSIONS: After Small Schools initiative and Expect Success Performance Before and After Small Schools This map illustrates OUSD schools in 2007 and their API performance range. Notice the number of red schools (n=5). Twelve new Small Schools were opened in 2006-07.

41 OUSD New Small Schools External Evaluation - 40 - Academic Gains: New Small Schools vs. Comparison * Schools * Comparison School = School that students would have attended if the new school had not opened. Performance Before and After Small Schools

42 OUSD New Small Schools External Evaluation - 41 - Academic Gains: New Small Schools vs. Comparison * Schools

43 OUSD New Small Schools External Evaluation - 42 - Academic Gains: New Small Schools vs. Comparison * Schools * Comparison School = School that students would have attended if the new school had not opened.

44 OUSD New Small Schools External Evaluation - 43 - Academic Gains: New Small Schools vs. Comparison * Schools * Comparison School = School that students would have attended if the new school had not opened.

45 OUSD New Small Schools External Evaluation - 44 - Academic Gains: New Small Schools vs. Comparison * Schools * Comparison School = School that students would have attended if the new school had not opened.

46 OUSD New Small Schools External Evaluation - 45 - Academic Gains: New Small Schools vs. Comparison * Schools * Comparison School = School that students would have attended if the new school had not opened.

47 OUSD New Small Schools External Evaluation - 46 -

48 OUSD New Small Schools External Evaluation - 47 -

49 OUSD New Small Schools External Evaluation - 48 -

50 OUSD New Small Schools External Evaluation - 49 -

51 OUSD New Small Schools External Evaluation - 50 -

52 OUSD New Small Schools External Evaluation - 51 -

53 OUSD New Small Schools External Evaluation - 52 -

54 OUSD New Small Schools External Evaluation - 53 -

55 OUSD New Small Schools External Evaluation - 54 -

56 OUSD New Small Schools External Evaluation - 55 -

57 OUSD New Small Schools External Evaluation - 56 -

58 OUSD New Small Schools External Evaluation - 57 -

59 OUSD New Small Schools External Evaluation - 58 -

60 OUSD New Small Schools External Evaluation - 59 -

61 OUSD New Small Schools External Evaluation - 60 -

62 OUSD New Small Schools External Evaluation - 61 -

63 OUSD New Small Schools External Evaluation - 62 -

64 OUSD New Small Schools External Evaluation - 63 -

65 OUSD New Small Schools External Evaluation - 64 -

66 OUSD New Small Schools External Evaluation - 65 - FINDINGS: Both the model implementation rating and the school functioning rating were positively and significantly correlated with the measures of student, parent, and teacher satisfaction on the Use Your Voice survey. Higher model implementation and school functioning ratings assigned by teachers are linked to higher satisfaction ratings from students and parents (and other teachers). These findings support the validity of the implementation and functioning measures. Small Schools Satisfaction

67 OUSD New Small Schools External Evaluation - 66 - Forecast Methodology: Assumptions


Download ppt "OUSD New Small Schools External Evaluation - 0 - New Small Schools Evaluation Board Presentation September 26, 2007."

Similar presentations


Ads by Google