1. Accountability: Where Are We Going? Martha Musser, Coordinator NYS Education Department

2. Contract for Excellence NCLB Reauthorization NYC and other local accountability initiatives This may be the Dawning of the Age of Accountability

3. Contract for Excellence –Governor has proposed a Contract for Excellence that calls for an enhanced State accountability system, including: New accountability standards based on State assessments and other indicators of progress, such as graduation rates or college attendance and completion rates. Growth model by 2008-09. Value-added model by 2010-2011 based on new or revised state assessments. Expanded SURR system, resulting in the identification of up to 5% of State school’s by 2011- 2012 for restructuring or reorganization.

4. Contract for Excellence: Plans for Intervention School Review Teams conduct resource, program and planning audits of SINI and SRAP schools and assist all SINI and SRAP schools in development of improvement plans. Joint School Intervention Teams, whose members are either appointed by Commissioner or educators from the district, review and recommend plans for reorganizing or reconfiguring schools that are to be closed. Distinguished Educators assist low-performing schools and districts. The services of all the above are a charge to the school district.

5. Contract for Excellence: Now it’s Personal Commissioner shall define in regulation deficient district performance. School districts hiring a superintendent after the effective date of regulation must include a provision in superintendent’s contract that after two years of deficient district performance the superintendent will fully cooperate with a distinguished educator. After four years of deficient performance, school board must seek to remove a superintendent or provide a rationale to the Commissioner for why board should not take such action. After six years of deficient district performance, the Commissioner shall commence action to remove the board or board members.

6. Distinguished Educators and Deficient District Performance Consult with Commissioner on removal of superintendent. Serve as ex officio, non-voting members of Board of Education. Review all school and district improvement plans and either endorse or make recommendations to the Board of Education for change. Boards of Education must make changes as required by Distinguished Educators unless upon petition by Board of Education Commissioner decides there is a compelling reason not to implement the modification.

7. Status, Growth and Value- Added Models Defined Contract for Excellence requires SED to implement growth and value-added models and meet any Federal requirements for such models.

8. Status Models Defined Status or improvement models, the current requirement for measuring Adequate Yearly Progress (AYP) under No Child Left Behind, measure progress by tracking the improvement at the same grade levels within the school over time. (This year’s grades 3-5 compared to last year’s grades 3-5.)

9. Growth Models Defined Growth models generally refer to accountability models that assess the progress of a cohort of individual students over time with the intent of measuring the progress these students have made (Performance in fourth grade compared to performance in third grade).

10. Value-Added Models Defined Value-added models generally refer to a specific type of growth model in which student demographic data or other statistical controls are used to attempt to analyze the specific effects of a particular school, program, or teacher on student learning. These models ask whether the school has increased the measured achievement of students more than expected based on data from similar schools.

11. Purpose of Status, Growth, and Value Added Models Status model: Determine whether an increasing percentage of students are gaining proficiency over time. Growth model: Determine how much progress groups of students are making over time. Value-added model: Determine success of schools, programs, or teachers by measuring student growth over time while controlling for non-school variables that impact on student performance.

12. Value-Added Models: The Holy Grail of Accountability? –Much controversy about value-added models. –Some claim that good value-added models exist (Sanders), statistical models not based on vertical scales. –Some claim that the state of the psychometric art does not currently support vertical scales. –Value-added models are not easy: Less sophistication is required to make accurate determinations about status then growth. Acquisition of knowledge is not linear. Value-added models are heavily dependent upon the robustness of their assumptions. Value-added models break down more quickly as groups become smaller.

13. Growth Model System Requirements Growth Model System Requirements: Annual assessments in successive grades A unique student identification system Vertically aligned or scaled assessments Grade by grade standards A minimum of two years of assessment data per student Value-added systems require additional data on those demographic or resource factors for which the model seeks to control

14. NY and Growth Models NYS Grades 3-8 Testing Program uses Vertically Moderated Standards (VMS) Student progress is measured from grade-to-grade relative to proficiency in meeting the standards (rather than in terms of change in scale scores). VMS is applicable to some but not all growth model designs. NY will by the end of 2006-07 have only two years of test data, limiting our ability to create growth trajectories for individual students. SED’s goal is to select a growth model architecture by April.

15. NCLB: Status Report Revised State standards and Annual Measurable Objective’s (AMOs) established. 05-06 Accountability Decisions Released. Development of elementary/middle level attendance standards in progress. Regents continuing deliberations regarding raising high school graduation standards. Regents engaging in NCLB reauthorization advocacy.

16. State Standards State Standards for 06-07 are: –155 for Grade 3-8 ELA and math –165 for HS ELA and math State Standards for 07-8 are: –160 for Grade 3-8 ELA and math –170 for HS ELA and math State standard is used to determine which schools must do Local Assistance Plans (LAPs) and which schools and districts may be high performing.

17. School YearElementary-LevelMiddle-LevelSecondary-Level ELAMathELAMathELAMath 2002-0312313610781142132 2003-0412313610781142132 2004–0513114211693148139 2005–06138149126105154146 2006–07146155135117159152 2007–08154162144129165159 2008–09162168154141171166 2009–10169174163152177173 2010–11177181172164183180 2011–12185187181176188186 2012–13192194191188194193 2013–14200200200200200200 Original Annual Measurable Objectives for 2002–03 to 2013–14

18. School YearElementary & Middle-LevelSecondary-Level ELAMathELAMath 2005–0612286154146 2006–0712286159152 2007–08133102165159 2008–09144119171166 2009–10155135177173 2010–11167151183180 2011–12178167188186 2012–13189183194193 2013–14200200200200 Revised Annual Measurable Objectives for 2005–06 to 2013–14

19. Calculating AMOs In 2000-2001 and 2001-2002 combined, 20% of NY’s public school students were enrolled in school’s with PI’s below: –Grade 4 ELA: 123 –Grade 8 ELA: 107 –Grade 4 Math: 136 –Grade 8 Math: 81 In 2004-05, the percent of students below the 2004-2005 AMO’s were: –Grade 4 and 8 ELA combined: 13.3% –Grade 4 and 8 math combined: 4.2% 2005-2006 Grade 3-8 ELA results: –AMO at 13.3% = 122 –AMO at 20.0% = 133 2005-2006 Grade 3-8 math results: –AMO at 4.2% = 86 –AMO at 20.0% = 132

20. Limited English Proficient (LEP) Students  All LEP students in grade K–12 must take the NYSESLAT annually.  LEP students in grades 3 through 8 enrolled in U.S. schools (not including Puerto Rico) for less than one year (enrolled on or after January 2, 2006) were not required to take the NYSTP ELA assessment in January 2007. For such students who did not take the ELA assessment, valid scores on the NYSESLAT Reading/Writing and Speaking/Listening components will meet the ELA participation requirement.  The eligible LEP students must be identified in the repository using Program Service 0242.  NYSESLAT performance levels will not be used in calculating the Performance Index. LEP students meeting the criteria to use the NYSESLAT in lieu of the ELA will not be included in the Performance Index calculation.

21. 2002 Graduation-Rate Cohort Definition This cohort will be used to determine if the district or school meets the graduation-rate requirements for the 2006–07 school year. The 2002 graduation-rate cohort consists of all students, regardless of their current grade status, who were enrolled in the school on October 6, 2005 (BEDS day) and met one of the following conditions:  first entered grade 9 (anywhere) during the 2002–03 school year (July 1, 2002 through June 30, 2003); or  in the case of ungraded students with disabilities, reached their seventeenth birthday during the 2002–03 school year.

22. The State will exclude the following students when reporting data on the 2002 cohort:  students who transferred to another high school (excluded from the high school graduation-rate cohort) or district (excluded from the district graduation-rate cohort) or criminal justice facility after BEDS day 2005;  students who transferred to an approved alternative high school equivalency preparation (AHSEP) or high school equivalency preparation (HSEP) program (CR 100.7) after BEDS day 2005 and met the conditions stated on the next slide;  students who left the U.S. and its territories after BEDS day 2005 and before August 30, 2006; and  students who died after BEDS day 2005 and before August 30, 2006. 2002 Graduation-Rate Cohort Definition (cont’d)

23. Students will be removed from the cohort for the school and district from which they transferred to an AHSEP or HSEP program if the final enrollment record shows that on June 30, 2006 the student a) earned a high school equivalency diploma; or b) was enrolled in an AHSEP or HSEP program. Students will be removed from the school cohort if the enrollment records showed that the student transferred to a different high school and was working toward or earned a high school diploma. Students will be removed from the district cohort if the enrollment records show that the student transferred to a high school in a different district and was working toward or earned a high school diploma. 2002 Graduation-Rate Cohort (Transfers to GED Removed from Cohort)

24. 2003 Graduation-Rate Cohort Beginning with the 2003 graduation-rate cohort (used for accountability in 2007-08):  students are included in the cohort based on the year they first enter grade 9 (or for ungraded students, the year they turn 17).  students who have spent at least five months in a district/school during year 1, 2, 3, or 4 of high school are part of the district/school cohort unless they transfer to another diploma-granting program.

25. Inclusion Rules for the 2003 Graduation-Rate Cohort A student will be included in the district/school cohort if the student’s last enrollment record in the district or school shows:  that the student was enrolled for at least five continuous (not including July and August) months and the ending reason was not one of the following: transferred to another New York State district or school, died, transferred by court order, or left the U.S.  fewer than five month’s enrollment and an ending reason indicating that the student dropped out or transferred to a GED program and the student’s previous enrollment record in that district/school (assuming one exists): –indicates that the student dropped out or transferred to a GED program, and –that the student was enrolled in the district/school for at least five months.

26. School Status for All State Schools 2005-062006-07 NumberPercentNumberPercent Good Standing3,74384.4%3,74984.3% In Improvement Status under Title 1 50211.3%50611.4% Requiring Academic Progress1924.3%1934.3% Total in improvement status69415.6%69915.7% Total Schools4,4374,448

27. NYCRest of StateTotal SchoolsEnrollmentSchoolsEnrollmentSchoolsEnrollment Requiring Academic Progress - Year 191084536400434550888 Requiring Academic Progress - Year 26595738531614459118 Requiring Academic Progress - Year 3102151517184812739996 Requiring Academic Progress - Year 43682411151941251101823 Requiring Academic Progress - Year 51419748110321520780 Requiring Academic Progress - Year 651335005 Requiring Academic Progress - Year 761759006 In Need of Improvement - Year 16249523462810710877630 In Need of Improvement - Year 2584961733279579177574 In Corrective Action4432074232233056754379 Planning for Restructuring466155131304367791987 Restructuring - Year 1221705020204594237509 Restructuring - Year 25752336737416456077 Restructuring - Year 347468621074605754322 Total Title I SINI336309013170140465506449478 Total SRAP86143570107132129193275699 Grand Total422452583277272594699725177 Note: Enrollment is based on counts from October 2005 BEDS survey. Student Enrollment by Accountability Status

28. Statewide Status of SINI and SRAP Schools 2005-06 vs. 2006-07 Status 05-06Status Improved or Stayed the Same Status Became WorseTotal Schools In StatusPercent Improving or Staying the Same SINI 1617013146.6 SINI 233518439.3 Corrective Action19769520.0 Planning for Restructuring 12314327.9 Restructuring 117587522.7 Restructuring 211627315.1 SRAP 144287261.1 SRAP 228144266.7 SRAP 315445925.4 SPAP 4691540.0 SRAP 5 or 631475.0 Total24944469335.9 Note: A school’s status would improve or stay the same if the school made AYP in the area(s) of identification which gave the school its status. A school’s status becomes worse if it fails to make AYP in one or more of the area of identification which gave the school its status.

29. WEIGHTED PI BY ACCOUNTABILITY STATUS - ALL SCHOOLS 3-8 ELA3-8 MathHS ELAHS Math Acct Status Acct Enrol>30 Wtd PI Acct Enrol>30 Wtd PI Acct Enrol>30 Wtd PI Acct Enrol>3 0Wtd PI In Good Standing920,862163914,553168109,948183109,948185 In Need of Improvement - Year 137,28313237,0691373,6941523,694159 In Need of Improvement - Year 241,84513341,1321364,3861464,386150 In Corrective Action24,98812724,7471253,9891443,989149 Planning for Restructuring32,14311731,6391109,9711389,971145 Restructuring - Year 120,82611920,4471141,6191221,619130 Restructuring - Year 237,30411636,921116874132874139 Restructuring - Year 341,20811640,601111867103867124 Requiring Academic Progress - Year 19,3141549,2141548,3461808,346184 Requiring Academic Progress - Year 216,51615816,3641618,7001788,700182 Requiring Academic Progress - Year 39,7001509,60214563171696,317170 Requiring Academic Progress - Year 412,52614212,41814117,15114917,151150 Requiring Academic Progress - Year 53,7631253,7221133,6281323,628141 Requiring Academic Progress - Year 644311442799419116419118 Requiring Academic Progress - Year 71,236941,22091212118212118 Has No Status - Regulations Do Not Apply3161453081400000 Total Title I SINI235,597123232,55612225,40014025,400147 Total SRAP53,49814852,96714744,77316144,773164 Grand Total1,210,2731541,200,384158180,121171180,121175

30. NCLB Reauthorization: Regents have developed positions regarding: -Schools and districts with multiple federal designations -Growth Models -Targeted interventions -AYP Methodologies -Additional time to meet graduation standards -ELL testing issues -Implementation of choice and SES -Safe Schools

31. NCLB Reauthorization: What we are Hearing –Serious discussions about NCLB are beginning but reauthorization more likely to occur in 2009 than 2007, very unlikely to occur in 2008. –Key issues being discussed: Full funding Growth models N size and confidence intervals Required interventions, including SES and choice Assessments: LEP, SWD, high school Highly Qualified Teachers vs. High Quality Teaching National Standards Science Assessments

32. Other Accountability Initiatives: NYC Children First: –Valued Added Model. –More Ragu than KISS. –Each school receives letter grade (A-F) on school progress report. –Progress report consists of four categories: School Environment (15%) Student Performance (30%) Student Progress (55%) Additional Credit

33. Children First Progress Report School’s grade is based upon how well the school performs relative to other City schools (1/3 of grade) and to a peer group of approximately 40 schools with similar demographics (2/3 of grade). State assessments used to measure performance of students over time. Additional credit given to improved performance among low achieving students and various disaggregated groups. Factors such as parent, student, and teacher surveys; credit accumulation; PSAT scores used in determining school grade.

34. Challenges for SED Growing Pains Testing Times Vision 2020 A Cart Before the Horse?

35. Challenges Ahead for School Districts Immediate: Single Grade 3-8 Performance Index makes schools and districts responsible for more disaggregated groups. New standard setting for grade 3-8 assessments may challenge middle schools even more. Changes in testing practices for LEP students require that bilingual education programs emphasize rapid acquisition of English as well as fluency in the native language. New graduation standards will raise expectations for middle schools and force high schools to reveal “hidden students.” Longer term: If Governor and Legislature show school districts the money, they are also going to demand that districts show results.

36. Challenges Ahead: Lists, Lists, Lists How do we make all of these work together: SURR SINI, DINI SRAP, DRAP IDEA Districts Title III AMAO’s Persistently Dangerous Schools High School Initiative Highly Qualified Teachers

37. Key Questions: How do we design accountability models that compel movement from awful to adequate without impeding the movement from good to great? How do we move from beating the odds to changing the odds? This may be the Dawning of the Age of Accountability

38. More Information Ira Schwartz, Coordinator Office of School Improvement and Community Services (NYC) ischwart@mail.nysed.gov 718 722-2796 Accountability PowerPoint for 2006-07: http://www.emsc.nysed.gov/irts/accountability/2005- 06/accountability-rules-Nov2006_files/frame.htm Manuals for NYS Student Identifier System and the Repository System: http://www.emsc.nysed.gov/irts/SIRS/home.shtml