1. NCLB: Then and Now

2. “…to ensure that all children have a fair, equal, and significant opportunity to obtain a high-quality education and reach, at a minimum, proficiency on challenging State academic achievement standards and state academic assessments” Purpose of No Child Left Behind

3. Critical Components of NCLB Accountability Annual testing in language/arts reading and mathematics for all public school students in grade 3-8 and at least once in high school. Additional academic indicator in elementary and middle schools plus high school graduation rate. Requirement for increasing percentages of students to demonstrate proficiency each year (Annual Measurable Objective = AMO). Requirement to disaggregate and hold schools and districts accountable for the performance of students by racial/ethnic, and low-income, LEP, and SWD groups. Requirement for 95% participation in State Assessments. Requirement that disaggregated groups, schools, and districts demonstrate Adequate Yearly Progress (AYP) either through exceeding the Effective Annual Measurable Objective (EAMO) or by closing the gap in terms of students not proficient (Safe Harbor). Schools and Districts that do not make AYP are identified as in Need of Improvement. Such schools and districts are subject to increasing levels of assistance, interventions, and consequences.

4. What Critics are Saying Goals of NCLB are inappropriate or unrealistic leading states to: –Game the system and define proficiency down –Overemphasize test preparation –Narrow the curriculum NCLB’s accountability system: –Is both overly simplistic and overly complex –Unfairly disadvantageous certain schools and districts –Diverts resources from school improvement to ineffective choice and SES programs.

5. What Supporters Are Saying –NCLB is pushing us to improve assessment and data collection. –NCLB is forcing us to confront the gap in student performance. –NCLB is fostering choice and competition in education. –NCLB is providing additional resources to those most in need. –NCLB is ensuring that there are consequences for performance.

6. New York’s NCLB Plan: A brief History –Submitted December 2002 –Revised and Approved January 2003 –Revisions submitted: October 2004 April 2005 June 2005 October 2005 March 2006 July 2006

7. M.L. Mencken said: For every complex problem, there is a solution that is clear, simple, and wrong.

8. Participation Rate for schools with 40 or more students enrolled on test day Check the Performance Index for each group with 30 or more students Every group’s PI is NOT equal to or greater than the Effective AMO Every group’s PI is equal to or greater than the Effective AMO Find safe harbor targets in ELA or math and determine if the group met the science qualification for safe harbor A group whose PI is below its Effective AMO did NOT make safe harbor School did not test 95 percent of every group of 40 or more School tested 95 percent of every group of 40 or more Each group whose PI is below its Effective AMO made safe harbor NO AYP AYP Determining AYP in Elementary- and Middle-Level ELA or Math for Schools with 30 or More Continuously Enrolled Students Schools with 30 to 39 students Compute weighted average of 2002- 03 and 2003-04 participation rate for groups below 95% Above 95% Below 95%

9. Will Change do you Good? Assessment Program Annual Measurable Objectives Methodology for Determining Elementary- and Middle-level accountability Methodology for Determining District-level Adequate Yearly Progress Definitions of high school ELA and math accountability and graduate rate cohorts Requirements for meeting participation rate Determining Adequate Yearly Progress for English language learners Determining Adequate Yearly Progress for Students with disabilities.

10. Grades and Levels For 2002-2003, 2003-2004, and 2004-05 School Years, NY had three grade levels: –Elementary (Grade 4) –Middle (Grade 8) –High School Beginning in 2005-2006, NY has two grade levels: - Elementary-Middle (Grades 3-8) - High School

11. Calculating the Grade 3-8 Performance Index GradeNumber Levels of Students 1234 36014142012 4806124022 5601220208 TOTAL20032468042 Total % 16% 23% 40% 21% Index = (23+40+21+40+21)=145

12. School YearElementary-LevelMiddle-LevelSecondary-Level ELAMathELAMathELAMath 2002-0312313610781142132 2003-0412313610781142132 2004–0513114211693148139 2005–06138149126105154146 2006–07146155135117159152 2007–08154162144129165159 2008–09162168154141171166 2009–10169174163152177173 2010–11177181172164183180 2011–12185187181176188186 2012–13192194191188194193 2013–14200200200200200200 Original Annual Measurable Objectives for 2002–03 to 2013–14

13. School YearElementary & Middle-LevelSecondary-Level ELAMathELAMath 2005–06XX154146 2006–07XX159152 2007–08XX165159 2008–09X+YX+Y171166 2009–10X+2YX+2Y177173 2010–11X+3YX+3Y183180 2011–12X+4YX+4Y188186 2012–13X+5YX+5Y194193 2013–14200200200200 Revised Annual Measurable Objectives for 2005–06 to 2013–14

14. Calculating AMOs In 2000-2001 and 2001-2002 combined, 20% of the NY’s public school students were enrolled in school’s with PI’s below: –Grade 4 ELA: 123 –Grade 8 ELA: 107 –Grade 4 Math: 136 –Grade 8 Math: 81 In 2004-05, the percent of students below the 2004-2005 AMO’s were: –Grade 4 and 8 ELA combined: 13.3% –Grade 4 and 8 math combined: 4.2% Estimated 2005-2006 Grade 3-8 ELA results: –AMO at 13.3% = 122 –AMO at 20.0% = 133 –Average of Original Grade 4 and 8 2005-06 AMO: 132

15. Sample Identification of School for Improvement Status School A fails to make AYP in the following groups: –Grade 4 ELA White Students in 2004–05 –Grade 3-8 Math Economically Disadvantaged Students in 2005–06 School A is not identified for improvement because it has not failed to make AYP for two consecutive years in the same subject and level. School B fails to make AYP in the following groups: –Grade 4 ELA Asian Students in 2004–05 –Grade 3-8 ELA LEP Students in 2005–06 School B is identified for improvement because it has failed to make AYP for two consecutive years in the same subject and grade (grade 4 ELA).

16. 2002-2003: One bite at the apple system 2003-2004 and 2004-2005: Three bites at the apple system 2005-2006: Two Bites at the apple system: District Level Accountability based on subject. For a district to fail to make AYP in a subject it must fail to make AYP at all levels in that subject. For example, if a district fails to makes AYP in Grade 3-8 ELA and fails to make AYP in High School ELA in 2005-2006 and 2006-07, the district is identified as a DINI for ELA. District-Level Accountability

17. Guide to Accountability Cohorts High schools are accountable for three areas: English and mathematics performance; English and mathematics participation; and graduation rate. A different cohort of students is measured in each of these areas. Further, the cohort used to measure English and mathematics performance has been redefined beginning with the 2002 cohort; the cohort used to measure graduation rate has been redefined beginning with the 2003 cohort.

18. 2005-06 School Year

19. Accountability Cohort Definition 1999 – 2001 Cohorts: Accountability cohort consists of all students, regardless of their current grade status, who were enrolled in the school on BEDS day of the third year following first entry into grade 9 (anywhere) or in the case of ungraded students with disabilities, reached their nineteenth birthday during the school year and did not subsequently transfer to another high school, become incarcerated, leave the country, or die. 2002 and Beyond Cohorts: Accountability cohort consists of all students, regardless of their current grade status, who were enrolled in the school on BEDS day of the fourth year following first entry into grade 9 (anywhere) or in the case of ungraded students with disabilities, reached their twentieth birthday during the school year and did not subsequently transfer to another high school, become incarcerated, leave the country, or die.

20. Accountability Cohort Examples: 2005-2006 Old rules: A student who entered grade 9 in September 2001 in School A and dropped out of School A prior to October 2003 is not part of the ELA and math accountability cohort for School A. Revised rules: A student who entered grade 9 in September 2002 in School B and dropped out of School B prior to October 2005 is not part of the ELA and math accountability cohort for School B. Revised rules exclude students who dropped out between October 2004 and October 2005.

21. Accountability Cohort Examples: 2005-2006 Old rules: A student who entered grade 9 in September 2001 in School A and transferred to School B prior to October 2003 is part of the ELA and math accountability cohort for School B. Revised rules: A student who entered grade 9 in September 2002 in School A and transferred to School B prior to October 2005 is part of the ELA and math accountability cohort for School B. Revised rules include students who transferred into School B between October 2004 and October 2005 and remained enrolled as of October 2005.

22. Graduation Cohort Definition 1999 – 2002 Cohorts: Accountability cohort consists of all students, regardless of their current grade status, who were enrolled in the school on BEDS day of the third year following first entry into grade 9 (anywhere) or in the case of ungraded students with disabilities, reached their nineteenth birthday during the school year and did not subsequently transfer to another high school, become incarcerated, leave the country, or die. 2003 and Beyond Cohorts: Accountability cohort consists of all students, regardless of their current grade status, who were enrolled in the school for five consecutive months following first entry into grade 9 (anywhere) or in the case of ungraded students with disabilities, who reached their seventieth during the school year and did not subsequently transfer to another high school, become incarcerated, leave the country, or die.

23. Graduation Rate Cohort Examples Students not included in the West High School cohort: –CURRENT: A student who entered grade 9 at the school in September 2002 and dropped out prior to October 2004 – NEW: A student who entered grade 9 in September 2003 and enrolled in the school less than five months prior to dropping out.

24. Graduation Rate Cohort Examples Students included in the West High School cohort: –CURRENT: A student who entered grade 9 in September 2002, was enrolled in the school in October 2004, and dropped out of the school after October 2004 and did not reenter a degree-granting program. –NEW: A student who entered grade 9 in September 2003, enrolled in the school for five months and dropped out prior to June 2007 and did not reenter a degree-granting program.

25. Implications of Changes Graduation Rate and ELA and math accountability cohort will become more independent. Accountability cohort will include more students who transfer into a school but will exclude more dropouts. The calculated Performance Index for most schools and districts is higher using this definition. Graduation cohort will include more students who drop out. The calculated Graduation Rate for most schools and districts is lower using this definition.

26. One Good Thing The “first test” as a senior rule has been eliminated. Students highest ELA and math performance during cohort period may be used for accountability beginning with 2002 cohort.

27. Accountability for Limited English Proficient Students 2002-2003 to 2005-2006: Accountability and participation for certain LEP students on Grade 4 and 8 ELA could be based on performance on New York State English as a Second Language Achievement Test (NYSESLAT). Students held accountable based on NYSESLAT tests were not required to take the regular ELA assessments. Beginning in 2006-2007: Only newly arrived English language learners (less than one year in the US) may take the NYSESLAT in lieu of regular Grade 3-8 ELA assessments to meet participation requirements. NYSELSAT scores no longer used to calculate Performance Index. Beginning in 2006-2007: Performance of former LEP students may be included in LEP subgroup for up to two years following exit from LEP status.

28. Students with Disabilities Participation in State Assessments General assessment based on grade the student is in, or not assigned grade level (ungraded): based on chronological age (which is consistent with birth dates of non- disabled peers)

29. Students with Disabilities Participation in State Assessments New York State Alternate Assessment (NYSAA) not assigned grade level (ungraded): based on chronological age chart on field memo

30. NYSAA Linked to grade level content standards Changes in NYSAA administration format and timetable beginning this year http://www.vesid.nysed.gov/specialed/alterasses sment/changesadmin.htm

31. Accountability for Students with Disabilities Modified Standards (2%) –For 2005-06, Grade 3-8 ELA and Math: 34 index points added to SWD group if SWD subgroup met participation rate and SWD subgroup was only one to fail to make AYP. If Performance Index with inclusion of 34 points meets or exceeds AMO, the school or district has made AYP on that criterion

32. IDEA District Accountability: Levels of Intervention In need of assistance In need of intervention In need of substantial intervention

33. District in Need of Assistance School district requires technical assistance in order to improve performance Criteria: –Graduation rate is more than 18.5% and less than 35%; and/or –Drop out rate is 20% to less than 33%; and/or –Math and ELA State assessment results: score on performance index in two or three areas is below the State average for SWDs and district did not make AYP in these areas. 58 school districts are “in need of assistance”

34. District in Need of Intervention Requires direct State involvement and oversight to improve performance Criteria: –Graduation rate is 18.5% or less; and/or –Drop out rate is 33% or higher; and/or –Math and ELA State assessment results: Score on performance index in four areas is below State SWD average and district did not make AYP in these areas. 17 school districts are “in need of intervention”

35. District In Need of Substantial Intervention Criteria: –History of chronic noncompliance or substantial failure to achieve progress and/or –Systemic agency-wide problem of noncompliance and poor performance of students with disabilities No school districts identified this year as in need of substantial intervention.

36. Are we done yet? More possible changes: –Increase in the graduation standard from 55% –Replacement of elementary and middle level science with attendance rate. –Addition of high graduation and attendance rates as criteria for identification of School Under Registration Review. –New Federal rules on accountability for SWDs. –New racial/ethnic groups for accountability? –Growth model?

37. Challenges Ahead Single Grade 3-8 Performance Index makes schools and districts responsible for more disaggregated groups. New standard setting for grade 3-8 assessments may challenge middle schools even more. Changes in testing practices for LEP students require that bilingual education programs emphasize rapid acquisition of English as well as fluency in the native language. New graduation standards will raise expectations for middle schools and force high schools to reveal “hidden students.”

38. Challenges Ahead: Lists, Lists, Lists SURR SINI, DINI SRAP, DRAP IDEA Districts Title III AMAO’s Persistently Dangerous Schools High School Initiative

39. Growth Models: The Holy Grail of Accountability –Much controversy about Growth Models –Some claim good growth models exist (Sanders) –Some claim that the state of the psychometric art does not currently support valid and reliable growth models –Growth models are not easy: Less sophistication required to make accurate determinations about status then growth. Acquisition of knowledge is not linear. Growth models break down more quickly as groups become smaller.

40. If You Take Away Nothing Else.. If a schools fails to make AYP, it is most often because of the performance of its most vulnerable populations, but Systemic problems most often manifest themselves in the most vulnerable populations.

41. More Information Ira Schwartz, Coordinator Accountability, Policy, and Administration New York State Education Department Office of School Improvement and Community Services ischwart@mail.nysed.gov 718 722-2796