Lessons Learned from AYP Decision Appeals Prepared for the American Educational Research Association Indiana Department of Education April 15, 2004

Why provide an AYP appeal? NCLB requires balancing of: NCLB requires balancing of:  Accountability.  Confidentiality.  Reliability. Section 1116(b)(2) of ESEA requires “opportunity to review and present evidence” before a school is identified for improvement. Section 1116(b)(2) of ESEA requires “opportunity to review and present evidence” before a school is identified for improvement.

Why provide an AYP appeal? If the SEA determines that a school has not achieved AYP for two years in a row, the SEA must provide a school with an opportunity to review the data, including academic assessment data, on which a proposed identification for school improvement is based. (Non-regulatory guidance) If the SEA determines that a school has not achieved AYP for two years in a row, the SEA must provide a school with an opportunity to review the data, including academic assessment data, on which a proposed identification for school improvement is based. (Non-regulatory guidance)

Why provide an AYP appeal? It is an issue of fundamental fairness. It is an issue of fundamental fairness.

What limits are placed on appeal? Each SEA’s annual determination of school progress is based on the application of formulas defined and approved in its accountability system. Therefore, with rare exceptions, only statistical errors in the underlying data would provide cause for a reconsideration of the school’s status. (Non- regulatory guidance) Each SEA’s annual determination of school progress is based on the application of formulas defined and approved in its accountability system. Therefore, with rare exceptions, only statistical errors in the underlying data would provide cause for a reconsideration of the school’s status. (Non- regulatory guidance)

What limits are placed on appeal? However, if the principal or a majority of the school.s parents believe that the identification was made in error for statistical or other substantive reasons, the principal may provide supporting evidence to the LEA, and the LEA, in conjunction with the SEA, must consider it. (Non- regulatory guidance) However, if the principal or a majority of the school.s parents believe that the identification was made in error for statistical or other substantive reasons, the principal may provide supporting evidence to the LEA, and the LEA, in conjunction with the SEA, must consider it. (Non- regulatory guidance)

What limits are placed on appeal? School may appeal its category placement based on objective factors the school considers relevant because the annual assessment data does not provide an accurate picture of school improvement and performance, including significant demographic changes in the student population, errors in data, or other significant issues. (Indiana rule) School may appeal its category placement based on objective factors the school considers relevant because the annual assessment data does not provide an accurate picture of school improvement and performance, including significant demographic changes in the student population, errors in data, or other significant issues. (Indiana rule)

Indiana’s Appeal Process Administrative Orders and Procedures Act applies. Administrative Orders and Procedures Act applies. Appeals are based on written submission. Appeals are based on written submission. Objective data are required. Objective data are required. Appeals must be based on specific criteria. Appeals must be based on specific criteria.

Indiana’s Appeal Criteria State test data are incorrect – DOE proposes adjustment. State test data are incorrect – DOE proposes adjustment. Test data are correct and complete but do not correctly portray school – School must submit evidence of achievement based on measurement aligned with: Test data are correct and complete but do not correctly portray school – School must submit evidence of achievement based on measurement aligned with:  Indiana Academic Standards.  ISTEP passing score.

Indiana’s Appeal Criteria Test data are correct but provide an incomplete picture of the school or corporation – Test data are correct but provide an incomplete picture of the school or corporation –  School submits data explaining demographic or school organization changes and prior test data for students.  School submits other data showing achievement growth of students for whom test data were incomplete.

Indiana’s Appeal Criteria Other data are correct but do not correctly portray school or school corporation – School submits evidence of unusual circumstance(s) affecting attendance or participation. Other data are correct but do not correctly portray school or school corporation – School submits evidence of unusual circumstance(s) affecting attendance or participation.

What happened? Over 300 appeals were filed. Over 300 appeals were filed. Almost none met the appeal criteria. Almost none met the appeal criteria. It was the first year of AYP for all schools and also the first year that student level data were collected. It was the first year of AYP for all schools and also the first year that student level data were collected.

How did we handle it? Reopened data submission. Reopened data submission. Considered Title I schools separately. Considered Title I schools separately. Processed appeals. Processed appeals.

Why did they appeal? A number of factors related to the implementation of the ESEA are troublesome to school people, should be troublesome to everyone, and provide justification for a flexible policy in granting appeals, and specifically: A number of factors related to the implementation of the ESEA are troublesome to school people, should be troublesome to everyone, and provide justification for a flexible policy in granting appeals, and specifically:  The single goal for each subject area applies to every identified student group.  Failure to demonstrate AYP was attributable to only one student group.

Why did they appeal? The procedures to establish AYP determinations are inconsistent with those specified in ESEA and deny the school board, superintendent, principals, and parents their rights as specified in ESEA. The procedures to establish AYP determinations are inconsistent with those specified in ESEA and deny the school board, superintendent, principals, and parents their rights as specified in ESEA. The Elementary and Secondary Education Act is inconsistent with the Individuals with Disabilities Education Act (IDEA). The Elementary and Secondary Education Act is inconsistent with the Individuals with Disabilities Education Act (IDEA).

Why did they appeal? The intent of the NCLB is that a corporation or school will be given its AYP determination at the end of the year and will have the ensuing school year to take corrective action relative to the conditions that led to the determination. The intent of the NCLB is that a corporation or school will be given its AYP determination at the end of the year and will have the ensuing school year to take corrective action relative to the conditions that led to the determination. Special education students who participated in the alternate assessment were counted as “Not Proficient” in AYP determinations. Special education students who participated in the alternate assessment were counted as “Not Proficient” in AYP determinations.

Why did they appeal? It is inconsistent to determine that a high performing school (overall) has not made AYP. It is inconsistent to determine that a high performing school (overall) has not made AYP. Students in the school or school corporation demonstrated improvement (sometimes based on norm-referenced tests), and the school should have been considered to have demonstrated AYP. Students in the school or school corporation demonstrated improvement (sometimes based on norm-referenced tests), and the school should have been considered to have demonstrated AYP.

Why did they appeal? AYP determinations included students who had poor attendance. AYP determinations included students who had poor attendance. AYP determinations included students who would have passed ISTEP+ but for some temporary intervening condition, such as emotional distress. AYP determinations included students who would have passed ISTEP+ but for some temporary intervening condition, such as emotional distress. Limited English Proficient (LEP) students were included in state testing even though they had no chance of passing. Limited English Proficient (LEP) students were included in state testing even though they had no chance of passing.

What did we learn? We must communicate better. We must communicate better. There was merit to some of the arguments, as evidenced by new flexibility in: There was merit to some of the arguments, as evidenced by new flexibility in:  Alternate assessment.  English Language Learner assessment.  Participation rate calculation.

What do we still need to know? Are we identifying the right schools? Are we identifying the right schools?

How will we know? Continue appeal process. Continue appeal process. Track scale score increases. Track scale score increases. Evaluate the validity of our system – Evaluate the validity of our system –  A Framework for examining Validity in State Accountability Systems