1 Approaches to Implementing the 2% Cap for Adequate Yearly Progress NCES Summer Data Conference Washington, DC July 2008

2 Nancy Stevens Office of Assessment, Accountability, and Data Quality Texas Education Agency Li-Chin Wu Division of Performance Reporting Texas Education Agency

3 Texas Before NCLB State Developed Alternative Assessment instructional level rather than enrolled grade level ARD committee set level and student performance standard about 7-8% of students Locally Determined Alternate Assessments locally developed or selected tests fewer than 1% of students

4 Texas After NCLB All students included in state assessment program Texas Assessment of Knowledge and Skills (TAKS) Performance of all students evaluated against grade- level achievement standards Federal cap limit on use of proficient results based on alternate or modified achievement standards in AYP performance measures: 1% alternate achievement standards (TAKS-Alt) 2% modified achievement standards (TAKS-M)

5 Texas After NCLB (cont.) Student performance will be a greater factor than the caps The caps will apply to a very small number of all students tested 1% and 2% caps represent very high standards

6 Goals for 2% Cap Students: promote appropriate assessment decisions for students with disabilities Statute: meet statutory requirements and intent Validity: minimize unintended consequences Equity: distribute exceeders and keepers across campuses fairly Simplicity: understandable easy to replicate by school districts Resources: staff and time

7 Incorporating School District Input During Policy Development Process State solicits feedback on options State selects approach that more closely reflects local decisions Before AYP Determinations Districts set campus caps or prioritize campuses During AYP Determinations Districts identify individual students whose proficient test scores are retained if cap 2% cap exceeded

8 Overall Design 1% cap Did not have to be same approach as 2% cap Fewer options considered By random assignment By disability category

9 Overall Design for 2% Cap One district-wide pool Rules for selecting students from district pool Separate pools for each campus Rules for assigning campus caps or ranking campuses Rules for selecting students from campus pools

10 Campus Pools Option 1: Campus Cap Determine cap for each campus based on current and/or historical proportion of district students: receiving special education services tested on alternate assessments proficient on alternate assessments Rules for selecting keepers/exceeders if campus exceeds cap Rules for allocating extra spaces if campus does not use all allowed under campus cap

11 Campus Pools Option 1: Campus Cap (cont.) Pros Reflects local policy decisions Potentially rewards campuses that historically and appropriately serve high number of students with disabilities

12 Campus Pools Option 1: Campus Cap (cont.) Cons Potentially rewards campuses that over- identify students for alternate assessment May encourage concentrating programs on specific campuses or discourage mainstreaming in order to maintain campus cap

13 Campus Pools Option 1: Campus Cap (cont.) Cons Slight variations in testing from year to year may result in changes to campus cap May be difficult to implement Could result in the district missing AYP

14 Campus Pools Option 2: Campus Ranking Rank campuses Select proficient scores from highest ranked campus first, going down the list until district cap limit is reached Rules for selecting students from campus pool

15 Campus Pools Option 2: Campus Ranking (cont.) Example: Strategic Campus Ranking Rank campuses strategically: Highest stage identification for SIP Title I missed AYP in prior year Title I campus

16 Campus Pools Option 2: Campus Ranking (cont.) Example: Strategic Campus Ranking (cont.) Pros Balance perceived inequities in AYP interventions (Title I vs. non-Title I campuses) Simple to understand

17 Campus Pools Option 2: Campus Ranking (cont.) Example: Strategic Campus Ranking (cont.) Cons Reward campuses with performance problems Not consistent with intent of NCLB May not help top-ranked campuses Could result in the district missing AYP

18 Selection of Students for 2% Cap By Random Assignment By Test Score By Grade Level By Maximum Benefit

19 Selection of Students (cont.) District pool or separate campus pools Significantly different outcomes Single or combined selection criteria First or primary sort is greater factor in determining outcomes Final unique sort as a tie-breaker

20 Selection of Students By Random Assignment Students randomly selected up to the cap limit District or campus pools Does not need tie-breaker Can be used as final tie-breaker with other methods

21 Selection of Students By Random Assignment (cont.) Pros Simple to understand Simple to implement for most districts Impartial over time No unintended policy consequences (cannot be manipulated)

22 Selection of Students By Random Assignment (cont.) Cons Cannot be replicated by districts May not appear to be fair in any one year Does not provide any incentive – disconnect between campus behavior and outcomes

23 Selection of Students By Test Performance Students sorted from lowest to highest test score and keepers selected up to the cap limit District or campus pools Can be used in conjunction with other criteria Needs a final tie-breaker

24 Selection of Students By Test Performance (cont.) Pros Encourages testing higher performing students on the regular test Simple to understand Can be replicated by districts Simple to implement Most similar to method used in Texas with SDAA/LDAA

25 Selection of Students By Test Performance (cont.) Cons If implemented at the district level: may be perceived as punitive toward campuses with strong instructional programs may not result in fair distribution of keepers and exceeders across campuses

26 Selection of Students By Grade Level Students sorted from highest to lowest grade and keepers selected up to the cap limit District-level approach Needs to be used with at least one more criteria

27 Selection of Students By Grade Level (cont.) Pros Provides strong incentive for elementary schools to focus instruction on maintaining grade-level proficiency and testing on regular grade-level assessment Rewards high schools that have successfully accelerated instruction so that students previously instructed and tested below grade level are meeting grade-level modified academic achievement standards

28 Selection of Students By Grade Level (cont.) Pros High schools, which are overrepresented among campuses not meeting AYP, are least adversely affected by the cap Simple to understand Can be replicated by districts Simple to implement

29 Selection of Students By Grade Level (cont.) Cons Has appearance of being unfair to elementary schools AYP results for elementary schools may be adversely affected disproportionately Positive instructional incentives may be short-term May have unintended consequences long-term

30 Selection of Students By Maximum Benefit Select proficient results from each campus that will result in the maximum benefit for the campus Select number and type of students (student groups) needed for the campus to meet AYP Campus-level approach Criteria for each campus based on need

31 Selection of Students By Maximum Benefit(cont.) Pros Potentially minimizes the number of campuses that miss AYP solely due to selection criteria for the 2% cap Uses state data processing capacity to select students that districts and campuses would likely select if 2% cap implemented locally

32 Selection of Students By Maximum Benefit(cont.) Cons Students included in the 2% cap will be selected from student groups that do not meet the AYP standards Selection based on campus need could result in the district missing AYP

33 Texas AYP Combination Method Campus Ranking By campus type High School Combined Elementary/Secondary School Middle/Junior High School Elementary School By grade (high to low) By percent special education (high to low)

34 Texas AYP Combination Method (cont.) Campus Ranking Based on fall enrollment data District opportunity to modify campus ranking

35 Texas AYP Combination Method (cont.) Student Selection in 3 Stages First by maximum benefit for campus (campus pool) From highest to lowest ranked campus Select students needed for campus to meet AYP Skip campuses that already meet AYP or will not meet AYP for subject

36 Texas AYP Combination Method (cont.) Student Selection in 3 Stages (cont.) Second by maximum benefit for district (district pool) Third by random selection (district pool)

37 Texas AYP Combination Method (cont.) Pros Campus ranking by grade level has many of the advantages of selecting students by grade level Provides incentive for elementary schools to focus instruction on maintaining grade-level proficiency Rewards high schools that have successfully accelerated instruction

38 Texas AYP Combination Method (cont.) Pros District input before AYP determinations does not interfere with processing timelines Supports local policy decisions on selection of appropriate tests for students with disabilities

39 Texas AYP Combination Method (cont.) Pros Student selection uses state data processing capacity to provide maximum benefit to campuses in implementing 2% cap Second selection for maximum benefit to district removes potential disadvantages of processes that focus on campuses

40 Texas AYP Combination Method (cont.) Cons Benefits of ranking campuses by grade level may be short-term District input into campus ranking resource intensive for little gain and potentially negates benefits of ranking by grade level Disadvantages of selection by maximum benefit – students disproportionately selected from student groups that do not meet AYP

41 Example District - 2% Cap Scenario F: Campus 1 meets AYP *** Campus 2 missed AYP *** *** Campus 3 missed AYP *** Campus 4 meets AYP *** District missed AYP *** Seven Student Groups:A - All B - African American H - Hispanic W - White E - Economically Disadvantaged S - Special Education L - LEP