1 Mining achievement data to guide policies and practices on assessment options Scott Marion Brian Gong Mary Ann Simpson National Center for the Improvement of Educational Assessment NCEO Teleconference February 6, 2006

2 Study Questions Are most special education students low performing? Are most low-performing students in special education? Is there a relationship between disability code and performance on statewide assessments?

3 Method Recent assessment data, Math and ELA from 5 states (2003, 2004) –States are geographically distributed, but no large states are included –Detailed IDEA classification available for students from 2 of these states Examine student performance by special education status, all 5 states Examine student performance by detailed IDEA classification, 2 states

4 Analysis In the spirit of Tukey, we used very simple descriptive statistics and graphical representations of the results The following graph was done with a crosstab of scale score values by special education status and then transformed into a stacked histogram

5 Vertical Lines shows lowest 3 percent & NCLB Proficient

6 What does the chart mean? Note that special education students are represented at all scale scores in the distribution Note that special education students are represented at all scale scores in the distribution Importantly, a significant number of general education students are found in the lowest scoring three percent of students Importantly, a significant number of general education students are found in the lowest scoring three percent of students

7 Specific Disability By State (% of Special Ed. Students) Disability State 1 (grades 4, 8) (n = 14,922) State 2 (grades 4,8) (n = 21,954) Mental Retardation3%13% Learning Disability54%47% Emotional Disturbance9%7% Speech/Language Impairment16%14% Multiple Disabilities<1%1% Health Impairment<1%1% Orthopedic Impairment<1%1% Other Health Impairment15%13% Visual Impairment<1% Autism2% Traumatic Brain Injury1%<1% Deaf-Blindness0%<1% Developmental Delay0%<1%

8 Percent of Special Education Students Proficient by State and Disability Disability State 1 (grades 4, 8) (n =1,389 ) State 2 (grades 4,8) (n = 4,048) MathematicsELAMathematicsELA Mental Retardation6%2%19%23% Learning Disability33% 19%11% Emotional Disturbance36% 15%13% Speech/Language Imp.54%55%46%47% Multiple Disabilitiesn/a 42%65% Health Impairment30%42%28%19% Orthopedic Impairment57%42%30%35% Other Health Impairment41%44%27%21% Visual Impairmentn/a 34%32% Autism54%45%44%53% Traumatic Brain Injuryn/a 26%29% Deaf-Blindnessn/a Developmental Delayn/a Overall- Special Ed39% 26%22% Overall- General Ed71%74%62%

9 Percent of Special Education Students Proficient by State and Disability Disability State 1 (grades 4, 8) (n =1,389 ) State 2 (grades 4,8) (n = 4,048) MathELAMathELA Mental Retardation 6%2%19%23% Learning Disability 33% 19%11% Emotional Disturbance 36% 15%13% Speech/Language Imp. 54%55%46%47% Multiple Disabilities n/a 42%65% Health Impairment 30%42%28%19% Orthopedic Impairment 57%42%30%35% Other Health Impairment 41%44%27%21% Visual Impairment n/a 34%32% Autism 54%45%44%53% Traumatic Brain Injury n/a 26%29% Overall- Special Ed 39% 26%22% Overall- General Ed 71%74%62%

10 Notes from the detailed analyses Significant variability in percent of special education students scoring proficient by disability category Notable variability across the two states in percent proficient within the same disability category Why? –Is likely due to different definitions of proficient? –Is it due to different ways of classifying students into disability categories (note the percent of special education students in various disability categories across states)

11 Additional considerations How do we define/classify the students who should be eligible for the 2% flexibility? –What do these analyses tell us? –Preliminary work suggests that the lowest scoring 2% of students on the general assessment are not a stable group over time –Longitudinal analyses will help provide insight regarding this issue –Disability categories do not provide a useful means of determining eligibility for many reasons, including variability of performance and variability of classifications