Designing Accessible Reading Assessments Examining Test Items for Differential Distractor Functioning Among Students with Learning Disabilities Kyndra Middleton The University of Iowa April 10, 2007

Designing Accessible Reading Assessments Purpose of the Study To examine whether different distractor choices functioned differentially for students with learning disabilities who did not receive an accommodation, students with learning disabilities who received a read-aloud accommodation, and students with learning disabilities who received an accommodation other than a read-aloud To help determine whether a test can be modified for students with learning disabilities by removing a distractor choice while maintaining adequate test validity and information

Designing Accessible Reading Assessments Instrument Used 4th grade English Language Arts assessment from a criterion-referenced statewide test –Operational test data –Reading (42 MC items) –Writing (33 MC items)

Designing Accessible Reading Assessments Sample Used 30,000 non-LD students sampled from 298,622 students 9,056 LD students who did not receive an accommodation 4,727 LD students who received an accommodation based on their IEP/504 plan 1,371 LD students who received an accommodation based on their IEP/504 plus a read aloud accommodation

Designing Accessible Reading Assessments Sample Used contd SubgroupFar Below Basic Below Basic BasicProficientAdvancedTotal Number of Students No Disability4%11%31%30%25%30,000 Learning Disability no accommodation 32%34%25%7%2%9,056 Learning Disability IEP/504 35%38%23%4%1%4,727 Learning Disability IEP/504 & read aloud 26%36%30%6%1%1,371 Percentage of Students at Each Proficiency Level

Designing Accessible Reading Assessments Sample Used contd Reference-Focal Comparisons Reference GroupFocal Group No Disability (Group 0) Learning Disability no accommodation (Group 20) No Disability (Group 0) Learning Disability IEP/504 (Group 21) No Disability (Group 0) Learning Disability IEP/504 & read aloud (Group 22) Learning Disability no accommodation (Group 20) Learning Disability IEP/504* (Group 21) Learning Disability no accommodation (Group 20) Learning Disability IEP/504 & read aloud (Group 22) Note: IEP = Individualized Education Plan * = comparison did not show DIF so was not included in the DDF analyses

Designing Accessible Reading Assessments Procedure Examine items that previously displayed DIF for DDF –DDF: when two groups that have been matched on ability have different probabilities of selecting a distractor Standardized Distractor Analysis (SDA) –Distinguishes between distractors –Identifies uniformly and nonuniformly biased distractors –An extension of standardized p-difference

Designing Accessible Reading Assessments Procedure Used contd Equation used to test for DDF: STD(i) = : negligible DDF : moderate DDF : large DDF

Designing Accessible Reading Assessments Results 70% of the items that displayed DIF also displayed DDF 100% of DDF occurred with a comparison between the read aloud and some other group 64% of the distractors that displayed DDF were in favor of the read aloud group

Designing Accessible Reading Assessments

Results contd Note: +: moderate DDF in favor of the focal group ++: large DDF in favor of the focal group -: moderate DDF in favor of the reference group *R: DIF in favor of the reference group *F: DIF in favor of the focal group Shaded box: Items that did not exhibit DIF

Designing Accessible Reading Assessments Results contd 17% that assessed reading standards showed DDF 9% that assessed writing standards showed DDF No observed pattern across content or cognitive area between groups

Designing Accessible Reading Assessments Results contd Item that displayed large DDF was the most difficult item that displayed DIF One item displayed DDF in each of the distractors (two favoring the read aloud group and one favoring the non-LD group) Item that displayed DDF in two of its distractors was a spelling item –Both were homophones –Additional difficulty caused by read aloud

Designing Accessible Reading Assessments Conclusions/ Future Research Measurement dissimilarity between read aloud group and other groups Exploratory study: More research needed to determine whether read aloud actually alters tests validity Matched on ability to provide more information at extremes