1. N C E O National Center on Educational Outcomes Accommodation Decisions: Policy, Training, and Monitoring as Critical Aspects of an Objective Approach Martha L. Thurlow National Center on Educational Outcomes www.nceo.info CCSSO National Conference on Student Assessment Orlando, FL June 17, 2008

2. N C E O National Center on Educational Outcomes The challenges in selecting assessment accommodations are many – they include: Use of evidence to judge the effectiveness and validity of accommodations Appropriate policy setting, with strong rationales about the validity of accommodations – when experimental studies have not been conducted or cannot be conducted Training of decision makers Monitoring of what actually happens to ensure that there is validity

3. N C E O National Center on Educational Outcomes Summaries of Published Evidence Summaries available since 1999 (with spotty reviews before that) Tindal & Fuchs summary in 1999 NCEO summaries of 1999-2001; 2002-2004; 2005-2006 Sireci, Scarpati, & Li summary of 2005 ( in Review of Educational Research ) The evidence basis for accommodations is very important – yet, does not answer all the policy questions.

4. N C E O National Center on Educational Outcomes YearsNumber of Studies Average Per Year 1990 through 1992113.7 1993 through 1995186.0 1996 through 1998299.7 1999 through 20014615.3 2002 through 20044916.3 2005 through 20063216.0 Number of Accommodations Studies

5. N C E O National Center on Educational Outcomes Research Methods Number of Studies Method 2002-2004* 2005-2006** Experimental or Quasi- experimental 21 (43%) 18 (56%) Review of extant data 17 (35%)6 (19%) Survey/Interview 7 (14%)5 (16%) Meta-analysis 2 (4%)0 (0%) IEP intervention 1 (2%)0 (0%) Product evaluation 1 (2%)0 (0%) Other (i.e., case studies, observation) 0 (0%)3 (9%) * From Johnstone, Altman, Thurlow, & Thompson (2006). ** From Zenisky & Sireci (2007).

6. N C E O National Center on Educational Outcomes What is Studied? Extended time, oral administration are most often studies (followed by computer administration) Studying multiple accommodations at one time has fallen dramatically The most frequently allowed accommodations (large print, individual, small group, magnification, braille) are studied less often; accommodations used by small numbers of students are rarely studied

7. N C E O National Center on Educational Outcomes Policy Setting Requires More Than Research Evidence Research evidence may focus on different standards from those of the state, or be based on different selection criteria for students using the accommodation States may find that conducting research is very difficult to do – given all the complexities that arise when doing this research

8. N C E O National Center on Educational Outcomes Steps States Need to Take Examine data on use of accommodations Develop good policies, documenting the rationale for each accommodation Provide guidance for human suppliers of accommodations Provide training of teachers and decision makers Monitor on test days, and examine relation to decisions and instruction

9. N C E O National Center on Educational Outcomes Examine Data Multiple Ways By category of disability By grade or school level By type of accommodation or specific accommodations of interest By other factors that may be important  Ethnicity  Socioeconomic status

10. N C E O National Center on Educational Outcomes From NCEO analysis of APR data – Thurlow et al., 2008

11. N C E O National Center on Educational Outcomes From NCEO analysis of APR data – Thurlow et al., 2008

12. N C E O National Center on Educational Outcomes Policy Setting Think about the need for changes each year. States with Read Aloud Question Changes in Policy – comparing 2003 to 2005 and 2007 Number of States No changes since 200319 One change since 200319 Two changes since 200312 From NCEO Data Viewer at http://data.nceo.info

13. N C E O National Center on Educational Outcomes Constrain the Number of Students Using Certain Accommodations Massachusetts Texas Examples from States

14. N C E O National Center on Educational Outcomes Test Administrator Reads Aloud ELA Comprehension Test: The test administrator reads the ELA Reading Comprehension test to a student. Note: Reading aloud the ELA Composition writing prompt, or the Mathematics, Science and Technology/Engineering, and/or History and Social Science tests is standard accommodation 16. The decision to use nonstandard accommodation 26 must be made separately from the decision to use standard accommodation 16. 1.The student has a specific disability that severely limits or prevents him or her from decoding text, or from comprehending decoded text, even after varied and repeated attempts to teach the student to do so. The student must be a virtual non-reader, not simply reading below grade level. 2.The student has access to printed materials only through a reader, and/or is provided with spoken text on audiotape, CD, video, or other electronic format during routine instruction, except while the student is actually being taught to decode. 3.The accommodation is documented in the student’s IEP or 504 plan, after the team has considered the conditions listed above and in Section C of this Chapter in determining whether the student is eligible for this accommodation. Special Instructions: a)All passages and test items must be read word-for-word, exactly as written. The test administrator may not provide assistance to the student regarding the meanings of words, intent of any test item, or responses to test items. The test administrator must read in a neutral tone, without emphasis on any terms, passages, or response options, and with no detectable changes in inflection. The test can be read aloud in one of the following ways, which must be specified in the IEP or 504 plan:  Test administrator reads entire test session (passages and test items) word-for-word exactly as written. OR  Test administrator reads selected words, phrases, or sentences as directed by the student. The student points to the word, phrase, or sentence that he or she needs read aloud. a)If the test is read aloud, it must be administered in a separate setting, either individually or to a small group (2-5 students). When reading aloud to a small group of students, follow the procedures outlined in the appendix. b)Reading aloud the ELA Reading Comprehension test to a student is nonstandard accommodation 26. Reading aloud other MCAS tests to a student is standard accommodation 16. Massachusetts – Read Aloud

15. N C E O National Center on Educational Outcomes Eligibility for Dyslexia Bundled Accommodations A student who meets the following criteria is eligible to receive the three bundled accommodations on English or Spanish TAKS, including TAKS (Accommodated), reading tests at grades 3–6, or English TAKS, including TAKS (Accommodated), reading tests at grades 7 and 8. A student not receiving special education services must be identified with dyslexia. A student receiving special education services must either be identified with dyslexia or have a severe reading disability that exhibits the characteristics of dyslexia, causing the student to lack word- identification skills and to have difficulty reading words in isolation. The student must routinely receive accommodations in classroom instruction and testing that address the difficulties he or she has reading words in isolation. Authority for Decision For a student with dyslexia not receiving special education services who meets both criteria above, the decision to provide the bundled accommodations must be made either by the student’s placement committee as required by Section 504 of the Rehabilitation Act of 1973 or by the committee of knowledgeable persons as outlined in The Dyslexia Handbook. In both of these cases, the committee’s decision must be documented in writing in accordance with district policies and procedures. Texas – Bundled Accommodations

16. N C E O National Center on Educational Outcomes Number of States Figure 2. States with Written Guidelines for Access Assistants in 2003 and 2005

17. N C E O National Center on Educational Outcomes Monitoring Accommodations Do decisions made by the IEP team get carried out on the day of testing? Do accommodations used during testing reflect – to the extent appropriate – accommodations that are used during instruction? Do accommodations get recorded properly on assessment forms? Etc!