2. Critical Issues Related to ELL Accommodations Designed for Content Area Assessments The University of Central Florida Cocoa Campus Jamal Abedi University of California, Davis July 7, 2011 1

3. What Do We Need to Know About Accommodations Before Using Them for ELLs? (Learning from years of research) 1.Effectiveness: How effective are accommodations in making assessment more accessible to ELL students? 2.Validity: How valid is the outcome of the accommodated assessment when compared to a non-accommodated assessment? 3.Differential Impact: How universally applicable are these accommodations to ELL students with different background characteristics? 4.Comparability: Can accommodated and non- accommodated assessment outcomes be aggregated? 5.Relevance: How appropriate are the accommodations used for these students? 6.Feasibility: How feasible are these accommodations in large-scale assessments? 2

4. 1. Effectiveness  Accommodations are effective if they make content-based assessments more accessible to ELLs  Accessibility of assessment focus on language and cultural accessibility  ELL students performance improves under accessible assessments 3

5. 2. Validity Accommodations should not alter the focal construct Accommodations are not intended for non-ELLs To test the validity of accommodated assessment, non-ELL students should be tested under the accommodated and non-accommodated testing conditions 4

6. The most important condition in the use of accommodations for ELLs is validity Research findings suggest that providing accommodations may increase ELL students’ performance, while also benefiting non-ELL students. Some forms of accommodation strategies, such as the use of a glossary with extra time, raised performance of both ELL and non-ELL students There is not enough research support for many of the accommodations currently being used in national and state assessments. The only way to make judgments about the efficiency and validity of these accommodations is to use them in experimentally controlled situations with both ELL and non-ELL students. 5

7. How the validity and effectiveness of an accommodation can be tested in an experimentally controlled field study? ELL Status/ Accommodation Accommodated Non-Accom ELL G1G2 Non-ELL G3G4 Effectiveness: Comparing G1 with G2 Validity: Comparing G3 with G4 6

8. Evidence Many states are aware of the validity concerns regarding accommodations that may alter the construct. Thus, accommodations are grouped into two distinct categories: (1) Accommodations and (2) Modifications.  Accommodations are changes in test format, directions, etc. that do not alter the construct.  Modifications, however, are changes that alter the construct. The main question, however, is whether the categorization of accommodations is based on credible research evidence. 7

9. 3. Differential Impact Differential impact is the aspect of accommodation that has received less attention than other aspects Background variables that impact decisions regarding accommodation include the following (ELL is a heterogeneous population: one size may not fit all):  Level of English proficiency  Level of proficiency in native language (native language testing)  Accommodation used in instruction  Language of instruction 8

10. 4. Comparability The outcomes of accommodated and non- accommodated assessments should be comparable The concept of comparability is directly linked with the concept of validity The outcomes of invalid accommodations can not be combined with non- accommodated outcomes Comparability has serious implications for scoring and reporting assessment outcomes 9

11. 5. Relevance  Accommodations that are created and used for students with disabilities (SWD) may not be relevant for ELLs  Backgrounds and needs of SWDs are different from those of ELLs  Accessibility features that are identified for SWDs are different from the accessibility features for ELLs  Cultural and linguistic accessibilities are the features for ELLs 10

12. 6. Feasibility  Accommodations that are effective and valid may not be useful if they can not be easily implemented  Feasibility of accommodations is of particular importance in large-scale assessments  For example, providing a dictionary to all student or testing student one-on-one in large scale assessments may not be feasible  Logistical issue may limit our ability to provide appropriate accommodations to ELLs 11

13. Conclusion  There is not enough research to support many of the accommodations that are currently used in national and state assessments.  The only way to make judgments about the efficiency and validity of these accommodations is to use them under an experiment in which both ELL and non-ELL students are randomly assigned to accommodated and non- accommodated testing condition.  The results of studies nationwide have provided support for a few accommodations used for ELL students. 12

14. Conclusion Examples of research-supported accommodations: Providing a customized dictionary is a viable alternative to providing traditional dictionaries. The linguistic modification of test items that reduce unnecessary linguistic burdens on students is among the accommodations that help ELL students without affecting the validity of assessments. Computer testing with added extra time and glossary was shown to be an effective and valid accommodation (we will elaborate on these accommodations in our next section) 13