1. ELL-Language-based Accommodations for Content Area Assessments The University of Central Florida Cocoa Campus Jamal Abedi University of California, Davis July 7, 2011 1

2. ELL Language-Based Accommodations  English dictionary  English glossary  Bilingual dictionary/glossary  Customized Dictionary  Native language testing  Read-aloud test items or directions  Linguistically modified test  Computer testing with pop-up glossaries 2

3. English Dictionary  Providing an English Dictionary is another commonly used accommodation for ELL students (Abedi, Courtney, & Leon, 2003; Abedi, Lord, Boscardin & Miyoshi, 2000).  The use of a dictionary and extra time affect the performance of all students (Abedi, Lord, Hofstetter & Baker, 2000; Hafner, 2001; Maihoff, 2002; Thurlow, 2001; Thurlow & Liu, 2001).  By gaining access to definition of content-related terms, recipients of a dictionary may be advantaged over those who did not have access to the dictionaries. This may compromise the validity of assessment (Abedi, Courtney, Mirocha, Leon & Goldberg, 2005).  The dictionary as a form of accommodation suffers from yet another major limitation, the feasibility issue (Abedi, Courtney, Mirocha, Leon & Goldberg, 2001).  Consequently, the results of accommodated and non- accommodated assessment may not be aggregated. 3

4. English Glossary  English glossary with extra time raised performance of both ELL and non-ELL students (Abedi, Hofstetter & Lord, 2000).  ELL students’ performance increased by 13% when they were tested under the glossary with extra time accommodation.  While this looks promising, it does not present the entire picture.  Non-ELL students also benefited from this accommodation, with an increase of 16% (Abedi, Hofstetter & Lord1998, 2000).  Thus, the results of the accommodated outcome cannot be aggregated with the non-accommodated outcome. 4

5. Customized Dictionary  Customized dictionary was introduced as a more valid alternative to English/bilingual dictionaries (Abedi, Courtney, Mirocha, Leon & Goldberg, 2001).  It is a cut-and-paste of the actual dictionaries.  In only includes terms that are: (1) in the test and (2) non- content related.  Results of studies suggest that it is highly effective and valid accommodation for ELL students. 5

6. Linguistically Modified Test  There are, however, some accommodations that help ELL students with their English language needs without compromising the validity of assessment.  Studies suggested that the linguistically modified version of the tests items is an effective and valid accommodation for ELL students (Abedi, Hofstetter, Lord & Baker, 2000; Maihoff, 2002; Rivera & Stansfield, 2001).  This accommodation also helped students with learning disabilities.  Thus, an accommodation may have the potential to be effective and help provide valid assessment outcomes for ELL students. 6

7. Computer Testing  Research findings suggest computer testing as an effective and valid accommodation for ELL students (Abedi, Courtney, Mirocha, Leon, & Goldberg, 2001).  ELL students show higher levels of motivation on the assessments administered by computer.  Different types of accommodations that have been shown to be useful for ELL students may be incorporated into the computer testing system (Abedi, et al, 2011). 7

8. Native Language Testing  Translating tests into students’ native language is an accommodation used by many states across the country (Abedi, Lord, Hofstetter & Baker, 2000; Rivera, Stansfield, Scialdone & Sharkey, 2000).  Issues concerning translation and content coverage across the forms must be seriously considered.  Students’ background variables, particularly their level of proficiency in L1 and L2, must be studied before considering this accommodation.  Using native language assessment may not produce desirable results if the language of instruction and assessment are not aligned (Abedi, Hofstetter & Lord, 2004). 8

9. Examining Complex Linguistic Features in Content-Based Test Items Unnecessary complex linguistic features slow down the reader, make misinterpretation more likely, and add to the reader’s cognitive load; thus interfering with concurrent tasks. These features include: Concrete vs. abstract or impersonal presentations Item length Unfamiliar Vocabulary Nominal heaviness Relative clause Conditional clause Passive voice Long noun phrases Subordinate clauses 9

10. Impact of language factors on content assessments for ELLs Continuum of Linguistic Complexity and Item Characteristics 1 Items with no linguistic complexity: Familiar or frequently used words; word length generally shorter Short sentences and limited prepositional phrases Concrete item(s) and a narrative structure No complex conditional or adverbial clauses No passive voice or abstract or impersonal presentations 10

11. Impact of language factors on content assessments for ELLs Continuum of Linguistic Complexity and Item Characteristics 2 Items with a minimal level of linguistic complexity: Familiar or frequently used words; short word length Moderate sentence length with a few prepositional phrases Concrete item(s) No subordinate, conditional, or adverbial clauses No passive voice or abstract or impersonal presentations 11

12. Impact of language factors on content assessments for ELLs Continuum of Linguistic Complexity and Item Characteristics 3 Items with a moderate level of linguistic complexity: Unfamiliar or seldom used words Long sentence (s) Abstract concept (s) Complex sentence/conditional tense/adverbial clause(s) A few passive voice or abstract or impersonal presentations 12

13. Impact of language factors on content assessments for ELLs Continuum of Linguistic Complexity and Item Characteristics 4 Items with a high level of linguistic complexity: Relatively unfamiliar or seldom used words Long or complex sentence(s) Abstract concept(s) Difficult subordinate, conditional, or adverbial clause(s) Passive voice/ abstract or impersonal presentations 13

14. Impact of language factors on content assessments for ELLs Continuum of Linguistic Complexity and Item Characteristics 5 Items with a maximum level of linguistic complexity: Highly unfamiliar or seldom used words Very Long or complex sentence(s) Abstract concept(s) Very difficult subordinate, conditional, or adverbial clause(s) Many passive voice/ abstract or impersonal presentations 14

15. Sample Original and Revised Item Below is an example of a test items that deemed to be linguistically complex and a linguistically modified version of the items.Below is an example of a test items that deemed to be linguistically complex and a linguistically modified version of the items. Original: If Y represents the number of newspapers that Lee delivers each day, which of the following represents the total number of newspapers that Lee delivers in 5 days?Original: If Y represents the number of newspapers that Lee delivers each day, which of the following represents the total number of newspapers that Lee delivers in 5 days? –A) 5 + Y –B) 5 x Y –C) Y + 5 –D) (Y + Y) x 5 Modified: Lee delivers Y newspapers each day. How many newspapers does he deliver in 5 days?Modified: Lee delivers Y newspapers each day. How many newspapers does he deliver in 5 days? (Adopted from Abedi, Lord & Plummer, 1997, p. 21) 15

16. Linguistic Modifications made on the item Conditional clause changed to separate sentence Two relative clauses removed and recast Long nominals shortened Question phrase changed from “which of the following represents“ to “how many” Item length changed from 26 to 13 words Average sentence length changed from 26 to 6.5 words Number of clauses changed from 4 to 2 Average number of clauses per sentence changed from 4 to 1 16

17. Conclusions and Recommendation Assessments and instructions for ELLs : Must be based on a sound psychometric principles Must be controlled for all sources of nuisance or confounding variables Must be free of unnecessary linguistic complexities Must include sufficient number of ELLs and SWDs in its development process (field testing, standard setting, etc.) Must be free of biases, such as cultural biases Must be sensitive to students’ linguistics and cultural needs 17

18. Conclusions and Recommendations Accommodations: Must be relevant in addressing assessment issues for ELL students Must be effective in making assessments more accessible to ELL students Should not alter the construct being measured Must provide results that can be aggregated with the assessment outcomes under standard conditions Must be feasible in national and state assessments 18

19. Conclusions and Recommendations Examples of research-supported accommodations: Providing a customized dictionary is a viable alternative to providing traditional dictionaries. The linguistic modification of test items that reduce unnecessary linguistic burdens on students is among the accommodations that help ELL students without affecting the validity of assessments. Computer testing with added extra time and glossary was shown to be a very effective, yet valid accommodation (Abedi, Courtney, Leon and Goldberg, 2003) 19