2. C R E S S T / U C L A Issues and problems in classification of students with limited English proficiency Jamal Abedi UCLA Graduate School of Education & Information Studies Center for the Study of Evaluation National Center for Research on Evaluation, Standards, and Student Testing Paper presented at the 2002 Annual Meeting of the American Educational Research Association New Orleans, LA

3. C R E S S T / U C L A Classification of ELL Students There is a growing concern over the assessment and classification of language minority students. However, before developing a valid and reliable assessment system, a well-defined, objective definition of the term “LEP” or “ELL” must be obtained. Unfortunately, the criteria for identifying LEP students are not used uniformly across the nation. In several language background studies conducted at UCLA /CRESST (Abedi and Lord, 2001; Abedi, Lord, and Plummer, 1995; Abedi, Lord, and Hofstetter, 1997, Abedi, Hofstetter, Baker, and Lord, 1998), one of the major problems encountered was the lack of a commonly acceptable definition for limited English proficiency. 1

4. C R E S S T / U C L A 2 Classification of Students with Limited English Proficiency There are many different criteria by which a student can be classified as LEP. Among the most important of these criteria are being speaker of a language other than English and scoring low on the English proficiency tests. The first criterion, i.e., being a non-native English speaker, is defined in Los Angeles area schools based on the information from the Home Language Survey. The Home Language Survey For many schools in Los Angles area, the Home Language Survey is the only source of information used to determine the need for a student to be tested for English Proficiency. Recent dialog over the type of bilingual instruction causes reporting inaccurately for the purpose of assuring that their children be treated no differently from their Anglo classmates. Other concerns for the student whose parents may have citizenry issues have led to a more relaxed treatment of the home surveys than what the district would prefer. Questions have been raised about the accuracy of a survey completed by a parent who is illiterate or who has no familiarity with written English.

5. C R E S S T / U C L A Assessment of Students’ Language Proficiency in English Language proficiency and achievement tests in English are commonly used for identification and assessment of LEP students. According to Hopstock, Bucaro, Fleischman, Zehler, and Eu (1993) eighty- three percent of school districts use English language proficiency tests alone or with other techniques to decide if a student is LEP. The English proficiency tests used frequently for such purposes are the Bilingual Syntax Measure (BSM), the Idea Proficiency Test (IPT), the Language Assessment Battery (LAB), the Language Assessment Scales (LAS), the Maculaitis Assessment Program (MAC), and the Peabody Picture Vocabulary Test (PPVT). 3

6. C R E S S T / U C L A Achievement Tests Achievement tests in English are used by approximately 52 percent of school districts to help identify LEP students, assign them to school services,and reclassify them from LEP status. Commonly used achievement tests are: the California Achievement Test (CAT), Iowa Test of Basic Skills (ITBS), Metropolitan Achievement Test (MAT), Stanford Achievement Test (SAT), and the Comprehensive Test of Basic Skills (CTBS). Zehler, et al. (1994) did a comprehensive review of these language proficiency tests and found major differences in all the content areas in which the tests were compared. An even more serious criticism of these language proficiency and achievement tests is the problem of the validity and reliability of these tests for LEP populations and the exclusion of LEP students from the norming group for these tests. For example Abedi & Leon (1999) found that language factors may be an additional source of measurement error in the assessment of LEP which may reduce the reliability of the tests considerably. Abedi, Leon & Mirocha (2001) found that language factors in content-based assessment may seriously undermine the validity of the tests and may be considered a source of construct-irrelevant variance (Messick, 1994, p.14). 4

7. C R E S S T / U C L A 5 Data Sources Site 1. Site 1 is a large urban school district. ITBS test data were obtained. There were 36,065 students in the grade 3 (7,270 bilingual), In grade 6 there were 28,313 students (3,341 or 11.8% bilingual) and in grade 8, there were 25,406 students (2,306 or 9.1% were bilingual). Site 2. There were 414,169 students in the grade 2 population (125,109 or 30.2% were LEP), in grade 7 there were 349,581 students (73,993 or 21.2% LEP). In grade 9 there were 309,930 students (57,991 or18.7% LEP). Stanford 9 test data were obtained for all students in Grades 2 to 11 for the 1997-1998 academic year. Site 3. There were 12,919 students in the grade 10 population (431 or 3.3% LEP). In grade 11 there were 9,803 students in the population (339 or 3.5% LEP). Site 4. There were 13,810 students in the grade 3 (1,065 or 7.7% LEP). In grade 6 there were 12,998 students in the population (813 or 6.3% LEP), in grade 8 there were 12,400 students (807 or 6.5% LEP).

8. C R E S S T / U C L A Findings Relationship between language proficiency test scores and LEP classification. Since LEP classification is based on students’ level of language proficiency and because LAS is a measure of language proficiency, one would expect to find a perfect correlation between LAS scores and LEP levels (LEP versus non-LEP). The results of analyses indicated a weak relationship between language proficiency test scores and language classification codes (LEP categories). Table 1. Correlation between LAS rating and LEP classification for Site 4 Correlation G2 G3 G4 G5 G6 G7 G8 G9 G10 G11 G12 Pearson r.223.195.187.199.224.261.252.265.304.272.176 Sig (2-tailed).000.000.000.000.000.000.000.000.000.000.000 N 587 721 621 1002 803 938 796 1102 945 782 836 6

9. C R E S S T / U C L A 7

10. Table 4. Correlation coefficients between LEP classification code and ITBS subscales for Site 1 Grade Reading Math Concept Math Problem Math & Estimation Solving Computation Grade 3 Pearson r -.160 -.045 -.076.028 Sig (2-tailed).000.000.000.000 N 36,006 35,981 35,948 36,000 Grade 6 Pearson r -.256 -.154 -.180 -.081 Sig (2-tailed).000.000.000.000 N 28,272 28,273 28,250 28,261 Grade 8 Pearson r -.257 -.168 -.206 -.099 Sig (2-tailed).000.000.000.000 N 25,362 25,336 25,333 25,342 8

11. C R E S S T / U C L A Table 5. Correlation coefficients between LEP classification code and Stanford 9 subscales for Site 2 Grade Reading Language Science Math Spelling Social Sci Grade 3 Pearson r -.415 -.352 -.299 -.275 -.305 -.277 Sig (2-tailed).000.000.000.000.000.000 N 376,986 373,669 77,855 386,369 385,699 62,317 Grade 5 Pearson r -.443 -.370 -.339 -.329 -.358 -.319 Sig (2-tailed).000.000.000.000.000.000 N 358,720 366,523 81,951 370,435 370,689 73,975 Grade 7 Pearson r -.450 -.390 -.363 -.318 -.403 -.338 Sig (2-tailed).000.000.000.000.000.000 N 336,309 334,827 102,595 340,094 341,745 86,894 Grade 9 Pearson r -.416 -.346 -.318 -.287 -.346 -.298 Sig (2-tailed).000.000.000.000.000.000 N 293,667 293,320 297,057 298,558 86,366 295,022 Grade 11 Pearson r -.387 -.334 -.295 -.225 -.311 -.290 Sig (2-tailed).000.000.000.000.000.000 N 225,113 223,912 225,671 227,217 58,354 223,891 9

12. C R E S S T / U C L A Table 6. Correlation coefficients between LEP classification code and Stanford 9 subscales for Site 3 Grade Reading Science Math Grade 10 Pearson r -.131 -.088 -.029 Sig (2-tailed).000.000.003 N 11,158 10,231 10,301 Grade 11 Pearson r -.140 -.095.005 Sig (2-tailed).000.000.658 N 8,740 7,900 8,040 10

13. C R E S S T / U C L A 11 Table 7. Correlation coefficients between LEP classification code and Stanford 9 subscales for Site 4 Grade Reading Math Math Computation Application Grade 3 Pearson r -.178 -.067 -.120 Sig (2-tailed).000.000.000 N 14,050 14,282 14,208 Grade 6 Pearson r -.232 -.087 -.142 Sig (2-tailed).000.000.000 N 13,354 13,364 13,299 Grade 8 Pearson r -.228 -.088 -.125 Sig (2-tailed).000.000.000 N 12,484 12,579 12,337 Grade 10 Pearson r -.252 NA -.102 Sig (2-tailed).000.000 N 9,499 9,778

14. C R E S S T / U C L A 12 Findings and Conclusions  For an effective instruction and a valid and reliable assessment for English language learners, a well-defined, objective definition of the term “ELL” and “LEP” is needed.  The results of studies nationwide suggest, however, that such definition is not provided.  The results of our analyses on large-scale data did not show a strong relationship between LEP classification and students’ level of English proficiency.  Analyses on the distribution of some English language proficiency tests showed a negatively-skewed distribution suggesting that the English proficiency test items did not have enough discrimination power.  The results of analyses also indicated that for grades 3 through 5, low scoring students tend to remain classified as LEP in most of these districts. As grade level increases, however, the variation in agreement among districts also increases. There appears to be an increasing tendency to reclassify low scoring students as grade level increases.  The results of our analyses also suggested that there was not any single criterion that highly correlates with LEP classification code. This may be due to psychometric characteristics of the measures or due to issues on the validity of LEP classification or, most likely, a combination of both.  Thus, the use of multiple criteria is recommended in assessments particularly in high- stakes assessments (e.g., in LEP classification). However, technical issues in using multiple criteria must be considered.