English Language Development Assessment (ELDA) Background to ELDA for Test Coordinator and Administrator Training Mike Fast, AIR CCSSO/LEP-SCASS March 1, 2004 Background to ELDA for Test Coordinator and Administrator Training Mike Fast, AIR CCSSO/LEP-SCASS March 1, 2004

Overview 1. Project Organizations 2. Some Key Facts 3. Federal Regulations (NCLB) 4. Off-the-Shelf Assessments 5. Summary of ELDA 6. General Features of ELDA 7. Listening 8. Speaking 9. Reading 10. Writing

1.Council of Chief State School Officers (CCSSO) 2.Limited English Proficient State Collaborative on Assessment and Student Standards (LEP-SCASS) 3.State of Nevada Department of Education Project management Project steering committee Lead SCASS state 1. Project Organizations (1 of 2)

4.American Institutes for Research (AIR) 5.Center for Study of Assessment Validity and Evaluation (C-SAVE) University of Maryland 6.Measurement Inc. (MI) Test developers Test validity and reliability research Field test operations 1. Project Organizations (2 of 2)

K-12 LEP enrollments:10% (national average) Alaska: 30% California: 25% New Mexico: 24% From 1986 – 1997: 125% increase (nationally) Projected increase for : similar or even greater 2. Some Key Facts

Provide an annual assessment of English proficiency and show annual progress in acquisition Include all LEP students in English language proficiency assessments Obtain independent measures of reading, writing, listening, speaking Report a measure for comprehension Measure social + academic skills Align assessments with standards Provide an annual assessment of English proficiency and show annual progress in acquisition Include all LEP students in English language proficiency assessments Obtain independent measures of reading, writing, listening, speaking Report a measure for comprehension Measure social + academic skills Align assessments with standards 3. Federal Regulations (NCLB)

Do NOT measure progress (but focus on placement) Are NOT aligned to standards Do NOT measure academic skills Top out at a level somewhat below full English proficiency 4. Off-the-Shelf Assessments

ClusterListeningSpeakingReadingWriting K-2Under development Note: 2 = number of forms of assessment being field-tested in Spring Summary of ELDA

ELDA is aligned to state ESL standards ELDA has vertical alignment across clusters ELDA has 5 levels of performance standards, with rigorous definition of FEP at Level 5 and realistic definition for beginners at Level 1 ELDA is a test of 4 language skills, not of academic content - no content area prior knowledge requirements ELDA is aligned to state ESL standards ELDA has vertical alignment across clusters ELDA has 5 levels of performance standards, with rigorous definition of FEP at Level 5 and realistic definition for beginners at Level 1 ELDA is a test of 4 language skills, not of academic content - no content area prior knowledge requirements 6. General Features of ELDA (1 of 3)

ELDA domains each designed around 4 topic areas (3 academic and 1 social): Math, Science, Technology English Language Arts Social Sciences School Social ELDA domains each designed around 4 topic areas (3 academic and 1 social): Math, Science, Technology English Language Arts Social Sciences School Social 6. General Features of ELDA (2 of 3)

Reading and Listening: Multiple choice items only Writing: Constructed response and multiple choice items Speaking: Constructed response items only Spring 2004 field test: 1 operational form Spring 2005 operational test: 2 operational forms 6. General Features of ELDA (3 of 3)

Final form 50 MC items 60 MC items Field test additions for Spring MC items + 15 MC items 7. ELDA - Listening (1 of 4) DESIGN

STANDARDS Measure ability of student to: Determine main idea Identify supporting ideas Comprehend spoken instructions Comprehend key vocabulary/phrases Draw inferences, predictions, conclusions Determine speaker’s attitude, perspective STANDARDS Measure ability of student to: Determine main idea Identify supporting ideas Comprehend spoken instructions Comprehend key vocabulary/phrases Draw inferences, predictions, conclusions Determine speaker’s attitude, perspective 7. ELDA - Listening (2 of 4)

TEXT TYPES Short phrases (1 utterance, 1 test item) Short dialogues (2 exchanges, 1 test item) Extended dialogues (multiple exchanges, 2 test items) Short presentations (30 seconds, 2 test items) Extended presentations (only at 6-8, 9-12) (1 minute) (4 test items) TEXT TYPES Short phrases (1 utterance, 1 test item) Short dialogues (2 exchanges, 1 test item) Extended dialogues (multiple exchanges, 2 test items) Short presentations (30 seconds, 2 test items) Extended presentations (only at 6-8, 9-12) (1 minute) (4 test items) 7. ELDA - Listening (3 of 4)

OTHER FEATURES Natural sounding language All test items are multiple choice Test items do not require knowledge external to the stimulus Complete test contained on tape/CD Complete test except stimuli contained in test booklet Stimulus is heard twice Some graphic support OTHER FEATURES Natural sounding language All test items are multiple choice Test items do not require knowledge external to the stimulus Complete test contained on tape/CD Complete test except stimuli contained in test booklet Stimulus is heard twice Some graphic support 7. ELDA - Listening (4 of 4)

MSTELASSSchool Social Connect1 task Tell1 task Expand1 task Reason1 task 8. ELDA - Speaking (1 of 4) DESIGN

8. ELDA - Speaking (2 of 4) STANDARDS Measure ability of student to: Connect Tell Expand Reason STANDARDS Measure ability of student to: Connect Tell Expand Reason

OTHER FEATURES 16 speaking tasks per cluster Structure of each task: input, prompt, scaffold, repeat prompt Graphic support to provide motivation for student to structure response Complete test contained on tape/CD Test booklet contains graphics only Student responses are taped OTHER FEATURES 16 speaking tasks per cluster Structure of each task: input, prompt, scaffold, repeat prompt Graphic support to provide motivation for student to structure response Complete test contained on tape/CD Test booklet contains graphics only Student responses are taped 8. ELDA - Speaking (3 of 4)

Scoring scale: 0-2 Scoring rubric focuses on:  Appropriateness  Rhetorical features  Correctness  Quality and quantity Scoring scale: 0-2 Scoring rubric focuses on:  Appropriateness  Rhetorical features  Correctness  Quality and quantity 8. ELDA - Speaking (4 of 4) CONSTRUCTED RESPONSE SCORING & RUBRICS

Final form 50 MC items 60 MC items Field test additions for Spring MC items + 15 MC items 9. ELDA - Reading (1 of 4) DESIGN

9. ELDA - Reading (2 of 4) STANDARDS Measure ability of student to: Comprehend simple and short sentences/texts Follow written instructions Determine main idea Identify supporting ideas Determine meaning of key vocabulary Draw inferences, predictions, conclusions Identify writer attitude/point of view Analyze style/form STANDARDS Measure ability of student to: Comprehend simple and short sentences/texts Follow written instructions Determine main idea Identify supporting ideas Determine meaning of key vocabulary Draw inferences, predictions, conclusions Identify writer attitude/point of view Analyze style/form

9. ELDA - Reading (3 of 4) TEXT TYPES Early reading (1 or 2 sentence passages) Reading instructions (short passages from textbooks/teacher handouts) Passage comprehension (5 passages,  5-6 test items per passage; each passage  words) TEXT TYPES Early reading (1 or 2 sentence passages) Reading instructions (short passages from textbooks/teacher handouts) Passage comprehension (5 passages,  5-6 test items per passage; each passage  words)

OTHER FEATURES All test items are multiple choice Early reading passages require filling in of gaps or responding to comprehension questions Multiple choice options for instructions items are mainly graphic For longer comprehension passages, 5-6 comprehension test items Some graphic support to aid comprehension OTHER FEATURES All test items are multiple choice Early reading passages require filling in of gaps or responding to comprehension questions Multiple choice options for instructions items are mainly graphic For longer comprehension passages, 5-6 comprehension test items Some graphic support to aid comprehension 9. ELDA - Reading (4 of 4)

Final form 3 SCRs 1 ECR 15 MCs 3 SCRs 1 ECR 15 MCs 4 SCRs 1 ECR 15 MCs Field test additions for Spring SCRs 1 ECR 5 MCs + 2 SCRs 1 ECR 5 MCs + 2 SCRs 1 ECR 5 MCs 10. ELDA – Writing (1 of 5) DESIGN

STANDARDS Planning and organizing Revising and editing Writing STANDARDS Planning and organizing Revising and editing Writing 10. ELDA - Writing (2 of 5)

TEXT TYPES Narrative Descriptive Expository Persuasive TEXT TYPES Narrative Descriptive Expository Persuasive 10. ELDA - Writing (3 of 5)

OTHER FEATURES Constructed Response Items Scoring rubrics on scale of 0-3 (SCRs) and 0- 4 (ECRs) SCR rubrics focus on comprehensibility ECR rubrics focus on comprehensibility and discourse features OTHER FEATURES Constructed Response Items Scoring rubrics on scale of 0-3 (SCRs) and 0- 4 (ECRs) SCR rubrics focus on comprehensibility ECR rubrics focus on comprehensibility and discourse features 10. ELDA - Writing (4 of 5)

Revising & Editing Short stimulus that simulates student writing Three questions per stimulus – Errors in text – Adding topic or concluding sentences Planning & Organizing Outline or graphic organizer as stimulus Three questions per stimulus Multiple Choice Items 10. ELDA – Writing (5 of 5)