Presentation is loading. Please wait.

Presentation is loading. Please wait.

Center for Applied Linguistics Standards, Structure, & Field Testing of ACCESS for ELLs Jessica Motz Center for Applied Linguistics Washington, DC January.

Similar presentations


Presentation on theme: "Center for Applied Linguistics Standards, Structure, & Field Testing of ACCESS for ELLs Jessica Motz Center for Applied Linguistics Washington, DC January."— Presentation transcript:

1 Center for Applied Linguistics Standards, Structure, & Field Testing of ACCESS for ELLs Jessica Motz Center for Applied Linguistics Washington, DC January 2005 Illinois 28 th Annual Statewide Conference for Teachers Serving Linguistically and Culturally Diverse Students

2 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 2 Outline 1.Background on the WIDA Project 2.WIDA Standards for English Language Proficiency – language vs. content 3.Structure of the ACCESS for ELLs Test 4.Sample Items 5.Pilot Test Results 6.Field Testing Update 7.Training for ACCESS for ELLs 8.Other Issues/Questions/Discussion

3 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 3 Origins of the WIDA Consortium Ten states, representing approx. 275,000 ELLs 2002: U.S. Dept. of Education Enhancement Grant Competition 3 Original States: Wi sconsin D elaware A rkansas Early Additions: District of Columbia Rhode Island, Maine, New Hampshire, Vermont Later Additions: Illinois, Alabama

4 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 4 Multiple WIDA Components Language Proficiency English Language Proficiency Standards Large-Scale Assessment (ACCESS for ELLs) Classroom Instruction & Assessment Academic Content Alternate Assessment (Alternate ACCESS) Professional Development Standards Assessment Instruction Validation and Research

5 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 5 WIDA Partners State Leadership Tim Boals, Wisconsin DPI, Chair Steering Committee Lead Developer (Standards and Project PI) Margo Gottlieb, Illinois Resource Center Item Specification Development Fred Davidson, UIUC Test Development Language Testing Division, CAL Professional Development Lorraine Valdez-Pierce, George Mason University Technical Applications (Database, Desire2Learn) University of Wisconsin - Oshkosh

6 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 6 Centrality of the ELP Standards Classroom Assessment Framework Large-Scale Assessment Framework English Language Proficiency Standards & Performance Definitions Model Performance Indicators: Classroom Model Performance Indicators: Large-Scale

7 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 7 Overall Organization of Standards Frameworks for Classroom & Large-Scale Assessment (2) English Language Proficiency Standards (5) Language Domains (4) Grade Level Clusters (4) Language Proficiency Levels (5) Model PIs are the lowest level of expression of the standards Model Performance Indicators

8 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 8 The WIDA ELP Standards Standard 1SI English language learners communicate in English for social and instructional purposes in the school setting. Standard 2 LA English language learners communicate information, ideas and concepts necessary for academic success in the content area of Language Arts. Standard 3MA English language learners communicate information, ideas and concepts necessary for academic success in the content area of Math. Standard 4SC English language learners communicate information, ideas and concepts necessary for academic success in the content area of Science. Standard 5 SS English language learners communicate information, ideas and concepts necessary for academic success in the content area of Social Studies.

9 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 9 The Levels of English Language Proficiency ENTERING BEGINNING DEVELOPING EXPANDING Never LEP Formerly LEP BRIDGING

10 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 10 Criteria for Proficiency Level Definitions Comprehension and use of the technical language of the content areas Extent of language (text or discourse) control Development of phonological, syntactic, and semantic understanding or usage ENTERINGBEGINNINGDEVELOPINGEXPANDINGBRIDGING 12345

11 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 11 Large-Scale Standards: SC Reading

12 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 12 Large-Scale Standards: SC Reading Classify living organisms (such as birds and mammals) by using pictures or icons

13 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 13 Large-Scale Standards: SC Reading Interpret data presented in text and tables in scientific studies

14 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 14 Large-Scale Standards: SC Reading

15 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 15 Assessment Forms Non-secure form for initial screening (July 1) One for each grade level cluster - with items at all 5 proficiency levels Kindergarten form - individually administered Secure forms for annual testing Two (initially: 100 and 200) for each grade level cluster Tier A: Proficiency levels 1-3 Tier B: Proficiency levels 2-4 Tier C: Proficiency levels 3-5

16 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 16 Tier Alignment with Proficiency Levels ENTERINGBEGINNINGDEVELOPINGEXPANDINGBRIDGING Annual ACCESS for ELLs Tier A Tier B Tier C

17 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 17 WIDA Steering Committee Administration Decisions Listening (15%): minutes, machine scored Reading (35%): minutes, machine scored Writing (35%): Up to 1 hour, rater scored Speaking (15%): Up to 15 minutes, administrator scored

18 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 18 Structure of ACCESS for ELLs Grade Level and Tier K A (adaptive) A B C 100 (roll-out Spring 2005) 999 (used to produce screener) 200 (roll-out Spring 2006) Listening group admin, machine scored Reading group admin, machine scored Speaking individual admin, adaptive, TA scored Writing group admin, rater scored Domains Forms

19 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 19 Item Creation Process (Fall 2003 – present) Item specifications drafted Item writers assembled from nominated ESL teachers in consortium states Item writers trained using Blackboard distance- learning management software Item writers submitted items electronically Items reviewed by External Reviewers, also trained via Blackboard Items reviewed and revised internally & organized into themes…

20 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 20 Thematic Folders of items arranged onto pilot test forms Forms piloted in 5 WIDA districts Pilot analysis & feedback incorporated; items created, revised by ESL teachers Thematic Folders of items arranged onto field test forms Forms field tested in 8 WIDA states Field test analysis occurring presently For more information on item development process: Jim Baumans session Item Creation Process, continued… (Fall 2003 – present)

21 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 21 Pilot Testing Results

22 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 22 Pilot Test Participation April – May Districts: Kenosha & Milwaukee, WI; Chicago & Cicero, IL; Washington, DC Approx students in grades 1-12

23 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 23 Listening Test Multiple choice minutes

24 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 24 Sample Items: Listening, Science 1-2

25 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 25 Science Listening 1-2 P1 PI: identify objects according to chemical or physical properties from pictures and oral statements SCRIPT: A seed is small. Find the small seed.

26 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 26 Science Listening 1-2 P2 PI: match objects with their chemical or physical properties from pictures and oral statements SCRIPT: One day the seed will grow into something large, round, and heavy. Find what the seed grows into.

27 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 27 Science Listening 1-2 P3 PI: identify and group objects according to chemical or physical properties from oral statements SCRIPT: Seeds grow into plants. Find something else that grows.

28 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 28 Percent Correct on Each Item (Grades 1-2: n = 173) P1scoredP2scoredP3scored Mean

29 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 29 Percent Correct on Each Item (by Tier)

30 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 30 Percent Correct on Each Item (by Grade Level)

31 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 31 PLEVEL Mean ITEMDIFF GRADELEV g12 g35 g68 g912 Average Listening Item Difficulty Across Grade Clusters (by Proficiency Levels)

32 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 32 Average Reading Item Difficulty by Grade Level Cluster across P-Levels

33 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 33 Reading Test Multiple choice minutes

34 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 34 Examining Reading Items Across a Strand for the Different Tiers Language Arts, Reading, Grades 3-5

35 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 35 Sample Item (Field Test): Reading, Lang. Arts, Grades 3-5, Tier C

36 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 36 #11: To Jessica, he was the best dog in the world. This sentence shows Jessicas opinion. Which of the following also shows an opinion? PI (p3): Identify language associated with stating opinions found in fiction or non-fiction text. #12: When Jessica saw the sign for the lost dog, why did she believe it was Blue? PI (p4): Differentiate between statements of fact and opinion found in various reading selections. #13: Why does the woman say at the end of the story, Hes your dog, all right!? PI (p5): Identify authors reasons or intent for selecting facts or opinions found in fiction or non- fiction from grade-level language arts text. Items Tied to Performance Indicators: Tier C Items

37 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 37 Reading Items Adapted for Tier A Simpler text and more graphic support Items at proficiency levels 1, 2, and 3 For example: 1. Which is Blue? PI (p1): Match labels or identify facts from pictures and phrases. 2. I know he has white spots. Which words in this sentence tell you it is a fact? PI (p2): Identify language associated with stating facts found in short fiction or non-fiction text supported by pictures or graphics 3. [Same item from Tier C] To Jessica, he was the best dog in the world. This sentence shows Jessicas opinion. Which of the following also shows an opinion? PI (p3): Identify language associated with stating opinions found in fiction or non-fiction text.

38 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 38 Tier Alignment with Proficiency Levels on Test Forms (L, R, W) ENTERINGBEGINNINGDEVELOPINGEXPANDINGBRIDGING Annual ACCESS for ELLs Tier A Tier B Tier C

39 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 39 Writing Test Up to 1 hour 4 tasks per tiered form: 1.SI 2.MA 3.SC 4.LA/SS Model provided to give background and structure for the task

40 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 40 Lang. Arts Writing 6-8 P5 PI: defend positions or stances using original ideas with supporting details Sample Item: Writing Grades 6-8

41 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 41

42 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 42 Speaking Test: Adaptive Format

43 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 43 Sample Item: Speaking

44 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 44 Task (Proficiency Level) 1 Example: Grades 3-5 SI Speaking 3-5: P1 Respond to WH- questions First lets talk about things people do outside. This is a picture of people in a park. Im going to ask you some questions about this picture. Q1: (Point to TREE) What is this? Q2: (Point to BALL) What is this? Q3: (Point to DOG) What is this? Q4: (If necessary) What else do you see in this picture (OR) What other things do you see in this picture?

45 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 45 Task (Proficiency Level) 2 Example: SI Speaking 35: P2 Ask and respond to questions Now listen carefully. Ive just asked you some questions about this picture. Now I want you to ask me some questions about it. (OR) Pretend you are the teacher and want to ask me some questions about this picture. For example, you could ask me, Where are the people? OK? Q1: (Point to BOY ON BIKE) What do you want to know about him? (OR) Ask me a question about him. Q2: (Point to PICNIC TABLE) What do you want to know about this? (OR) Ask me a question about this. Q3: What other things do you want to know about his picture? (OR) Whats another question you can ask me about (anything in) this picture? (Answer students question.)

46 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 46 Task (Proficiency Level) 3 Example: SI Speaking 35: P3 Exchange personal information Now let me tell you something about these children. (Point to CHILDREN PLAYING CATCH) Their names are Alex and Leticia. They like to play catch. Q1: Do you like to play catch? Q2: (If Yes) What else do you like to do? Q3: (If No) What do you like to do? Q4: What do you like about __________? (OR) Tell me something about ___________. Q5: (If necessary) Tell me more.

47 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 47 Field Testing: Overview

48 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 48 Field Test Participation Request: Group-administered sections, 600 per form (minimum 400 per form) Individually-administered sections, 225 per form Includes 8 WIDA states Proportional representation of states (approx. 5.5%) Approx students total (~3500 from Illinois)

49 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 49 Initial Field Test Analyses Concurrent calibration of MC items (Rasch) Separate scale construction for each domain Raw score to scale score on the screener NOTE: Steering Committee determined the following weights when a single level designation is needed: Writing 35% (e.g 2) Reading 35% (e.g. 2) Listening 15% (e.g. 5) Speaking 15% (e.g. 4) Composite = 2.75 (not yet 3)

50 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 50 Sample Feedback received to date Input from test administrators and coordinators: Test Length: The test is taking longer to administer than anticipated. Test Difficulty: The test is more difficult than anticipated. Grades 1-2 Test: Too challenging for fall first graders; these students should take K test. Grades 9-12 Test: There is not enough authentic literature on the Reading Test. CAL and DPI responded to these concerns via e- mail and the D2L (online training) discussion board.

51 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 51 Timeline for Development and Implementation Field Testing (incl. English speakers): September - January 2005 Analysis of Field Test Data: November January 2005 Setting Cut Scores: January 2005 Spring Roll-Out of Form 100 (AL, ME, VT): February - May 2005 Practice Items for Fall Roll-out Available: March 15, 2005 Screener Available: July 2005 Fall Roll-Out of Form 100 (IL, DC): Fall 2005

52 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 52 Training for ACCESS for ELLs

53 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 53 Training for ACCESS August 2004: SEA WebEx conference on logistics and data needed for Field Test September 2004: LEA Coordinators trained online via Desire2Learn at UW – Oshkosh September – November 2004: Field Test Administrators trained online via D2L December 2004 – January 2005: Speaking Field Test Administrators trained in-person (DC, WI) March – May 2005: Operational Test Administrators giving spring roll-out test trained

54 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 54 WIDA Field Test Administrator Workshop

55 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 55 Operational ACCESS Test Administrator Training: Possible Scenarios Beginning March hour total online training for all sections of the test Additional, as-needed training on scoring the Speaking test, with online or live (state/district) trained facilitators. For more information and updates:

56 IRC Conference Jan 24-27, 2005 Standards, Structure & Field Testing of ACCESS for ELLs Jessica Motz 56 Other Issues/Questions/Discussion


Download ppt "Center for Applied Linguistics Standards, Structure, & Field Testing of ACCESS for ELLs Jessica Motz Center for Applied Linguistics Washington, DC January."

Similar presentations


Ads by Google