Administering the ACCESS for ELLs® Speaking Test This training module addresses the organization and administration procedures specific to the ACCESS for ELLs® Speaking test component. Participants will begin to understand the task level expectations upon which student responses will be evaluated as they move through the different parts of the Speaking test. A more detailed treatment of the scoring procedure is addressed in the Scoring the Speaking Test training module. Emily Evans, Center for Applied Linguistics January 2007 New Jersey Department of Education Developed by the Center for Applied Linguistics

Training Objectives To understand the background and structure of the ACCESS for ELLs® Speaking test To be able to administer the ACCESS for ELLs® Speaking test To be able to reliably score the Speaking test

Background on the Speaking Test Addresses the performance indicators from the WIDA standards for Speaking. Assesses the type of speech a student would typically use in school and instructional contexts across each of the WIDA standards. Conducted in a one-on-one, question-answer interview. All questions are standardized and read from a script. Student responses to questions are assessed for proficiency using a scoring rubric. The full set of WIDA standards is available for download at: www.wida.us/resources/standards/wizard. The background details for the WIDA-ACCESS Placement Test speaking component are identical to these for ACCESS for ELLs®.

Format of the Speaking Test Speaking test consists of three scripted sections A warm-up in which the test administrator puts the student at ease The test questions A wind-down in which the test administrator leaves the student with a positive impression of his or her performance on the test. Test questions are grouped into thematic folders (identified as “parts” within the test). Test questions are presented in a structured sequence determined by their intended proficiency level. Test is “adaptive,” that is, questions are presented until the student reaches his or her performance ceiling. The warm-up, though scripted, can be modified to accord with the degree of familiarity the test administrator already has with the student. The wind-down should leave the student with a positive feeling about the test. The description of a “thematic folder” is given in the PowerPoint titled “Framework for the ACCESS for ELLs® Test. An adaptive test avoids the need to ask questions of a student that are almost certainly beyond his or her ability to respond to.

Pathway Through Speaking Test Warm-up Student Response Assign Score Test Questions Loop through questions until student reaches performance ceiling Further details about the logical pathway through the speaking test parts and tasks are presented later in this PowerPoint. Wind-down

Format of a Thematic Folder The speaking test includes three thematic folders, identified as “parts” within the test Part A Tests the SI standard at proficiency levels 1, 2, and 3 Part B Tests the LA & SS standards at proficiency levels 1, 2, 3, 4, and 5 Part C Tests the MA & SC standards at proficiency levels 1, 2, 3, 4, and 5 Kindergarten test includes only Part A and Part B Each thematic folder includes a set of tasks and each task a set of questions. Each task with its accompanying questions is aimed at eliciting speech at one particular proficiency level. Check the PowerPoint titled “Framework of the ACCESS for ELLs® Test” for the definition and examples of a ‘thematic folder.’

Example Speaking Test Thematic Folder What the student sees—Picture Cue What the test administrator sees—Script Part A Speaking tasks are referenced by the letter “T.” The number following the “T” indicates the proficiency level of the task; e.g. T1 is a tasked aimed at proficiency level 1.

Speaking Picture Cue Booklet Picture Cue booklets are printed two sided and bound in a flip chart format. The picture cue faces the student and the script faces the test administrator. The script also includes a thumbnail graphic of what the student is seeing. Student TA A different test administration procedure applies for the K speaking test. Refer to the PowerPoint titled “Administering the ACCESS for ELLs® Kindergarten Test” for details. TA Student Script Side Picture Cue Side

Reading the Test Script Sample of Script In the script, the test administrator reads aloud all text that is bolded. Text that is in the regular font (not bold) includes staging and navigation instructions to the test administrator. These instructions are not read aloud. Instructions introduced by “if” signal conditional questions, which are asked only if warranted by the student’s previous responses. Additional instructions regarding the script are contained in the lead pages to the scripts themselves.

Navigating the Test Part A: SI Part B: LA/SS Part C: MA/SC T1 T1 T1 T2 If score on task is ?, Meets or Exceeds, go to next task. If score on task is ?, Meets or Exceeds, go to next task. If score on task is ?, Meets or Exceeds, go to next task. T4 T4 At a minimum 3 tasks must be administered to each student: Part 1 T1, Part 2 T2, and Part 3 T3. At a maximum, all 13 tasks would be administered. Obviously the time needed for an individual test will vary with the number of tasks administered. T5 T5 or or or If score on task is Approaches or No Response, go to Task 1 of Folder B. If score on task is Approaches or No Response, go to Task 1 of Folder C. If score on task is Approaches or No Response, stop the Speaking Test.

The Speaking Rubric

Task Level Expectations Every task and question asked the student is based on a set of expectations for what the response will look like. Areas of speech around which scoring expectations are based Linguistic Complexity Expectations of the quantity and organization of the student’s verbal response Vocabulary Usage Expectations of the student’s use of appropriate vocabulary for grade level and proficiency level Language Control Expectations of the student’s control of English grammar, word choice in context, and the English sound system The issue of task expectations is critical to training on the scoring rubric. Emphasize that the number of the task, e.g., T1, T2, etc. determines the level of expectations of the student’s response in regard to linguistic complexity, vocabulary usage, and language control. In certain tasks, particularly for those at the higher proficiency levels, some of the questions that initiate the task are lead-in questions to the task that actually presents the expectations for the level. In these cases, you must be careful to assess the student’s response for the final question in the task, not the lead-ins.

Speaking Test Scoring Scale The Scoring Scale Speaking Test Scoring Scale Exceeds Exceeds task level expectations in quantity and/or quality Meets Meets all task level expectations in quantity and quality Approaches Approaches task level expectations, but falls short in quantity and/or quality No Response No response; response incomprehensible; student unable to understand directions “Meets” is highlighted on the scale to emphasize that a task is designed to elicit speech that will meet expectations of the proficiency level it targets. Detailed instructions on how to interpret the scoring scale are contained in the ACCESS for ELLs® District and School Test Administration Manual and in the PowerPoint module titled “Framework for the ACCESS for ELLs® Test”.

Scoring Rules A rating of Meets or Exceeds each receives a point value of 1. There are no extra points awarded a score of Exceeds. The Exceeds rating indicates a strong expectation that the student will be able to respond with at least a Meets rating to the following task in the test. A rating of Approaches or No Response each receives a point value of 0. The 0 point value reflects the fact that the student could not meet one or more of the requirements of the scoring rubric. The Exceeds score should set up a strong expectation that the student will succeed on the following task, though this will not always be the case.

Scoring Rules If in doubt between a score of Meets or Approaches, you can mark in the center column below the question mark (?) on the score sheet and administer the next task. If the student gives a performance that meets the task level expectations on the next task, assign that task a score of Meets, and go back and assign the task, previously scored a question mark, a score of Meets. If the student gives a performance that very clearly fails to meet the task level expectations on the next task, it is most likely that the performance was also deficient on the previous task. Assign the current task a score of Approaches or No Response as appropriate, and go back and assign the task in question a score of Approaches. The “?” mark on the scoring sheet indicates some lingering doubt that the student’s performance fully meets the requirements for the task level, even after any optional questions have been asked.

The Scoring Sheet The Speaking Test Scoring Sheet is found as the last page of the student test booklet. In some states that have a longer test window for the speaking test than for the other tests, the Speaking Test Scoring Sheet is a separate document. Test administrators must make a mark on the scoring sheet immediately after the student responds to the last question in a task. The mark represents the student’s performance on the complete task, not individual questions on the task. The K speaking test utilizes a different score reporting format. Refer to the PowerPoint titled “Administering the ACCESS for ELLs® Kindergarten Test” for details. In saying that the test administrator’s mark reflects an assessment of the complete task, we mean that the administrator sees evidence across all the questions that the student has met the requirements of each criterion on the scoring rubric. It’s important to understand that the student does not have to show evidence of meeting the criteria for each question asked. It is extremely important to closely study the examples in the PowerPoint titled “Scoring the ACCESS for ELLs® Speaking Test” (also included in the D2L online course and on the D2L CD) to reliably score the test.

The Scoring Sheet The scoring sheet should be filled out completely; that is, a score for each of the 13 task items must be marked. Any task not administered because the student has reached a ceiling level within a part should be marked as Not Administered. If a “?” is marked for a task and resolved with a Meets or Exceeds by administering the following task, it is not necessary to erase the mark. The illustrated score sheet applies only to the grade 1-12 tests. The Kindergarten test uses a different convention for recording scores. Consult the PowerPoint titled “Administering the ACCESS for ELLs® Kindergarten Test” for more details about K scoring.

Questions or Comments? For more information, please contact the WIDA Hotline: 1-866-276-7735 or www.wida.us/helpform World Class Instructional Design and Assessment, www.wida.us Center for Applied Linguistics, www.cal.org Metritech, Inc., www.metritech.com