Presentation on theme: "Administering the ACCESS for ELLs® Listening, Reading, and Writing Tests In this training module participants will receive a comprehensive orientation."— Presentation transcript:
1 Administering the ACCESS for ELLs® Listening, Reading, and Writing Tests In this training module participants will receive a comprehensive orientation to the organization of the ACCESS for ELLs® test booklet and administration procedures for the Listening, Reading, and Writing components. Participants will walk through test administration logistics by discussing sample group test session schedules. They will also be introduced to the materials, such as test administrator scripts and practice test items for each of the components from different theme folders in the three tiers (A, B, and C). In viewing sample test items, participants will begin to understand the tasks students encounter on the ACCESS for ELLs® test.Emily Evans, Center for Applied Linguistics January 2007 New Jersey Department of EducationDeveloped by the Center for Applied Linguistics
2 Training ObjectivesTo understand the organization of the ACCESS for ELLs® test bookletTo understand administration requirements of the Listening, Reading and Writing components of the ACCESS for ELLs®
3 Outline General Information Test Administration Overview Listening TestReading TestWriting Test
4 General InformationAll test sessions must take place within a state’s testing window.NJ’s testing window is April 16th – June 1st, 2007Tests are secure and you must follow state test security regulations.The Test Administration Manual contains guidance on test accommodations for students with disabilities.The Test Administration (TA) Manual contains detailed information on Test Administration. This slide contains a few general points. The list of acceptable accommodations for students with disabilities can be found in the TA Manual.Your school must administer the test within your state’s testing window.Security is very important so make sure to follow those regulations carefully. Specific information about security can be found in the Test Security Training presentation.
5 Test Administration Overview Listening, Reading and WritingAdministered in groups of up to 20 studentsCentrally scored by MetriTech, not by the Test AdministratorEach grade level cluster and each tier must have separate group sessionsThe administrator’s scripts arrive with the test bookletsScripts are different for each test formListening, Reading and Writing tests have scripted instructions in the Test Administrators Script which parallels the test booklet. Make sure to follow the scripts exactly.When scheduling test sessions, be sure to schedule separate sessions for each test form. For example, 35A is a different test than 35B. The scripts and the test forms will be different, so you must administer them in separate sessions. Do not mix tiers or grade levels in a testing session.Scripts are specific to the test form so they are not the same for every Tier (Tiers refer to A B or C)Note: There is a separate PowerPoint presentation explaining the administration procedures for the Speaking portion of the ACCESS for ELLs® test.
6 Test Booklet Organization Test booklet order: 1) Listening 2) Reading 3) WritingEach test will begin with practice items or a modelTests are organized by theme folders – A series of questions about one topic with graphic and/or text supportEach theme folder for Listening and Reading contains 3-5 items (test questions)Each theme folder for Writing contains 1-4 items per theme folder in Tier A for all grade levels and form 1-2B. The theme folders for the rest of the forms contain one item per theme folder.There are practice questions or a model at the beginning of each test. Questions in the Listening and Reading sections are organized into theme folders which contain sets of 3-5 questions on one topic/theme with graphic and/or text support.On the Writing test, more questions are given for Tier A for all grade levels and form 1-2B to provide extra support for the writing.
7 Test Administration Times Listening and Reading are administered together in one group sessionListening Administration: minutesBreak in between Listening and Reading sections: 5 minutesReading Administration: minutesLogistics: 15 minutes (approximately)(including break in between sections)Total: 75 minutes (approximately)Writing is administered in a separate group sessionWriting Administration: 60 minutes15-20 students per group sessionThe listening and reading sections are administered in one session. Together they will take approximately 75 minutes when logistical considerations are factored in such as taking attendance or distributing and collecting test materials.Use your own judgment with the break in between the Listening and Reading sections.Note also that the length of the oral passages gets longer with increasing proficiency; thus, the Tier C Listening Test takes longer to administer than the Tier A Listening Test.The writing section is administered in a separate group session and will also take approximately 75 minutes when logistical considerations are factored in.If you think it is appropriate or necessary to schedule both sessions listening/reading and writing on the same day, make sure that an adequate break is provided in between sessions. Generally schools administer the L/R and Writing tests on separate days. An example of how to schedule a group of 90 students is given on the next slide.
8 Scheduling Guidelines Example Assume there are 90 students in the same grade level cluster with 30 students per Tier tested 15 at a timeYou can test all students in both Listening/Reading and Writing in a total of 12 sessions over 4 daysThis example assumes there is a Single Test Administrator in the schoolNumber of test takers per time blockThis slide shows an example of how test sessions could be scheduled for a group of 90 students that has 30 students per tier.This is just an example to give you an idea of how scheduling might take place. There are many other options depending on how many students and administrators your school has.The testing schedule is determined by the School Testing Coordinator. In this example, Tier A, B and C are each split in half and tested on separate days.The first group of 45 students (15 from Tier A, 15 from Tier B, and 15 from Tier C) are tested on Monday and Tuesday. The second group of 45 students are tested on Wednesday and Thursday.Again, two tiers (ex tier A and tier B) can not be tested at the same time. Similarly 35A and 68A can not be tested in the same session. The scripts are different for each tier and for each grade level.The example on this slide assumes there is a single Test Administrator in the school, so there would be no simultaneous sessions. If the school had multiple test administrators, simultaneous sessions (in different rooms) could be scheduled depending on the specific situation.
9 How to read the Script Read all of the text in Bold aloud example: “Part A: Classroom Objects”The text which is not bolded serves as a guideline and should not be read aloudexample: “Pause”, “… make sure all students are in the right place.”This page is an excerpt from the beginning of the Test Administrator’s Script that you will receive.
10 How to read the Script (con’t) The test item is in Blue and should be read aloud
11 Listening Test Overview Format: Multiple choiceTime: minutesScoring: Machine scored (by MetriTech)Weight:30% of comprehension score15% of composite score6-7 thematic folders with at least one from each standard (Math, Science, Language Arts, Social Studies, Social Instructional)Test items read aloud only one timeThe listening test includes questions from all English Language Proficiency (ELP) standards: Math (MA), Science (SC), Language Arts (LA), Social Studies (SS), and Social and Instructional Language (SI).Only listening and reading factor into the Comprehension score. Listening is worth 30% of the comprehension score and Reading is worth 70%. The composite score includes all domains (Listening, Reading, Writing and Speaking). More information on the weights of each test can be found in the WIDA Background PowerPoint presentation.You cannot read the answer choices out loud.Under normal circumstances, you shall read each item only once. The exception is if there is an interruption while you are reading an item, such as an announcement over the loudspeaker, a fire drill, or a loud noise outside an open window. In that case, the item may be repeated one time only, for a total of two complete readings of the item.
12 Listening Test Overview (con’t) Keep the test going at a steady paceFollow pause times given in the scriptAfter an item is read, give students up to 25 seconds to mark their answersFollow the Test Administration Script exactlyGive students up to 25 seconds to mark their answers or until the last student is finished, whichever comes first. Do not pause longer that 25 seconds.The D2L course has sound files where you can listen to sample items being read. This will help you understand the pacing of the test better.
13 Listening Test: Introduction The beginning of the listening test starts with a scripted introduction.Read the bolded text, but not the plain text
14 Listening Test: Practice Problems Occur at the beginning of the test, after the introduction, and are scriptedMust be administeredHelp students understand the structure of the test
15 Listening Test: Practice Problems This is an example of how a practice problem is structured.
16 Listening Test: Practice Problems Script: First the boy and girl play catch, and then they ride their bikes.Find the set of pictures that shows this in the right order. PAUSE.This is the first question from the practice section with the picture the student sees.
17 Listening Item Example: Science 1-2 A What you see on the screen is an example of a theme folder. This theme folder has three questions.Notice there is a theme graphic at the beginning of the folder of a classroom with growing plants which sets the context for the three questions. This grounds the question, but may not be necessary to answer all of the questions. In the example on this slide all of the response options are graphics. The test specifications dictate whether the options are text and/or graphics. In some cases, of course, the response options will contain text instead of or in addition to graphics.This folder is an example from Tier A, so it covers proficiency levels 1, 2, and 3 (P1, P2, P3).
18 Listening Example Levels 1 and 2 - Scripts Script: A seed is small. Find the small seed.Script: One day the seed will grow into something large, round, and heavy. Find what the seed grows into.These are two questions from the previous theme folder. The script is listed above the question so you can see how the script relates to what the student is seeing. On the actual test, students would not see the script. These questions are at a proficiency level 1 and 2 for grades 1-2.
19 Reading Test Overview Format: Multiple choice Time: 35-40 minutes Scoring: Machine scored (by MetriTech)Weight:70% of comprehension score35% of composite score6-7 thematic folders with at least one from each standardFollow the Test Administration Script exactlyThis is the reading test overview. The reading test should be given following the listening test.The reading test may begin with some practice problems which are scripted. During the reading test you should follow the script.Answer choices may not be read out loud.The reading test includes questions from all English Language Proficiency (ELP) standards: Math (MA), Science (SC), Language Arts (LA), Social Studies (SS), and Social and Instructional Language (SI).
20 Reading Test Scripting Requirements Begins with an Introduction
21 Reading Practice Problems Practice problems follow the introduction and must be administered.In some cases, the correct answer to a practice problem will be filled in.
22 Sample Items Within a Reading Test Only on forms 1-2A and 1-2B1-2A occur after every folder and are scripted (the answer is filled in together as a group)1-2B occur after some folders and are not scriptedPurpose: Extra assistance for the studentThe exception: The listening test has one folder that has to do with patterns which includes a sample item. Otherwise, sample items are only on form 1-2A and 1-2B reading.On the 1-2A test sample problems occur after every folder as a group check-in and they are mandatory. The whole class fills in the correct answer together and the question is scripted.
23 Individual and Group Check-ins Intended to help guide students though parts of the test and are indicated with a stop signWhole-group check-ins – Check in with all of the students at the same time and explain the next part of the test. Occurs after every section in the 1-2A Reading Test.Individual check-ins – Students raise their hand when they arrive at a stop sign and you check to see the student has completed the section. Invite them to respond to any skipped questions.Halfway through 1-2B, 1-2C, 3-5A, 3-5B, 6-8A, 9-12AAt the end of every tier and grade levelThe reading section has check-ins. Whole group check-ins only occur on the 1-2A Reading test.As you walk around the room and monitor the students’ work, check to make sure that each question has been answered. Do not check whether responses are correct or not.If any responses are missing, point them out to the students and invite them to try again.When students have finished, check their work. Again, if any responses are missing, point them out to the students and invite them to try again.
24 Reading Item Example: Math 6-8 B This is an example of a graphics and text stimulus for a Theme Folder entitled Multicultural Club. Each Theme Folder will have graphic and/or text support. This is for grade level cluster 6-8 Tier B in the math content area.
25 Reading Item Example: Math 6-8 B These are two examples of questions based on the previous graphics and text stimulus for the Theme Folder.
26 Writing Test Overview Format: Student constructed responses Time: Up to 1 hourA break can be given halfway through the test if necessaryScoring: Rater scored (by MetriTech)Weight: 35% of composite scoreThe following standards are covered:Social Instructional (SI), Math (MA), Science (SC)On Tier B and C: Integrated Task (IT) - combines Language Arts (LA), Social Studies (SS), and Social Instructional (SI)Writing tasks are modeled to scaffold the studentFollow the Test Administration Script exactlyIn this section students produce a sample of their writing. Students are given prompts and sometimes the writing the student needs to do is modeled.Tier A only has three tasks SI, MA, and SC. Tiers B and C have an additional Integrated Task (IT)The writing section is 35% of the overall composite score but does not factor into the comprehension composite score.The writing section is administered in a separate group session and will also take approximately 75 minutes when logistical considerations are factored in.
27 Writing test organization Tier A3 folders covering Social Instructional (SI), Math (MA), Science (SC) standards. About 15 minutes for each section.*Form 1-2A only has SI standards and there are 4 folders.Tier B and C3 folders (about 10 minutes each) covering SI, MA and SC standards.*Form 1-2B doesn’t cover the SC standard.Fourth folder is an IT standard combining LA, SS, and SI standards (student is given about 30 minutes).A folder may contain several questions, but they will be on the same topic. This is why it is called a folder.
28 Individual and Group Check-ins Whole-group check-ins – used to guide students through the model in Tier A for all grade levels and in all Tiers for grades 1-2Individual check-ins – Students raise their hand when they arrive at a stop sign which occurs at the end of every writing question. Check to make sure that they have completed the writing task.After 10 minutes, 20 minutes, and 30 minutes, circulate in the room to monitor students’ progress.If necessary, you can prompt students who are lagging behind by saying, “Make sure you save enough time for the other parts.”
29 Writing Item Example: IT 6-8 C This is the theme folder for this integrated writing task for 6-8 tier C. Remember an IT task combines Language Arts (LA), Social Studies (SS), and Social Instructional (SI) standards.The student is given background text and graphics to stimulate writing in the LA, SS and SI standards. Part 1 gives some questions for the student to think about, and Part 2 models the structure of the writing and helps the student plan their composition. Part 3 reiterates the structure of the composition and gives the writing assignment.In the test development process, the writing questions are designed to elicit responses in line with the writing rubric.
30 Writing Performance Description Writing RubricLevelWriting Performance Description6ReachingA variety of sentence lengths of varying linguistic complexity in a single tightly organized paragraph or in well-organized extended text; tight cohesion and organizationConsistent use of just the right word in just the right place; precise vocabulary usage in general, specific, or technical languageHas reached comparability to that of English proficient peers functioning at the “proficient” level in state-wide assessments5BridgingA variety of sentence lengths of varying linguistic complexity in a single organized paragraph or in extended text; cohesion and organizationUsage of technical language related to the content area; evident facility with needed vocabularyApproaching comparability to that of English proficient peers; errors don’t impede comprehensibility4ExpandingA variety of sentence lengths of varying linguistic complexity; emerging cohesion used to provide detail and clarityUsage of specific and some technical language related to the content area; lack of needed vocabulary may be occasionally evidentGenerally comprehensible at all times, errors don’t impede the overall meaning; such errors may reflect first language interferenceThis is the rubric used for scoring writing items.*** The descriptions above pertain to the original text the student has produced, as opposed to the copied or adapted text, unless indicated.Generally, of the three features, give the first feature more weight than the other two because it is a more comprehensive characteristic of writing than the other two.Do not penalize the mechanical errors such as spelling as long as the comprehension is not impeded.Usually, more than two of the three features should fall into the description of one level. If the three components are spread over more than three consecutive levels, choose the middle level at your discretion.
31 Writing Rubric (con’t) LevelWriting Performance Description3DevelopingSimple and expanded sentences that show emerging complexity used to provide detailUsage of general and some specific language related to the content area; lack of needed vocabulary may be evidentGenerally comprehensible when writing in sentences; comprehensibility may from time to time be impeded by errors when attempting to produce more complex text2BeginningPhrases and short sentences; varying amount of text may be copied or adapted; some attempt at organization may be evidencedUsage of general language related to the content area; lack of vocabulary may be evidentGenerally comprehensible when text is adapted from model or source text, or when original text is limited to simple text; comprehensibility may be often impeded by errors1EnteringSingle words, set phrases, or chunks of simple language; varying amounts of text may be copied or adapted; adapted text contains original languageUsage of highest frequency vocabulary from school setting and content areasGenerally comprehensible when text is copied or adapted from model or source text; comprehensibility may be significantly impeded in original text*** The descriptions above pertain to the original text the student has produced, as opposed to the copied or adapted text, unless indicated.Generally, of the three features, give the first feature more weight than the other two because it is a more comprehensive characteristic of writing than the other two.Do not penalize the mechanical errors such as spelling as long as the comprehension is not impeded.Usually, more than two of the three features should fall into the description of one level. If the three components are spread over more than three consecutive levels, choose the middle level at your discretion.
32 Scoring an IT Writing item Rater scored by MetriTech using a rubricThe preparing and planning sections are not scored.Planning sections not scored by MetriTechMake sure the students are spending enough time on the actual composition because that is the only thing scored by MetriTech.
33 Example of Modeled Writing Item Math6-8Tier ALook at the sentence in the black box.Point to it in your test booklet. Scan the room and make sure all students are in the right place.It says, "For a math assignment, Andrei needs to find the area of a triangle.”Look at the diagram. The triangle is labeled with the Base and the Height. The formula for the area of a triangle is given as Area equals Base times Height times one half.Point to picture of triangle and formula in the book as you read it.Look at the next black box. It says “Andrei answered two questions about the math assignment.”What do the first question and answer say?Allow time for student response, or say:It says, “What is the name of the shape in the picture? Triangle.”What do the second question and answer say?It says, “What do you need to do to find the area of the shape? I need to multiply the base times the height times one half.”The script is on the right hand side of the slide to give you an idea of what it will look like.
34 Example of Modeled Writing Item (con’t) Math6-8Tier ALook at the last black box. It says, “The pictures show the steps to follow to find the area. Andrei explained each step in writing.”What did Andrei write for step 1?Allow time for student response, or say: First, I have to measure the length of the base. The base is six inches long.We can’t read everything Andrei wrote for step 2, only the beginning. What do you think he wrote for step 2?Allow time for student response, or say:Maybe he wrote something like, “Then I have to measure the height of the triangle. The height is four inches.”Step 3 shows us the formula again, and the math Andrei did to find the area of the triangle. On the lines we see the beginning of his answer. How did he begin?He began, “Now I have to use the formula to find the area of the triangle. I multiplied…”Go to the top of the next page. It says, “Now it’s your turn to write!” Point to this in your test booklet. Scan the room and make sure all students are in the right place.
35 Example of Writing Item Task Now look at the directions in the black box. They say, “Now you have to find the area of the rectangle.”Find number 1 and look at the directions. You need to answer the questions in the box. You can use Andrei’s writing as a model for your own. You can begin in a moment, but first find number 2. The pictures show you how to find the area of a rectangle. On the lines you need to write at least 1 sentence to explain each step in the process. Again, remember that you can use Andrei’s writing as a model for your own.Before you begin, are there any questions?Answer any questions.Now you can begin writing on this page. You can have up to 15 minutes to do this part. Remember to stop when you get to the stop sign.Monitor the students’ work. When students have finished, remind them to check their work.If a student finishes early and wants to go back to complete Part A or Part B, he or she may do so.This part is over when all the students have finished or 15 minutes have passed, whichever comes first.(If 15 minutes have passed and some students are still working, say: If you are not finished, take a few more seconds to finish up this part. Pause 15 seconds.)This is the question that follows the model on the previous slide.The ending of the test has the following script: “End the testing session by saying: This is the end of the Writing test. Please close your test booklets and I will come around and collect them. “
36 When the Test Administration is Complete Collect test materialsThis happens after the Listening/Reading sections AND the Writing sectionFollow test security policiesAfter the test is collected you must follow the test security policies for submitting the materials. More information on test security can be found in the Test Security Training PowerPoint presentation.
37 QuestionsorComments?For more information, please contact the WIDA Hotline: orWorld Class Instructional Design and Assessment,Center for Applied Linguistics,Metritech, Inc.,