Michigan Assessment Consortium Common Assessment Development Series Module 21 Assessment Administration and Scoring MAC CAD-PD Mod-6 BRF 20090821-0913 [comp].ppt 1

Narrated By: Ellen Vorenkamp Wayne RESA 2

Previous Modules Previously, you have Edited your test items Reviewed them content, bias and other factors Selected items for field testing Field tested the items Reviewed and revised the items after field testing Selected materials for your final test(s)

In This Module The next steps are to prepare the final tests in a format suitable for actual use These are assessments will be administered to students “for real” Then, the assessments will need to be scored, both SR and CR items Finally, reports of results will be prepared

Packaging the Items This activity is very similar to putting together field test forms, which was described in some detail in Module 16 Refer to this module for information on the steps in determining the length of the final tests, the number of items in each, and how the items would be packaged for the final tests

Packaging the Items The key difference are: the length of the final test versus a field test the number of standards selected for assessment, and the number of items per standard to be used A longer test means it is even more important that care is taken to lay out the test, all questions are keyed correctly

Printing the Test Materials This step is necessary if the test is to be administered on paper. Several steps may be involved: Finalize item wording for each item Desktop publish or re-type items using a consistent typeface and size Determine the most cost efficient way to print the test booklets

Printing the Test Materials Steps (continued): Consider how students will respond – in their booklets or on a separate answer document? Are any ancillary materials needed for this test – manipulatives, photographs, etc. These may need to be reproduced Make sure the needed materials are prepared before they are needed

Online Assessment For online assessments, determine the process for getting the items into the system Prepare final test copy, then proof it If a file upload can be used, make sure that the copy was uploaded accurately If the items are entered directly into the system, proof it to make sure the items ware entered appropriately

Assessment Administration Make sure that the directions for the correct administration are created Who is to be assessed? How will the assessment be administered? When will assessment occur? How will students’ responses be scored? Are there any special test directions? How and where should responses be returned for scoring?

SR Assessment Scoring As part of the preparation for the actual administration, a determination should be made about mode of assessment – paper-based or online For online assessments, describe for teachers in their directions which system will be used, as well as how to help students access the test online

SR Assessment Scoring For paper-based assessments, determine in advance where students will record their answers and how the responses will be scored: In their test booklets: hand-scored On a separate sheet: hand-scored On an electronic answer sheet: scanned Determine who will scan these sheets

CR Assessment Scoring The scoring of constructed-response items can present more of a challenge Ideally, the teachers who developed the common assessments will work together to score them Other teachers could be recruited for this as well This work is excellent PD

CR Assessment Scoring It is important for the scorers to be trained in advance of actual scoring The process of training usually start with the scorers trying the item(s) themselves to become familiar with what was asked of students Then, scorers are shown the scoring rubric and discuss each level of it

CR Assessment Scoring Next, samples of student work at each level of the scoring rubric are shown and discussed Developers should have pre-selected the best examples of each level of the rubric from field testing Try to reach agreement on the elements needed to get each score

CR Assessment Scoring Then, give each scorer some pre-scored samples (but which do not have the scores indicated on them) How much agreement was reached on each? If most give the samples the same scores, they are ready to start scoring If high levels of agreement are not reached, repeat the step above

CR Assessment Scoring It may be important to double-score responses if the results will have high stakes for students (or educators) Double scoring will assure that all scorers continue to use the same standards Even for lower stakes uses, double scoring at least a small sample of papers (e.g., 10% to 20%) will assure that

CR Assessment Scoring Continue to score all of the responses from students for one item before repeating this process for additional CR items Try to conduct the scoring in the shortest time period possible, so that scorers (teachers) don’t forget the rubric or how to score the students’ responses

Preparing Score Reports Once the scoring is completed, it is time to summarize the data and to prepare score reports that are needed by different users: Students and Parents Teachers School Administrators Public

Preparing Score Reports The online system or the electronic scoring system you used may prepare these (and other) score reports for you. If not, think about ways that you can report results that various users would benefit from More information on this is provided in a later module

Summary In this module, you learned: How to prepare the assessments to be administered Different ways the tests could be given How SR and CR items could be scored The analysis of the assessment results Preparation of score reports

Series Developers Kathy Dewsbury White, Ingham ISD Bruce Fay, Wayne RESA Jim Gullen, Oakland Schools Julie McDaniel, Oakland Schools Edward Roeber, MSU Ellen Vorenkamp, Wayne RESA Kim Young, Ionia County ISD/MDE

Development Support for the Assessment Series The MAC Common Assessment Development Series is funded in part by the Michigan Association of Intermediate School Administrators In cooperation with Michigan Department of Education Ingham and Ionia ISDs, Oakland Schools, and Wayne RESA Michigan State University