1. Summer Scoring Training Smarter Balanced Mathematics Deborah J. Bryant September 18, 2015

2. Purposes and Objectives Learn how the summative assessments are scored Practice using standard handscoring tools and processes Experience methods of applying standardized criteria to the evaluation of student work Use scoring information to improve instruction in the classroom

3. How Do Educators Benefit from Scoring Training? Enhanced assessment literacy – Deeper understanding of Smarter Balanced claims and targets – Better understanding of handscoring procedures – Opportunity to take a closer look at rubrics – Opportunity to see examples of student work at a range of score points Instructional Improvement – Ideas for improving instruction and evaluation of student work in the classroom

4. Standard Handscoring Materials Provided by Smarter Balanced – Standards for Mathematical Practice/Targets from the Mathematics Summative Assessment Blueprint Ensure raters have the tools to apply the scoring criteria consistently Include the following: – PT sets: stimuli and prompts – Scoring rubrics and exemplars – Samples: responses with scores and annotations – Practice set: sample responses for you to score

5. Summative Scoring at ETS Training—Summative scoring training is similar to the summer scoring training. Certification—Prospective raters must take and pass a certification test on a set of prescored responses. Calibration—Before raters start scoring, they must take and pass a calibration set to demonstrate that they are ready to score accurately. Double Scoring—10% of responses are double scored to verify that raters are scoring consistently with the rubric. Backrating—Scoring Leaders “read behind” raters and score responses raters have scored to provide another check.

6. Summative Scoring (cont.) Scoring Leaders: – Provide feedback to raters for additional content support; help prevent score drift; serve to maintain scoring pace. – Mentor struggling raters and/or direct them to preselected sample sets for retraining.

7. Scoring Terms Stimulus: Short scenario with useful information needed to solve problems. Prompts: The questions to which students are responding, also called “items.” Rubric: A standardized approach to score student responses. Math rubrics are item-specific. Exemplar: An example of a correct answer. It is important to note that equivalent answers are always accepted unless specific notes in the rubric indicate that only a certain set of answers will be considered as correct. Samples: Student responses that are prescored and annotated. Annotations are often just as important and insightful as the rubric.

8. Mathematics Assessment Performance Task (PT): A task with six prompts based on a common stimulus or scenario; some items will be scored based on responses to other prompts in the same PT. Non-PT (CAT): A single task with a single prompt. Other: Some machine-scored prompts need human scoring when administered on paper.

9. Sample Annotations An annotation is the rationale given for why a response receives full, partial, or no credit. The annotations written for samples provide further insight into how student responses were scored. It is important to read the rubric and annotations carefully.

10. Scoring Dependent Items Item 1 Response Item 2 Response Item 3 Response Item 4 Response Item 5 Response Item 6 Response A Dependent Item is an item that depends on the response to a previous item(s) in the PT. We call these previous items Parent Items or Related Items. If the reasoning is correct on item 5, but there is a follow-through error from items 3 and/or 4, award full credit.

11. Scoring Dependent Items (cont.) How can the dependent item receive full credit if the parent item received partial or no credit? If the student uses the answers he or she gave to the parent item and correctly applies them to find a solution to the dependent item, he or she can earn full credit for the answer on the dependent item. It is also possible that while answering the dependent item, the student realized his or her errors in the parent item and may correctly answer the dependent item without relying on the answer to the parent item.

12. Assessment Claims and Targets for Mathematics Summative Assessment Assessment Claims are broad evidence-based statements about what students know and can do as demonstrated by their performance on the assessments. Assessment Claims 1. Concepts & Procedures-40% 2. Problem Solving 40%3. Communicating Reasoning-20% 4. Modeling Data Analysis Within each claim there are several assessment targets which really define the nature of the performance expectations.

13. SBAC Practice Test Scoring Guide Grade 4 Performance Task Sample A TRIP TO THE ZOO Anna and her family go to the zoo. The gift store prices are shown in the table. Gift Store Prices GiftPrice Stuffed panda bear$9 Zoo magnet$4 Pack of 4 pens$6 Photo frame$8

14. SBAC Practice Test Scoring Guide Grade 4 Performance Task Sample Continued

15. Read the rubric and the exemplar. – Think about the different types of responses that a student may provide to correctly answer the prompt. – Discuss with your group or partner what information the student will need to answer the prompt. For this item, a full-credit response (2 points) includes -stating correct gifts that Anna and Ray can buy based on the student’s response to item 217 and the restrictions in the stem AND -explaining why there is enough money for the gifts based on the student’s response to item 217. For example: “Anna and Ray can buy a stuffed panda bear and a pack of 4 pens which will cost $13. There is $25 dollars left to buy gifts. $13 dollars is less than $25 dollars.”

16. SBAC Practice Test Scoring Guide Grade 4 Performance Task Sample Continued For this item, a partial-credit response (1 point) includes: -stating correct gifts that Anna and Ray can buy based on student’s response to item 217 and the restrictions in the stem. For example: “They can buy a stuffed panda bear and a zoo magnet.” For this item, a no-credit response (0 points) includes none of the features of a full- or partial-credit response. For example: “They can buy 4 stuffed panda bears.” This item is not graded on spelling or grammar.

17. SBAC Practice Test Scoring Guide Grade 4 Performance Task Sample Continued - Individually read and score Practice Set 1. -Discuss the practice papers with your group, providing evidence from the rubric and samples when you assign your scores. Try to reach consensus on a single score for each student response. -They would share the score that SBAC assigned to the practice papers along with the reasons why. A TRIP TO THE ZOO A TRIP TO THE ZOO Your Score Group Consensus Score SBAC Score Comments Practice 1 Practice 2 Practice 3

18. Closing Activity Reflect on today’s session, and discuss with your table group: – Strategies you learned that are relevant to classroom instruction and assessment – Ideas you will take back to the classroom – Other thoughts about scoring training and professional development Share ideas with larger group!

19. Interested in Scoring?/Questions?