1. TCAP-Alt PA K-2 Scoring Training Section 5: Practice Set (Andrew Pevensie), Modified and Homebound Rubrics, Cut Scores

2. Before We Start For this portion of the training, you will need the following documents: TCAP-Alt PA Scoring Guide 2012-2013 TCAP-Alt PA Scoring Rubrics (Regular, Modified, and Homebound) TCAP-Alt PA Content Standards and Alternate Performance Indicators Document (API document) TCAP-Alt PA Scoring-at-a-Glance Chart TCAP-Alt PA Scoring Notes TCAP-Alt PA Settings and Supports Chart TCAP-Alt PA Scoring Checklist If you do not have these documents, please pause the presentation until you have retrieved them.

3. At the end of training section 3, you should have completed the Practice Set. Let’s see how you did.

4. Practice Set: Andrew Pevensie Our practice set student is Andrew Pevensie, on page PS-C. We’ll be looking at the Reading/Language Arts section of his portfolio. He’s in grade 11, so we look at the APIs for grade level cluster 9-12.

5. Andrew – Set 1: Content First, Content: Do we have an evidence sheet and graph with matching dates? Yes, 10/15.

6. Check it off.

7. What is the content standard? Reading. Let’s put it on the checklist. R

8. Is there a valid API for that grade level and subject? Yes. R.5.4, with a description from the correct column of the API document.

9. Write R.5.4 on the checklist. R R.5.4

10. Is there an acceptable activity that is related to the API? It’s not about food or toileting, so let’s see if we have all three required components. Is it related to the API, making predictions? Yes. Can we see what Andrew is doing? Yes. Do we know enough about the materials used (or that none were needed) to know how he is doing the activity? Yes.

11. Check it off. R R.5.4

12. Does the graph have at least fifteen data points, large enough to see and centered in the box? Yes. Does it show progress (are there three days on which the student did two increments better than the lowest point with no more than five days in a row at the same level of success)? Yes. Lowest Point 1 2 3 15 dots

13. Check off the graph. That gives us a complete Content Standard Set, so we can circle it. R R.5.4

14. Andrew – Set 1: Choice Let’s look back at the evidence sheet, page PS-C, for Choice. Remember, choice is related to activity. Since we gave credit to the activity, we can look for a choice.

15. Is there a valid choice on the evidence sheet (type of choice indicated, two options offered, student’s choice marked or a note explaining that the student refused to choose)? Yes. The type of choice is “reward,” two options are given, and the student’s choice is indicated.

16. Does a choice of reward match the choice code given for that day on the graph? Yes, so we can give credit to this choice.

17. Write the code for “reward” on the checklist. R R.5.4 #

18. Since we were able to give credit to the choice on the evidence sheet, we can look for more types If eligible, are there additional occurrences of different types of choice on the graph? No, they’re all “reward,” so we can’t get anymore from here. All “reward”

19. Andrew – Set 1: Settings Now for Settings. Is instruction occurring in one or more inclusive settings? Yes, the graph shows Shop, which is an elective. It’s not a state- provided code, so the teacher has defined it.

20. Is Shop an appropriate setting for this API? Yes. Even though this activity (predicting events in The Three Musketeers) would not be done in Shop, the student wasn’t in Shop on the date of the evidence sheet. Shop would be an appropriate place in which to learn about making predictions.

21. Is there an appropriate signature title and code to verify that the setting is inclusive and indicate the setting to which the signature should be attached? Yes.

22. How many times can it count: multiple times (for a “Big 4” classroom) or once (for a specialty area or other inclusive setting)? Once. Let’s put it on the checklist. R R.5.4 # Sh

23. Andrew – Set 1: Supports Now for Supports. Remember, supports are related to activities. Since we gave credit to the activity, we can look for supports.

24. Since we gave credit to Shop as a setting, we know we had an inclusive setting with an appropriate signature, title, and code.

25. How many times does the Shop teacher count? He counts as often as Shop appears, so we can count him twice. R R.5.4 # Sh Sh Sh

26. Andrew – Set 1: Peer Interactions Now for Peer Interactions. Since we had a valid API, we can look for a peer interaction. Do we have a peer interaction on the evidence sheet or graph page? Yes, we have one on the evidence sheet. We hope it’s good, because there’s not one on the graph page to fall back on.

27. Can we tell what the student and peer are doing together that is related to the API? Yes. Is there a peer signature, an appropriate grade level, and a date that corresponds to a date on the graph? No. Missing signature and grade

28. Since we can only get one peer interaction per content standard, and we didn’t get this one, we put an X on the checklist. R R.5.4 # Sh Sh Sh X

29. We’ve gotten all we can from this evidence sheet/graph pair. Let’s move on to the next pair on pages PS-E and PS-F. This is still Andrew, so it’s still grade 11.

30. Andrew – Set 2: Content For Content, do we have an evidence sheet and graph with matching dates? Yes, 1/11.

31. Check it off. R R.5.4 # Sh Sh Sh X

32. What is the content standard? Writing. Put it on the checklist. Is it a different one? Yes. R W R.5.4 # Sh Sh Sh X

33. Is there a valid API for that grade level and subject? Yes, it’s W.1.17, with the correct description for Andrew’s grade level.

34. Here is it is, in the Content Standards and API document.

35. Write W.1.17 for the API. R W R.5.4 W.1.17 # Sh Sh Sh X

36. Is there an acceptable activity that is related to the API? There’s no food and no toileting here, so let’s see if we have all three components. Is it related to the API? Yes, making a dog training manual is a step toward writing about science. Can we tell what Andrew is doing? He’s making a manual. Do we know enough about the materials to know how he’s doing the activity? No. Is he writing on the computer? With paper and pencil? Is he using a pre-bound book, or binding the pages together himself? There’s no way to tell, so this activity is too vague to receive credit.

37. Put an X in the blank for this activity. R W R.5.4 W.1.17 # Sh Sh Sh X X

38. Is this an appropriate graph (meaning a line graph or a graph of dots)? Yes. Does the graph have at least fifteen data points? Yes. Does it show progress (at least three days on which the student did two increments better than the first occurrence of the lowest point, with no more than five days in a row at the same level of success)? Yes. Lowest Point 1 23 15 dots

39. Check off the graph, but because we don’t have a good activity, we don’t have a complete set, so we don’t circle it. R W R.5.4 W.1.17 # Sh Sh Sh X X

40. Andrew – Set 2: Choice Let’s look back at the evidence sheet, page PS- E, for Choice. What is Choice related to? Activity. Did we give credit to this one? No, so we can’t look for a choice here or on the graph. Normally, we would look for “evidence of choice,” but since we already have a valid choice, we don’t need the “evidence of choice” life preserver.

41. Andrew – Set 2: Settings Now, Settings: Is instruction occurring in one or more inclusive settings? We have SAC on the graph, but the code is not defined. It’s not one of the pre-defined codes provided by the state, and the teacher hasn’t defined it in the inclusion code section, so we can’t use it—even if it seems self-evident. SAC code not defined

42. Andrew – Set 2: Supports Now for Supports. Remember, supports are connected to activities. Since we could not give credit to the activity, t the most we could get would be “evidence of support.” Since we already have a valid support, we don’t need the “evidence of support” life preserver.

43. Andrew – Set 2: Peer Interactions Now let’s see what we have for Peer Interactions. Since we had a valid API, we can look for a peer interaction. Is there a peer interaction on either the evidence sheet or the graph page? Yes, there is one on the graph page. (On the evidence sheet, there is only a grade level, so we already know we can’t use that one.)

44. Can we tell what the student and peer are doing together that is related to the API? Yes. Is there a peer signature, an appropriate grade level, and a date that corresponds to a date on the graph? Yes.

45. That gives us a good peer interaction. Check it off on the graph. R W R.5.4 W.1.17 # Sh Sh Sh X X

46. We’ve gotten all we can from this evidence sheet/graph pair. Let’s move on to the last pair on pages PS-G and PS-H. This is still Andrew, so it’s still grade 11.

47. Andrew – Set 3: Content Let’s look at Content: Do we have an evidence sheet and graph with matching dates? Yes.

48. Check it off. R W R.5.4 W.1.17 # Sh Sh Sh X X

49. What is the content standard? Elements of Language. Write EL on the checklist. Is it a different one? Yes. R W EL R.5.4 W.1.17 # Sh Sh Sh X X

50. Is there a valid API for that grade level and subject? Yes, it’s EL.1.15, followed by a description from the correct column of the API document.

51. Write EL.1.15 for the API. R W EL R.5.4 W.1.17 EL.1.15 # Sh Sh Sh X X

52. Is there an acceptable activity that is related to the API? There’s no food and no toileting involved, so let’s see if we have all three required components. Is it related to the API? Yes. Writing sentences will help Andrew learn to write a paragraph. Can we tell what Andrew is doing? Yes, writing sentences about caring for rabbits. Do we know enough about what materials were used to know how he performed the activity? No, there are no materials. Did he write with a pencil on paper? Did he use a dry-erase marker on the whiteboard? Did he type on a communication device? Because these activities must be duplicatable, we must know, at the very least, either what he wrote with or what he wrote on, so we can tell how he wrote the sentences.

53. Put an X in the space for the activity. Remember, this also affects Choice and Supports. R W EL R.5.4 W.1.17 EL.1.15 # Sh Sh Sh X X X

54. Is this an appropriate graph (meaning a line graph or a graph of dots)? Yes. Does the graph have at least fifteen data points? No, there are only 14, so put 14 on the checklist. We don’t really need to worry about progress in this case, but do note that a graph with too few dots AND no progress would still be eligible for a score of 20 if there were no complete sets. 14 dots

55. Put a 14 in the space for the graph. Since we have a vague activity and only 14 graph points, we don’t have a complete set. Pu R W EL R.5.4 W.1.17 EL.1.15 # Sh Sh Sh X X X 14

56. Andrew – Set 3: Choice Let’s look back at the evidence sheet, page PS-G, for Choice. Since the activity did not receive credit, we can’t look for choices, either here or on the graph. We already have a creditable choice, so we don’t need the “evidence of choice” life preserver, but we would have it if we needed it. Note that we could not use the choice of a pen to clarify the activity. The activity description must be in the box designated for the activity.

57. Andrew – Set 3: Settings Now, Settings: Is instruction occurring in an inclusive setting? Yes, we have Shop and Small Animal Care, both electives, with defined codes for both. Is the setting appropriate for the API? Yes, in both cases. In Shop, Andrew might write a paragraph about using tools safely. In Small Animal Care, he might write about caring for a guinea pig.

58. Do we have an appropriate signature, title, and code to verify that the setting is inclusive and to indicate the setting to which the signature should be attached? Yes, for both.

59. How many times can it count: multiple times for a “Big 4” classroom) or once (for a specialty area or other inclusive setting)? Both are elective, so they each count once. Since we already have Shop, we can only use Small Animal Care. Pu R W EL R.5.4 W.1.17 EL.1.15 # Sh SAC Sh Sh X X X 14

60. Andrew – Set 3: Supports Now for Supports. Remember, supports are connected to activities. Since the activity on the evidence sheet did not receive credit, the most we can get is “evidence of support.” Since we already have a creditable support, we don’t need the “evidence of support” life preserver.

61. Andrew – Set 3: Peer Interactions Finally, we have Peer Interactions. Since we have a valid API, we can look for a peer interaction. Is there a peer interaction on the evidence sheet or graph page? Yes. There’s one on the evidence sheet, but not on the graph page.

62. Can we tell what the student and peer are doing together that is related to the API? Yes. Is there a peer signature, an appropriate grade level, and a date that corresponds to a date on the graph? Yes. Since it’s on the evidence sheet, the date is at the top of the page.

63. We have another good peer interaction, so let’s check it off. Pu R W EL R.5.4 W.1.17 EL.1.15 # Sh SAC Sh Sh X X X 14

64. Andrew: Final Scores Let’s compare our checklist to the rubric. For Content, ask: How many different Content Standard Sets do we have? We have one, for a score of 30. R W EL R.5.4 W.1.17 EL.1.15 X X 14 30

65. Next is Choice. We need three different types for the maximum score. Andrew has one, “reward,” for a score of 12. # 12

66. Now let’s look at Settings. Andrew has two, for a score of 6. Sh SAC 6

67. Next, Supports. Andrew has two, for a score of 8. Sh 8

68. Finally, let’s look at Peer Interactions. Andrew has two, for a score of 8. 8 X

69. Andrew’s total score is 64. Pu R W EL R.5.4 W.1.17 EL.1.15 # Sh SAC Sh Sh 30 12 6 8 8 X X X 14 Total Score: 64

70. How did you do? Remember, you can use your Scoring-at-a-Glance chart and your Scoring Notes to help you remember the rules and the scoring process. The charts on the last two pages will remind you what to do in special circumstances, such as missing dots on the graph or a missing date on the evidence.

71. Remember how, in Section 1, I told you we’d learn how to use the modified and homebound rubrics later? Now that you know how to score with the regular rubric, you’ll find the other two simple to use. We’ll start with the TCAP-Alt PA Modified Scoring Rubric.

72. This is the TCAP-Alt PA Modified Scoring Rubric

73. Notice that, for each dimension, the first three score points are the same. The rules for scoring them are also the same. It’s as if we cut off the right-hand side of the regular rubric so that the maximum scores have changed to: Content – 30 Choice – 12 Settings – 6 Supports – 6 Peer Interactions - 6

74. That means a portfolio scored with the modified rubric needs only one evidence sheet/graph pair for each content area/subject. We only need: Content – one complete content standard set Choice – one type of choice Settings – two inclusive settings Supports – one natural support Peer Interactions – one peer interaction

75. Next, let’s look at the TCAP-Alt PA Homebound Scoring Rubric.

76. This is the TCAP-Alt PA Homebound Scoring Rubric

77. Notice that Content and Choice have the same maximum points as the modified rubric, but there are no Settings, Supports, or Peer Interactions dimensions. It’s as if we cut off the bottom three dimensions from the modified rubric. The maximum scores are: Content – 30 Choice – 12 Settings, Supports, and Peer Interactions should all be scored NA, since they are not required.

78. Difference in Content Requirements Notice that homebound students need only 10 data points on the graph. The portfolios of students who attend special day schools can be scored with the homebound rubric, but they must have 15 data points.

79. Here are the cut scores for each performance level by rubric: RubricCutPerformance Level Regular81-100Advanced 51-80Proficient 26-50Basic 0-25Below Basic Modified49-60Advanced 31-48Proficient 16-30Basic 0-15Below Basic Homebound35-42Advanced 22-34Proficient 11-21Basic 0-10Below Basic

80. You are now ready to score live portfolios. Please use your checklist and refer often to your Scoring-at-a-Glance document and Scoring Notes. This concludes Section 4 of the TCAP-Alt PA K-2 Training. If you have questions, please contact Lori Nixon at the Tennessee Department of Education: lori.nixon@tn.gov. lori.nixon@tn.gov