1. Raising the Bar: Debunking Myths About Scoring and Data Charts Dan Wiener Administrator of Inclusive Assessment
Massachusetts Department of Elementary and Secondary Education

2. “Playing It Safe” to Get the Highest Score

3. What Does This Data Chart Tell You About the Student’s Achievement?
Student mastered after 2-3 attempts: probably was too easy.
Time better spent teaching more challenging skills.

4. Scoring Portfolios – Quick Review
Level of Complexity (LOC)
Unmatched Access Skills Entry Points Grade Level/Competency
To Standard
NOTE: Most portfolio strands are LOC= (Entry Points)
Massachusetts Department of Elementary and Secondary Education

5. Scoring Accuracy and Independence
Demonstration of Skills and Concepts (DSC) (Percent Accuracy)
Independence (IND) (Percent unassisted responses)

6. How is the overall score determined from the LOC, DSC, and IND scores?
The scores in LOC, DSC, and IND are combined into a “strand score,” based on the “final 1/3 time frame.”
Strand scores are averaged together to give an overall score in a content area.
Scoring is explained in Educator’s Manual (Pp )
This table is used to calculate strand scores for LOC=3
(Also see Educator’s Manual for MCAS-Alt)
Massachusetts Department of Elementary and Secondary Education

7. A score of 100% accuracy and 100% independence isn’t necessary!
Table shows that any scores above 51% accuracy and 51% independence earn a score of Progressing.
The take-away:
Students can attempt more complex entry points without fear of lowering their score. Therefore…
Push your students to learn more challenging skills.
Have students attempt new, different skills each year.

8. Data Charts that Confuse Scorers

9. “Stair-Step” Data Charts
Accuracy and independence are virtually identical
on every date.
Massachusetts Department of Elementary and Secondary Education

10. “Purple Line” Data Charts
Is this technically possible?
Massachusetts Department of Elementary and Secondary Education

11. Data Charts: Three Likely Scenarios

12. “Zero-Prompt Instruction:” Always Independent, Working on Accuracy
Massachusetts Department of Elementary and Secondary Education

13. “Errorless Teaching:” Always Accurate, Working on Independence
Massachusetts Department of Elementary and Secondary Education

14. Working on both Accuracy and Independence
Massachusetts Department of Elementary and Secondary Education

15. Some Thoughts About Accuracy and Independence

16. Accuracy and Independence: Corrections/Edits/Prompts
If student gets the answer correct, whether prompted or not, that response is accurate.
Write “P” next to the prompted response on a work sample.
If response is inaccurate, then it’s incorrect.
If a prompt of any kind is given, then the response is “non-independent” (i.e., prompted)
If corrections/edits are made only by the student, then it is an accurate and independent response.
Massachusetts Department of Elementary and Secondary Education
Massachusetts Department of Elementary and Secondary Education

17. Charting Accuracy and Independence for a Series of Responses
Question Number
Accuracy
Independence
Question 1
Correct response (accurate)
Verbal prompt
(not independent)
Question 2
Incorrect response
(inaccurate)
Question 3
Gestural prompt
Question 4
Question 5
No prompt
(independent)
Overall Percent
60% accuracy
(3/5 correct)
20% independence
(1/5 independent)
All prompted responses, even those with partial prompts, are considered not independent.

18. …But Percentages Must Make Sense!
This sample was not scorable because scorer couldn’t verify how four questions could be 43% accurate and 43% independent.

19. Portfolio data and evidence must be authentic!
Please do not fabricate information, falsify dates, replicate or mass-produce evidence or data for more than one student.
In cases where this occurs, investigations will be requested and scores will be invalidated.
Start early enough in the year to complete your students’ portfolios by March 29, 2018.
Attend training and review sessions, and ask for help if you need it, before it’s too late.

20. We sometimes hear this comment…
“I submitted exactly the same work for all my students, but some were scored Incomplete while others received a score.”
Maybe something wasn’t clear to one (or more) of the scorers.
If vague or difficult to understand, multiple interpretations may result.
This should signal the need for greater clarity.
…Or maybe the work wasn’t all the same.

21. In Conclusion…
Be clear and concise in describing how the work was created. Score all student work.
Document accuracy and prompts clearly, item-by- item.
Scorers should see which responses were accurate and independent.
If not obvious, explain how it was scored.
Review your portfolio with another teacher before submitting.
Use all available supports and resources.

22. Questions? Thank you!