1. DDI and Assessments in ELA
Designing Assessments that Provide Meaningful Data
David Abel, Fellow for Curriculum and Assessment/ELA
EngageNY.org

2. Session Objectives We will learn…
About the elements of a high-quality Common Core-aligned ELA assessment
How to create and choose assessment tasks that are rigorous, measure the standards and give comprehensive information about students at all levels
Who am I? what do I do? Why do I do it?
Engageny.org

3. Why do we assess…? Engageny.org
We assess because we need information about individuals
-sometimes for certification (driving, teaching)
-sometimes for actionable information
Engageny.org

4. Let’s begin…
Take 10 minutes to read the 6th grade passage (Sweet Science Comes Baked In), annotated test item, and student work samples 1-3 and answer this question … Given the standard the question is measuring, what can we say we know about the students in student responses #1-3?
(this should yield that students understand structure, can analyze how a sentence fits into an overall structure, but should also, with student response #1 (and perhaps #3) tell us that we don’t know if the issue is comprehension or meeting the standards--we’ll call this the “mystery zone” aka the known unknown.)
Engageny.org

5. The question MEASURES CCSS RI.6.5:
Analyze how a particular sentence, paragraph, chapter, or section fits into the overall structure of a text and contributes to the development of the ideas.
HOW THIS QUESTION MEASURES RI.6.5:
This question measures RI.6.5 because it asks students to analyze how a particular sentence fits into the overall structure and meaning of the text. More than one response may be correct, but a correct response will focus on Morgan’s background and family experience with baking science, as well as her aspirations.
(this should yield that students understand structure, can analyze how a sentence fits into an overall structure, but should also, with student response #1 (and perhaps #3) tell us that we don’t know if the issue is comprehension or meeting the standards--we’ll call this the “mystery zone” aka the known unknown.)
Engageny.org

6. Student Response #1
What do we know about this student, based on this response?
What DON’T we know about based on this response?
Engageny.org

7. Student Response #2
What do we know about this student, based on this response?
What DON’T we know about based on this response?
Engageny.org

8. Student Response #3
What do we know about this student, based on this response?
What DON’T we know about based on this response?
Engageny.org

9. The Known Unknown
When working with open-response questions that require COMPREHENSION and ANALYSIS as demanded by the standards, student responses such as #1 and, to an extent, #3 do not give you information about whether students didn’t perform well because they couldn’t meet the demands of the standard or because they couldn’t comprehend the complex text.
“... there are known knowns; there are things we know that we know. There are known unknowns; that is to say, there are things that we now know we don't know. But there are also unknown unknowns – there are things we do not know we don't know.”
Introduce this concept that we’ll be returning to when we talk about what the information we get from assessments—the Data of DDI
Engageny.org

10. The formative assessment advantage
Formative assessments can yield student work that gives richer information, given that they can be administered
Without the same time constraints/pressure that accompany the 3-8 ELA test
In a manner that connects directly to targeted instruction
Engageny.org

11. Common Core-aligned ELA assessment
What are the elements of a high-quality Common Core-aligned ELA assessment?
The CCSS for ELA/Literacy require comprehension, analysis and communication (writing/speaking) using evidence from text(s)
Open response writing items and/or performance tasks are the types of assessment questions that best provide a means for communication using evidence from text(s)
Engageny.org

12. Common Core-aligned ELA assessment
Grade-level standards add specific demands to the skills and knowledge of students on track for college and careers
Therefore, given the emphasis on textual evidence and the specific demands of grade-level standards, you need to use texts that can support measurement of the grade-level standards.
Example: CCSS.ELA-Literacy.RI.7.6 Determine an author’s point of view or purpose in a text and analyze how the author distinguishes his or her position from that of others.
Engageny.org

13. TEXT SELECTION IS THE FOUNDATION OF CCSS ELA ASSESSMENT
Texts at grade-level complexity
Questions that measure the CCSS
Rubrics to evaluate student work
Engageny.org

14. Or to put it another way…
The crucial components of CCSS ELA assessment are:
Texts that can support questions that measure the CCSS
Questions that measure the CCSS
Rubrics that give you good and actionable info about student performance
Engageny.org

15. Selecting texts
3-8 Assessment design documents begin on page 9 of your packet and include:
Passage selection guidelines
New York State Passage Selection Resources for Grade 3‐8 Assessments
Walkthrough of the docs
Quan/qual
Esp qual analysis
Guidelines—esp text content and quality
Engageny.org

16. Features of a Complex Text
Subtle and/or frequent transitions
Multiple and/or subtle themes and purposes
Density of information
Unfamiliar settings, topics or events
Lack of repetition, overlap of similarity in words and sentences
Complex sentences
Uncommon vocabulary
Lack of words, sentences or paragraphs that review or pull things together for the student
Longer paragraphs
Any text structure which is less narrative and/or mixes structures
Complex text contains any and all combinations of these features in many combinations.
The complexity level is determined by both quantitative and qualitative measures. The details of text complexity are well described in Appendix A of the Standards, one of the supplemental readings offered with this module. New tools have been developed since the Standards were developed to help determine qualitative text complexity. Those materials are available on
Students who struggle with reading almost always have gaps in their vocabulary and their ability to deal with more complex sentence structures. This too is well documented in research.
Too often, less proficient students are given texts at their level where they do not see these features, where the demands of vocabulary and sentence structure are lowered. Though this is done for the kindest of reasons, it has disastrous consequences. Day by day, differentiating by level of text during instructional time increases the achievement gap between high performers and those who struggle.
Students cannot address gaps in their vocabulary and develop skill with unpacking complex syntax text when they are not given the opportunity to work with material that provides these opportunities.
With that said, there is a place for providing students with text more appropriately matched to their individual reading abilities to build fluency and provide opportunity for increasing the volume of reading. But those texts cannot be the primary texts for instruction.
EngageNY.org
Engageny.org

17. Take out your Homework…
Does the text you used in your assessment meet the demands articulated in these review docs?
How about text #1? #2?
Which text would be better to measure the CCLS as articulated on pages 16-30?
Discuss with a partner
Engageny.org

18. Questions that measure the CCSS
Extended Response types measure CCSS Writing Standards (W1, 2)
CCSS Writing standards integrate Reading and Language Standards (W5, W9)
Short response types
Claim + evidence
Claim = statement of comprehension/analysis required by the standard
Evidence = proof of comprehension/analysis required by the standard
Engageny.org

19. Questions that measure the CCSS
Claim + evidence short response questions
Depending on the text, you can progress through the reading standards …
from high-level comprehension/what is a central idea of this text/(R2) …
to the more granular (R2-3)…
to craft and structure (R4-6)…
and, if using multiple texts, integration of knowledge and ideas (R7-9)…
All requiring a set number of details (at least 2) in support of the claim.
Once in this place, you can build on this to assess other standards (Writing, S&L, Language) with extended response questions and performance tasks
Engageny.org

20. Questions that measure the CCSS
How to write these questions? Don’t do the analyzing for the kids in your efforts to be specific, but... Don’t be so broad that nearly anything (or almost nothing) answers the question!
Engageny.org

21. Questions that measure the CCSS
Examples: (using Text #1)
How does the author's perspective build suspense for the reader?
How does the author support the idea that the narrator felt “detached” from his injured leg?
Non-examples: (using Text #1)
How would you feel if you were in this situation? What would you do?
How does the accident result in "an uncrossable gap” between the climbers?
Engageny.org

22. Take out your Homework…
Do your questions measure the CCSS(pages 16-30) ?
Do they provide actionable information about student ability to comprehend and analyze complex texts?
What are the strengths or weaknesses of your questions? Discuss with a partner.
Engageny.org

23. Rubrics Form follows function! How much information do you need?
Do you want a quick view of the student’s ability to display comprehension and analysis and mostly measure reading standards?  Go holistic a la short response rubric
Do you want to measure an extended writing assignment that can measure the reading, writing, and language standards?  Use the extended response rubric and get granular.
Sample rubrics on page 43-45
Engageny.org

24. Return to the student work packet…and student #1
Next steps could be:
Referring to an earlier close reading lesson/unit in which scaffolded instruction has happened targeting and assessing that standard
If the student struggled with the culminating assessment aligned to that standard, then it is possible the student is struggling with the standard
If the student performed well on the assessment, then it is possible that the student is struggling with comprehension of complex text
Engageny.org

25. Return to the student work packet…and student #1
Next steps could be:
Administering an assessment targeting the same standards via a read aloud to see if the student can meet the standard via speaking and listening
Students who are not reading on grade level are often capable of making inferences and conducting sophisticated analyses!
Targeted skills and comprehension strategy work with differentiated groups, followed by claim and evidence style questions to monitor comprehension
Students’ reading levels must be built up, via targeted interventions and a volume of independent reading.
Engageny.org

26. But! But you will need to make time to overwhelm the problem!
Without this the data will just tell you what you may already know….running in place.
Engageny.org

27. Partner Practice
With a neighbor, think about how you might need to adjust your assessment with students who are struggling to comprehend grade-level complex texts? 
Do you need to select a new text?  Revise the questions?  Both?
What else can you do to get good data?
Engageny.org

28. SUMMING UP Text selection matters
The standards can not be met without comprehension
This is challenging and time-consuming…But worth it!
Engageny.org

29. THANK YOU
David Abel
Engageny.org