1. DDI Session II: Analyzing and Tracking Data May 2014 David Abel, Fellow for Curriculum and Assessment/ELA EngageNY.org

2. Session Objectives Be able to describe what to look for when analyzing student work for a Common Core- aligned assessment Be able to create a data tracker for assessments Develop questions that drive data-analysis meetings around Common Core-aligned assessment data Engageny.org

3. AGENDA Introduction Warm Up: Quick Review of Assessment Design Looking at Student Work Tracking Student Work Leading a Data Meeting with Questions Q & A Engageny.org

4. Introduction Who am I? what do I do? Why do I do it? Engageny.org

5. Where we’ve been, where we’re going with DDI DDI session I ( Feb NTI): Assessment Design DDI session II (May NTI): Analysis* DDI session III (July NTI): Action Plan Engageny.org

6. Review of Session 1: Assessment Design Engageny.org What do we know about this student, based on this response? What DON’T we know about based on this response?

7. Review of Session 1: Assessment Design Key understandings o Text selection matters….a lot o Comprehension is king o Sometimes we do don’t know what we don’t know—but we aim to find out! Engageny.org

8. Analysis Engageny.org “Dusting for fingerprints” Tracking the data from the assessments

9. Analysis Engageny.org “Connecting the dots” Drawing conclusions from the data

10. The challenges of analysis Looking for that sweet spot between doable and useful! We don’t want to “granularize” content… …but we have to do something to look “under the hood” in order to move forward We want our students engaged in rich tasks… …but we want to dig into the work associated with the tasks to learn specifics aboutwhat our students know and can do We don’t want to reduce student data to a “checklist” where the complexities of expository writing are reduced to 0s and 1s …but we want usable information about how are kids are doing with respect to the demands of the Common Core Engageny.org

11. The challenges of analysis Tracking data Read the grade 8 “Brain Birds: Amazing Crows and Ravens,” on page 2, the prompt on page 5, and the rubric on page 7. Then, read student responses #1, #2, #3 and #4 (pages 8-11) In groups, looking at the prompt, the rubric, the responses, discuss what data you can track from these responses. Feel free to use the Data tracking sheet on page 13 of your handout if you want. Engageny.org

12. Sample Data tracking template STUDEN T NAME SCORE [DATA POINT #1] [DATA POINT #2] [DATA POINT #3] [DATA POINT #4] [DATA POINT #5] [DATA POINT #6] [DATA POINT #7] [DATA POINT #8] Student #1 2 [DATA DESCRIP TOR] [DATA DESCRIPT OR] Student #2 1 [DATA DESCRIP TOR] Student# 3 1 [DATA DESCRIP TOR] [DATA DESCRIPT OR] Student #4 0 [DATA DESCRIP TOR] [DATA DESCRIPT OR] Engageny.org DATAPOINTS: the WHAT of what you are capturing DATA DESCRIPTORS: the HOW/HOW MUCH of what you are capturing

13. Data Tracking: option 1 Follow the rubric Student Name ScoreValid Inference/Claim EvidenceMechanics/ spelling Student # 12YYY Student # 21NYN Student # 31YY Student # 40NNN Engageny.org PLUS: validates scores on rubric DELTA: not specific, does not tell us granular info Mostly literal recounting ??????????? >2 pieces of evidence

14. Data Tracking: option 2 Beyond the rubric Student Name ScoreValid Inference or Claim from text First example of textual evidence Second example of textual evidence Readability of writing and minor errors Student # 1 2YYYY Student # 2 1NYNN Student # 3 1NYNY Student # 4 0NNNN Engageny.org PLUS: validates score on rubric; more detail than option 1 DELTA: Mostly literal recounting >2 pieces of evidence

15. Data Tracking option #3 (adapted Erie 2 BOCES DDI tracker) Engageny.org DATA DESCRIPTORS P = Proficient D = DevelopingA = analysis L=literal XO=does not demonstrate Student Name DATAPOINTS Score from Short Response Rubric Valid Inference or Claim from text One example of textual evidence Second example of textual evidence Readability of writing and minor errors Student #1 2 Student #2 1 Student #3 1 Student #4 0

16. Data Tracking option #3 (adapted Erie 2 BOCES DDI tracker) Engageny.org DATA DESCRIPTORS P = Proficient D = DevelopingA = analysis L=literal XO=does not demonstrate Student Name DATAPOINTS Score from Short Response Rubric Valid Inference or Claim from text One example of textual evidence Second example of textual evidence Readability of writing and minor errors Student #1 2 PPPP Student #2 1 LLLXO Student #3 1 LLXO Student #4 0 LXO

17. Trend analysis One: acknowledge that for this task, the line between the evidence and the claim can be blurry Students 2-4 are not moving beyond literal claims, vs. student 1 who makes a claim about crow and raven intelligence. Students 2-4 are not able to produce writing without errors that interfere, to varying degrees, with readability Student 2 does not include a second piece of evidence Engageny.org

18. Trend analysis…with a touch a hypothesis Student 2 gives two pieces of literal evidence, and likely comprehends the question and aspects of the text. Student 3 gives one piece of literal evidence, and likely comprehends the question and aspects of the text. Student 4 largely reiterates the prompt, but possibly demonstrates some evidence of comprehension the question and/or the text. Engageny.org

19. The formative advantage Through scaffolding towards an assessment that is embedded in curriculum, you can gather important data points for tracking Example: from Grade 10 Module beginning on page 14(text except on page 12) Engageny.org

20. Deconstructing Grade 10 prompt Grade 10 Prompt: How does King use rhetoric to advance his purpose in paragraph 9? CCSS.ELA-LITERACY.RI.9-10.6 CCSS.ELA-LITERACY.RI.9-10.6 Determine an author's point of view or purpose in a text and analyze how an author uses rhetoric to advance that point of view or purpose. Engageny.org HOW: Explain/analyze RHETORIC: Student must know what this is, how it is used, how to identify it, how the author uses it PURPOSE: Student must know what the purpose of this paragraph is, and possibly how it relates to the purpose of the entire letter

21. Student-facing worksheet for 9-12 Engageny.org PROMPT: FOUNDATIONAL SKILLS: COMPREHENSION, WRITING ORGANIZATION, COMMAND OF GRAMMAR/MECHANICS Assessment Prompt ElementRESPONSE FOUNDATIONAL UNDERSTANDING I What vocabulary of the discipline do you need to know and understand in order to answer the prompt? FOUNDATIONAL UNDERSTANDING II What conceptual understandings of the text do you need in order to answer the prompt? What are the components of the response needed to answer the question? ANSWER (Claim) SUPPORTING EVIDENCE (textual evidence that supports claim) REASONING (connection of evidence to claim) SUPPORTING EVIDENCE (textual evidence that supports claim) REASONING (connection of evidence to claim) SUPPORTING EVIDENCE (textual evidence that supports claim) REASONING (connection of evidence to claim)

22. Data-analysis meetings What are the common understandings and norms that can drive productive data-analysis meetings? How to develop questions that will drive these meetings? How to use existing Bambrick models for these meetings? Engageny.org

23. Data-analysis meetings What are the common understandings and norms that can drive productive data-analysis meetings? THE LEAP OF FAITH Engageny.org

24. Data-analysis meetings DDI is organic—not one-size-fits-all. DDI is messy—sometimes the connected dots make a clear picture, sometimes they do not. The leap of faith necessary is to see value in this work, to want to dig in and play detective/investigative journalist in the interest of more effective instruction. Calibrate on understanding, because it takes a village to do this work! Engageny.org

25. Data-analysis meetings Teachers and administrators must come to consensus on DDI Teachers must reflect on what they need to know and be willing to brainstorm with administration. Administration must create a climate that is safe for this work Engageny.org

26. Data analysis meeting questions Using Bambrick model  Not going to be as neat and tidy, but you can grow this from Bambrick… Case Study from Erie 2 Chautauqua- Cattaraugus BOCES… Engageny.org

27. Data analysis meeting questions Engageny.org

28. Q&A Questions? Open issues? Strong opinions about the next session (action plan)? Engageny.org

29. THANK YOU David Abel dabel@mail.nysed.gov http://www.engageny.org/resource/regents- exams Engageny.org