1. Children First Intensive
Mathematics Best Practices Connecting Assessment, Inquiry and the Quality Review
December 9, ESA CFN 6 (Bob Cohen) and ESO Network 19 (Elvira Barone)
Co-Facilitators: Deena Abu-Lughod, SATIF; Frederica Capshaw, Instructional Support Specialist; Karen Ames, Achievement Coach
Contact us at or
Electronic copies available at

2. Agenda
8:00 Breakfast
8:30 Welcome, Introductions, Agenda, Rationale and Outcomes
9:00 Prediction and Analysis – Data tools and a new lens on School-Specific NYS 3-8 Mathematics Data to evaluate the effectiveness of the implemented curriculum (QR Statement 1.1)
10:00 Break
10:15 Lesson Planning and Research Lessons
11:30 Mathematics Resources, Lois Sharzer
12:15 Lunch
1:00 Breakout Groups: (A) Elementary -
(B) M/S and H/S-Donna Davis, National Consultant, Glencoe
2:30 Questions, Debrief, Evaluation

3. Rationale
The NYCDOE Progress Report uses NYS assessments to tell part of the story about how students performed and progressed in mathematics.
Item-by-item comparisons of a school’s students relative to the city’s students, and State benchmarks and “distinguishing” items identify strengths in a school’s teaching and its implemented curriculum.
Today, we will explore how teacher teams can use an inquiry approach to analyze assessment data, student work, and teacher work, to adjust curriculum, instruction, and assessments to improve learning outcomes. (QS 4.2)

4. What is Collaborative Inquiry?
Collaborative inquiry is a sustained process of investigation and action that empowers teachers to improve student achievement and close the achievement gap. Collaborative inquiry is:
Focused on student outcomes, using a systematic, data-informed approach.
Conducted by teams of teachers with a focus on small groups of students, paying close attention to those who are struggling while supporting the learning of all students.
Designed to develop and deepen rigorous, research-based instructional strategies and frameworks.
Currently, citywide, 990 of 3900 teacher teams (25%) are focusing on Mathematics.

5. Outcomes What are we going to know at the end of the day?
How to analyze item data in relation to your school’s implemented curriculum (QS 1.1)
How to analyze summative data to create a portrait of mastery and provide meaningful and actionable feedback on the effectiveness of classroom level, curricular and instructional decisions (QS 2.2)
How to use these items to evaluate and adjust instructional practices and to monitor progress (QS 5.2, 5.3)

6. Guide to Resources
Item Analyses and Test Items: Available on DOE web for , for evaluating and aligning curriculum and monitoring progress.
Curriculum Resources, pacing guides, strand trace, etc: Available on DOE web.
Trend Maps and State benchmarks: Available on ARIS Connect, for identifying power standards, and for identifying items that separate the 4s from the 3s, and the 3s from the 2s.
ItemData: Available in your school’s private community on ARIS Connect for viewing individual responses; available by past and current classroom (July and September versions).
Web resources: Available on ARIS Connect, for instructional and assessment support.
Lesson study toolkit:
Acuity ITAs, Predictives and Customized assessments: From home computer, acuity.mcgraw-hill.com/index.jsp
To Schedule Professional Development: c+Assessment+Professional+Development.htm

7. Activity #1: Item Analysis Predicting your outcomes
Think about the 2009 NYS assessment results in a grade you know well (you should be sitting at that table).
Look at the trend map for that grade, the Network averages by item relative to NYC, and the items themselves.
On which items do you think your students did well, relative to other students in the Network or City? On which items did they struggle?
Commit yourself to paper. On Worksheet #1, write down at least 3 strengths and 3 weaknesses and justify your predictions.

8. Trends Maps
The Trends Maps help you identify the “Power Standards”: those performance indicators that are most frequently tested on the State exams.
They show which items tested which performance indicator over the last few years and the percent of questions used to test each strand.
Let’s look at the Trend Map for Grade 8 together.
Be aware that although some indicators are not tested every year, they may involve necessary skills.

9. Gr 8 2009 Trend Map (from BOCES)

10. Priorities: NYS Math 2008 Distribution by Strand
Gr 3
Gr 4
Gr 5
Gr 6
Gr 7
Gr 8
Number Sense
52%
44%
40%
32%
13%
Algebra
16%
15%
20%
47%
Geometry
26%
17%
Measure ment
19%
18%
14%
Statistics
10%
12%
23% 0%
Note that this is the actual 2008 strand distribution. You also have NYSED’s Approximate Percentage of Questions Assessing
Each Strand document in your resource packet.

11. 2009 Item Analysis from Educator Resources

12. 2009 Items (from DOE Educator Resources)

13. Activity #1, cont.: Check your predictions
Examine your school-specific item analysis chart . How accurate were your predictions about your school’s strengths and weaknesses? What might explain the variance on the items? What are the implications? Share out.
Note: Let’s see if anyone identified at least 5 correctly. I have 3 copies of a book that we can give out as a prize.

14. Comparing ourselves with Benchmarks
How do you know which questions are strategic (matter the most)?
How do you know which strands and indicators represent a strength or a weakness for your school?
How do you know whether a question was difficult just for your own students or for all students?
Comparing our own scores with the State Benchmarks helps us answer these questions.
Then we must ask: To work strategically, what questions should we focus on? The ones that were difficult for all students or just for some? How should we deal with infrequently tested indicators?

15. What are the State Benchmarks?
The Benchmarks will tell you what percent of Level 2, Level 3 and Level 4 students answered each item correctly. This is a more useful indicator than the p-value provided in the Item Analysis.
The benchmarks help you identify which questions distinguish the Level 4 students from the Level 3 students and the Level 3 students from the Level 2 students.
Understanding these differences may help you select what to work on if you want to push 2s to 3s, and keep 3s and 4s from slipping.

16. Let’s understand how to read the files
Item 5 distinguished the Level 3 from Level 2 students
Item 6 distinguished the Level 4 from Level 3 students
On average, there was a 34 percentage point difference between 4s and 3s, and 25 points between 3s and 2s.

17. Clarification: ARIS Taxonomy
You saw these codes in the ItemData files. Focus on the 1st, 3rd and last characters to see the grade, strand, and indicator number.

18. Geometry: Gr 8 Distinguishing Questions
In Grade 8, X% of the questions deal with geometry. In this extract of “distinguishing questions”, what do you observe about geometry?

19. Algebra: Gr 8 Distinguishing Questions
In Grade 8, X% of the questions deal with algebra. In this extract of “distinguishing questions”, what do you observe about algebra?

20. Activity #2: Research Lesson Modeling
The following question represents a power standard, and one that distinguishes 4s from 3s.
Note: Let’s see if anyone identified at least 5 correctly. I have 3 copies of a book that we can give out as a prize.

21. Activity #2: Research Lesson Modeling
Use Worksheet #2 to record the following (individually):
Sequence of understandings a student must have in order to solve the problem (use the strand trace, if desired)
Tasks and experiences that help students develop these understandings
Solve the task and record your solution.
Indicate additional ideas about how other students might solve the item, anticipating common errors.
If the tasks spark any new thoughts about student understanding and how it develops, modify your concept map as needed.
Note: Let’s see if anyone identified at least 5 correctly. I have 3 copies of a book that we can give out as a prize.

22. Problem Solving Strategies (p. 43 Van De Walle)
1. Draw a picture, act it out, use a model. 2. Look for a pattern. 3. Guess and check. (Make an attempt; reflection can lead to a better idea.) 4. Make a table or chart. (Often for problems involving ratios or measurements.) 5. Try a simpler form of the problem. (Simplify quantities so the resulting task is easier to understand and analyze.) 6. Make an organized list. (Show the number of possibilities and verify that all possible outcomes have been included.) 7. Write an equation. (Convert the story into numbers or symbols)
Note: Let’s see if anyone identified at least 5 correctly. I have 3 copies of a book that we can give out as a prize.

23. Four Step Problem Solving Template
As you plan, consider the value of the 4-Step Problem Solving Template (adapted from Bob Gyles, Math Think Tank):
Note: Let’s see if anyone identified at least 5 correctly. I have 3 copies of a book that we can give out as a prize.

24. Activity #3: Small Group Research Lesson
At your table, select ONE question that is both important and that represents a weakness at one or more of your schools.
Use your copy of Worksheet #2 to record the following:
Sequence of understandings a student must have in order to solve the problem (use the strand trace, if desired)
Tasks and experiences that help students develop these understandings
Solve the task and record your solutions and ideas about student solutions. If the tasks spark any new thoughts about student understanding and how it develops, modify your concept map as needed.

25. Activity #3: Small Group, Cont.
Using Worksheet 3, select a few of the tasks/activities you have identified, and indicate them and the associated questions, the anticipated student responses, the teacher supports, and the points of evaluation.
How can this type of planning produce better outcomes in class?
What else would you add or want to do?

26. Activity #4: Monitoring Progress
Group A: Assume you have developed and delivered the lesson you just drafted to the students currently in that grade. How can you use the item or items from the 2009 State exam (and/or exams from previous years) to know whether this year’s students on that grade are performing better than last year’s students did? Group B: Assume you have delivered the lesson you just drafted to the same students who had trouble with that indicator last year (eg, a target population). How can you use the item or items from the 2009 State exam to know whether these students have now mastered the skill?

27. Both Collaborative Inquiry and Lesson Study
Emphasize collaboration around a common focus
Link directly to planning and classroom practice
Focus on a specific topic or problem related to goals for student learning
Use change strategies or research lessons as learning tools, rather than as products
Involve teachers’ use of existing curriculum and learning activities to improve them
Focus on the lesson or instruction, rather than the teacher
Involve team members who contribute equally
Involve live observation of students in the act of learning
Develop a common understanding by teachers of the context of student work.

28. Inquiry Cycle Lesson Study Cycle
Set Goals
Plan the lesson using research
Teach, observe, debrief
Revise and reteach
Reflect and share results

29. Lesson Study and Toolkit

30. Important Resources for our Schools
Special Thanks to:
Lois Sharzer
Lois Sharzer and Associates

31. Teacher Actions Reference Sheet
Before:
Activate prior knowledge: begin with a simple version of the task, brainstorm solutions, estimate or use mental computation. Be sure the problem is understood. Establish clear expectations.
During:
Let go! Once students understand what the problem is asking, hold yourself back from stepping in when students struggle. Listen actively to understand the student’s approach to the problem. Ask: What ideas have you tried so far? Can you tell me more about…? Why did you …? Provide appropriate hints. Provide worthwhile extensions.
After:
Encourage student-to-student dialogue; request explanations to accompany all answers; call on students for their ideas, encouraging those who are shy; encourage students to ask questions; be certain your students also understand what you understand. Listen actively without evaluation. Summarize main ideas and identify future problems.

32. Afternoon: Breakout Groups
Donna Davis, National Staff Developer, Glencoe
Deep Dive into Lesson Study
Text-based Discussion: What’s Sophisticated about Elementary Mathematics?
Tech Tools: An Exploration of Web Resources
Core Standards

33. Learning Intentions: How did we do?
How did we analyze data? (QS 1.1)
How did that analysis of summative data help us create a portrait of mastery? How did we think we could provide meaningful and actionable feedback on the effectiveness of classroom level, curricular and instructional decisions? (QS 2.2)
How did we consider using these items to evaluate and adjust instructional practices and to monitor progress? (QS 5.2, 5.2)
Please complete the Evaluation form and leave on your table.
We analyzed school-level item data in relation to the city average on each item to see where our school’s implemented curriculum was stronger or weaker relative to the city, and where there were items that overall, separated the 4s from the 3s and 3s from the 2s.
We discussed what it would look like if we were to provide meaningful and actionable feedback on the effectiveness of classroom level, curricular and instructional decisions.