1. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 1 Getting Ready for Math Constructed Response Questions Accountability Conference February 2009 Greensboro, NC Mike Gallagher Math Test and Measurement Consultant Accountability Services, NCDPI mgallagher@dpi.state.nc.us

2. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 2 Purposes Today Summative Assessment Use of Constructed Response (CR) questions: Understand NC’s direction and goals. Formative Assessment Use of CR questions. Evaluate an approach to use CR questions instructionally to improve results on both multiple choice and CR math questions.

3. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 3 Framework for Change The Response to the Framework for Change (R2F4C) calls for: Inclusion of constructed response (CR) questions in summative assessments Use of formative assessment as part of a balanced assessment system. See the Response on the DPI web site http://www.ncpublicschools.org/

4. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 4 Constructed Response and R2F4C (p.20) General Summative Assessment Recommendations 1. Use constructed-response and performance task items when such items are appropriate based on developed criteria. [Note: Criteria are due to the SBE August 2009.]

5. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 5 Math Timeline (from R2F4C) June 2009 Essential standards to SBE 2009-10 Item Development 2010-11 Field Test 2011-12 Operational Tests Note that test specifications will be set after standard adoption. Test specs will detail number and type of items.

6. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 6 Role of CR questions “Participating states stressed that …. [CR questions] … require students to demonstrate critical thinking and problem solving.” from American Diploma Project Algebra II End-of- Course Exam: 2008 Annual Report, August 2008. (Available on-line) CR questions had about 1/3 of the total point value on the test.

7. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 7 Example test design – Ohio Graduation Test Question Type Value Per Item ItemsPoints Poss. % of Ttl. Pts. Multiple Choice 132 70% Short Answer 251022% Extended Response 4149%

8. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 8 It’s Testing Time! Grade 8 Question* from NAEP How many boxes of tiles are needed? (See pages attached.) Since you may not have a ruler handy, the figure is 2 ½ inches by 4 inches. * Block: 2005-8M3 No.:18

9. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 9 Notes on the NAEP item This is an example of a Constructed Response (CR) question of the “Extended Response” type Students need practice with … “EXPLAIN” and “SHOW YOUR WORK.” !!!

10. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 10 Related Materials from NAEP After you work the grade 8 problem, check over the: Scoring Guide Sample Responses Student Results CR questions, compared with multiple choice, present a scoring challenge and an instructional opportunity.

11. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 11 What is Formative Assessment? Formative assessment is a process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to improve intended instructional outcomes (CCSSO FAST SCASS, 2006). Quoted in R2F4C, p. 14.

12. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 12 Instructional Opportunity – Descriptive Feedback Constructed response questions provide an excellent resource for formative assessment … i.e., Assessment for Learning What feedback could be given to students who generated the sample responses for the NAEP question?

13. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 13 Problem Solving Strategy (Copy Attached) The Strategy is a one-page sheet of bullet items on the DPI web site, www.ncpublicschools.org/accountability/testing/eog/math/ Scroll down to Formative Assessments for Problem Solving The strategy sheet can be used to guide Formative Assessment descriptive feedback for problem solving throughout the year.

14. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 14 What feedback to give? (for responses to the NAEP item) For “satisfactory” answer? -------------------------------------------------- For “partial” answer? -------------------------------------------------- For ‘minimal” answer? --------------------------------------------------

15. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 15 “Follow the Yellow Brick Road” YB Road: How to use math constructed response questions as part of ongoing formative assessment to help students improve their performance on current EOG/EOC math tests and on coming tests under essential standards.

16. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 16 The Yellow Brick Road (YB Road) is a different sort of road. Formative assessment does not have teachers doing more grading. Teachers guide students to more peer and self assessment.

17. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 17 YB Road – 1. Getting Started Use an initial quiz or test, which includes some “problem solving” questions. Use a class discussion to get to characteristics of good problem solving. At a later class, introduce the Polya strategy sheet and compare with the class developed characteristics.

18. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 18 YB Road – 1: Start Up Get students acquainted with Polya’s heuristic, using the Problem Solving Strategy sheet. Set the target for students: become strong math problem solvers. Explain that they will be getting and giving feedback on problem solving throughout the school year.

19. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 19 YB Road – 2: Regular Classroom Assessments – Design Be sure to include some “juicy” questions. Such items get at problem solving more than procedures. Such questions would not need to be tied to the current chapter: sample EOC/EOG questions, DPI indicators, NAEP released questions, ….. etc.

20. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 20 YB Road – 2: Regular Classroom Assessments – Design For some of the questions require students to explain: Student’s reasoning guides feedback. Students writing an explanation deepens understanding.

21. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 21 YB Road – 3: Regular Classroom Assessments -- Feedback When reviewing how students did on each assessment or test, give and model descriptive feedback on problem solving to the class. Oh, and remember: Teachers don’t need to do more grading. Guide students to more peer and self assessment.

22. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 22 YB Road – 3: Regular Classroom Assessments -- Feedback One to many. Teachers provide feedback to the class on common strengths and weaknesses. Many to many. Students provide feedback to each other and themselves. For self and peer assessment, students are providing feedback, not grading.

23. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 23 YB Road - 3 Use descriptive feedback that generalizes past this particular problem to the problem solving heuristic. Discussion example: Released NAEP grade 8 question about boxes of tile needed for a room. Block 2005-8M3, No. 18.

24. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 24 YB Road – 4 Ungraded Classroom Assessments Transition to more ungraded assessments as students develop with self and peer assessment. “Ungraded” as used here means that the results don’t go into the grade book. However, the assessments are corrected and feedback is given.

25. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 25 There is much to learn …. Go to the NCDPI web site: www.ncpublicschools.org/accountability Select “for educators” on the left. Scroll way down and select “Learn more about formative assessment” on the left. And … PD modules are coming!!

26. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 26 2 Books of Note G. Polya... How to Solve It, the book is from the 1950s, but it’s stood the test of time. R. Stiggins … Student-Based Classroom Assessment, Very readable textbook. Stiggins has done a lot on focusing assessment feedback to the learner.

27. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 27 NC Math Formative Assess. Menu Page Good Strategies Missing Adaptation of Polya’s Problem Solving Strategy. Key Classroom Activities Feedback Sheet Problem Sources www.ncpublicschools.org/accountability/testing/eog/math/ Scroll down to Formative Assessments for Problem Solving

28. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 28 A Message from the President! * “NCTM’s Assessment Principle indicates that assessment should not be done to students; rather, assessments are for students and should be used to guide and enhance their learning …” (continued) * Francis (Skip) Fennell, President of the National Council of Mathematics Teachers (NCTM)

29. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 29 President’s Message (cont) “Formative assessment involves using classroom-based assessments to collect feedback that can be used to improve teaching and learning.” From NCTM News Bulletin (December 2006) Complete message on the web at http://www.nctm.org/ … Go to the “President’s Corner” -- Just one page and well worth reading!

30. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 30 A Balanced Math Assessment Diet Type Exercises Problems Puzzles Projects Goal Assessment for Learning Assessment of Learning

31. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 31 Feedback becomes key for higher-level learning When it comes to learning, students are the key assessment users: “Am I succeeding? Am I improving over time? What should I do next to succeed? Does my teacher think I am capable of success? Do I think I am capable of success?” …from Student-Involved Classroom Assessment, 3 rd Edition. Richard J. Stiggins, 2001, Prentice-Hall.

32. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 32 Assessment for Learning inverts the pyramid of users Classroom Formative Assessment is Assessment for Learning For large scale, summative assessment, the key users are often national and state, moving down to teachers. For classroom formative assessment, the key users should start with the student.

33. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 33 Try it! Formative Assessment for Higher-Level Thinking “Students who participate in the thoughtful analysis of quality work, so as to identify its critical elements or to internalize valued achievement targets, become better performers. When students learn to apply those standards so thoroughly that they can confidently and competently … more

34. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 34 Higher-Level (cont) … evaluate their own and each others’ work, they are well down the road to becoming better performers in their own right (Stiggins, p. 25) Good papers to read and other resources at Stiggins’ ATI website: http://www.assessmentinst.com/

35. PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 35 Conclusion o Constructed response questions help with both summative and formative assessment. o In balanced assessment, students assess more.