1. Mathematics EQuIP Rubric & Quality Review Training 1

2. Welcome “Who’s in the Room”

3. maccss.ncdpi.wikispaces.net

4. Follow Us! NC Mathematics www.facebook.com/NorthCarolinaMathematics @ncmathematics http://maccss.ncdpi.wikispaces.net

5. Is it important to evaluate instructional materials? Why or Why not?

6. How do you currently evaluate instructional materials and resources?

7. 7 Session Goals Use the EQuIP quality review process to determine the quality and alignment of lessons and units to college- and career- ready standards in mathematics During this session, reviewers will: Develop their ability to use EQuIP criteria to provide observations about college- and career-readiness aligned instructional materials and make suggestions for improvement. Develop a common understanding of the EQuIP quality review process. Develop a common understanding of the rating scale and descriptors for the four rubric dimensions and the rating categories and descriptors for overall ratings. Develop their abilities to use EQuIP criteria, rating scales and rating descriptors to accurately rate instructional materials.

8. 1.Alignment: Before beginning a review, all members of a review team are familiar with the college- and career-ready standards for the grade band targeted. 2.Inquiry: Review processes emphasize inquiry rather than advocacy and are organized in steps around a set of guiding questions. 3.Respect & Commitment: Each member of a review team is respected as a valued colleague and contributor who makes a commitment to the EQuIP process. 4.Criteria & Evidence: All observations, judgments, discussions and recommendations are criterion and evidence based. 5.Respectful and Constructive Feedback: Lessons/units to be reviewed are seen as “works in progress.” Reviewers are respectful of contributors’ work and make constructive observations and suggestions based on evidence from the work. 6.Individual to Collective: Each member of a review team independently records his/her observations prior to discussion. Discussions focus on understanding all reviewers’ interpretations of the criteria and the evidence they have found. 7.Understanding & Agreement: The goal of the process is to compare and eventually calibrate judgments to move toward agreement about quality with respect to college- and career-readiness. EQuIP Quality Review: Principles & Agreements 8

9. Five Quality Review Steps When working with a review team: Have a review plan that considers the experience and expertise of all team members. Team members may choose to compare individual ratings after each dimension or wait until each person has individually rated and recorded all input for Dimensions II–IV before beginning discussion. Individuals should record their overall rating prior to discussion. Adjustments to ratings and/or commentary should take place as a part of the group discussion. 9

10. EQuIP Quality Review Process The EQuIP quality review process is a collegial process that centers on the use of criteria-based rubrics for English language arts (ELA)/literacy and mathematics. The criteria are organized into four dimensions: The Four Dimensions As educators examine instructional materials against the criteria in each dimension, they are able to calibrate their thinking, through the use of common standards for quality, and generate evidence-based commentary and ratings on the quality and alignment of materials. EQuIP Quality Review: Process & Dimensions 10 Alignment to the depth of the CCSS Key shifts in the CCSS Instructional supports Assessment

11. 11 The EQuIP Rubric for Mathematics – The 1-Page Version

12. 12

13. 13 Step 1. Review Materials Step 2. Apply Criteria in Dimension I: Alignment to the Depth of the college- and career-readiness standards Step 3. Apply Criteria in Dimensions II–IV Step 4. Apply an Overall Rating and Provide Summary Comments Step 5. Compare Overall Ratings and Determine Next Steps The 5 Steps

14. 14 Using the Electronic Quality Review Rubric PDF Form maccss.ncdpi.wikispaces.net

15. 15 Effective Feedback Writing effective feedback is vital to the EQuIP Quality Review Process. Below are the four qualities of effective feedback. Criterion-based: Written comments are based on the criteria used for review in each dimension. No extraneous or personal comments are included. Evidence Cited: Written comments suggest that the reviewer looked for evidence in the lesson or unit that address each criterion of a given dimension. Examples are provided that cite where and how the criteria are met or not met. Improvement Suggested: When improvements are identified to meet criteria or strengthen the lesson or unit, specific information is provided about how and where such improvement should be added to the material. Clear Communication: Written comment are constructed in a manner keeping with basic grammar, spelling, sentence structure and conventions.

16. 16 Step 1

17. Five Quality Review Steps Step 1 Step 1. Review Materials Record the grade and title of the lesson/unit on the Quality Review Rubric PDF Scan to see what the lesson/unit contains and how it is organized Read key materials related to instruction, assessment and teacher guidance Study and work the tasks that serve as the centerpiece for the lesson/unit, analyzing the content and mathematical practices the tasks require 17

18. 18 Grade 4 — CPALMS 2-Digit Array Multiplication Step 1. Review Materials Sample Lesson: Number and Operations in Base Ten

19. 19 Step 2: Dimension I

20. 20 The EQuIP Rubric for Mathematics – The PDF Version

21. 21 Step 2. Apply Criteria in Dimension I: Alignment to the Depth of the CCSS Identify the grade-level CCSS that the lesson/unit targets Closely examine the materials through the “lens” of each criterion Indicate each criterion for which clear and substantial evidence is found Record input on specific improvements needed to meet criteria or strengthen alignment Enter a rating of 0–3 for Dimension I Note: Dimension I is non-negotiable. For the review to continue, a rating of 2 or 3 is required. If the review is discontinued, consider giving general feedback that might help developers/teachers make decisions regarding next steps Five Quality Review Steps Step 2

22. 22 Step 2: Criteria for Dimension I: Alignment to the Depth of the CCSS The lesson/unit aligns with the letter and spirit of the CCSS:  Targets a set of grade-level CCSS mathematics standard(s) to the full depth of the standards for teaching and learning  Standards for Mathematical Practice that are central to the lesson are identified, handled in a grade-appropriate way and well connected to the content being addressed  Presents a balance of mathematical procedures and deeper conceptual understanding inherent in the CCSS Now, let’s try Step 2.

23. 23 Criterion:  Targets a set of grade-level CCSS mathematics standard(s) to the full depth of the standards for teaching and learning Observations and suggestions: The lesson …. Step 2. Apply Criteria in Dimension I: Alignment Normed Response: Number and Operations in Base Ten

24. 24 Criterion:  Targets a set of grade-level CCSS mathematics standard(s) to the full depth of the standards for teaching and learning Observations and suggestions: This lesson addresses half of the standard for grade four - 4.NBT.B.5. It does provide a depth of understanding for teaching multiplying a 2-digit number by a 2-digit number using arrays. The lesson states that it will take one hour to present, yet further in the lesson, the writer states that the lesson may take more time for students to practice the procedure. Additional time might be considered in the Overview box by C-PALMS, so teachers understand that it is okay to take the time to go deeper. Step 2. Apply Criteria in Dimension I: Alignment Normed Response: Number and Operations in Base Ten Sample

25. 25 Criterion:  Standards for Mathematical Practice that are central to the lesson are identified, handled in a grade-appropriate way and well connected to the content being addressed Observations and suggestions: The lesson plan …. Step 2. Apply Criteria in Dimension I: Alignment Normed Response: Number and Operations in Base Ten

26. 26 Criterion: Standards for Mathematical Practice that are central to the lesson are identified, handled in a grade-appropriate way and well connected to the content being addressed Observations and suggestions: The SMPs addressed in this lesson are #7 and #3. However, SMP 7 was weaker in its alignment, especially in critiquing the work of others. SMP 6 was not listed; however, the reviewers felt the lesson did a good job with this SMP especially with precise academic language. The author included these and not the C-PALMS group. The SMPs are critical in helping to get to the rigor for the lesson. A note to teachers in the specific area where those SMPs are located will help teachers connect to the content to inform their understanding of the standard. Step 2. Apply Criteria in Dimension I: Alignment Normed Response: Number and Operations in Base Ten Sample

27. 27 The lesson/unit aligns with the letter and spirit of the CCSS:  Presents a balance of mathematical procedures and deeper conceptual understanding inherent in the CCSS Observations and suggestions: Step 2. Apply Criteria in Dimension I: Alignment Normed Response: Number and Operations in Base Ten

28. 28 The lesson/unit aligns with the letter and spirit of the CCSS:  Presents a balance of mathematical procedures and deeper conceptual understanding inherent in the CCSS Observations and suggestions: The balance between the procedure and conceptual understanding is a strength of this lesson. The teacher helps students get to place value understanding in order to build the procedure of multiplication of numbers greater than one digit. This lesson will lead students to an efficient algorithm for multiplication. Step 2. Apply Criteria in Dimension I: Alignment Normed Response: Number and Operations in Base Ten Sample

29. 29 Using Dimension Ratings and Descriptive Scales To Synthesize Judgment Rating Scale for Dimensions I–IV: 3: Meets most to all of the criteria in the dimension 2: Meets many of the criteria in the dimension 1: Meets some of the criteria in the dimension 0: Does not meet the criteria in the dimension

30. 30 Using Dimension Ratings and Descriptive Scales To Synthesize Judgment Rating Scale for Dimensions I–IV: 3: Meets most to all of the criteria in the dimension 2: Meets many of the criteria in the dimension 1: Meets some of the criteria in the dimension 0: Does not meet the criteria in the dimension Descriptors for Dimensions I–IV: 3: Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations 2: Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations 1: Developing toward CCSS Quality — needs significant revision, as suggested in criterion-based observations 0: Not representing CCSS Quality — does not address the criteria in the dimension

31. 31 Using Dimension Ratings and Descriptive Scales To Synthesize Judgment Rating Scale for Dimensions I–IV: 3: Meets most to all of the criteria in the dimension 2: Meets many of the criteria in the dimension 1: Meets some of the criteria in the dimension 0: Does not meet the criteria in the dimension Descriptors for Dimensions I–IV: 3: Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations 2: Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations 1: Developing toward CCSS Quality — needs significant revision, as suggested in criterion-based observations 0: Not representing CCSS Quality — does not address the criteria in the dimension

32. 32 Observations/Feedback and Rating Rating: ? Step 2. Apply Criteria in Dimension I: Alignment Normed Response: Number and Operations in Base Ten

33. 33 Observations/Feedback and Rating This lesson addresses half of the standard for grade four - 4.NBT.B.5. It does provide a depth of understanding for teaching multiplying a 2-digit number by a 2-digit number using arrays. The lesson states that it will take one hour to present, yet further in the lesson, the writer states that the lesson may take more time for students to practice the procedure. Additional time might be considered in the Overview box by C-PALMS, so teachers understand that it is okay to take the time to go deeper. The SMPs addressed in this lesson are #7 and #3. However, SMP 7 was weaker in its alignment, especially in critiquing the work of others. SMP 6 was not listed; however, the reviewers felt the lesson did a good job with this SMP especially with precise academic language. The author included these and not the C- PALMS group. The SMPs are critical in helping to get to the rigor for the lesson. A note to teachers in the specific area where those SMPs are located will help teachers connect to the content to inform their understanding of the standard. The balance between the procedure and conceptual understanding is a strength of this lesson. The teacher helps students get to place value understanding in order to build the procedure of multiplication of numbers greater than one digit. This lesson will lead students to an efficient algorithm for multiplication. Rating: 3 (Meets most to all of the criteria in the dimension) Step 2. Apply Criteria in Dimension I: Alignment Normed Response: Number and Operations in Base Ten Sample

34. 34 Compare Criterion-Based Checks, Observations/Feedback and Rating What is the pattern within our team in terms of the criteria we have checked? Do our observations and feedback reference the criteria and evidence (or lack of evidence) in the instructional materials? Step 2. Apply Criteria in Dimension I: Alignment Sample Lesson: Number and Operations in Base Ten

35. 35 Consider the rating for Dimension I: Is the overall rating for alignment a 3 or 2? Does the quality of the alignment to college- and career- readiness standards warrant continuing with the review? If yes, continue with Dimensions II – IV. Step 2. Apply Criteria in Dimension I: Alignment Sample Lesson: Number and Operations in Base Ten

36. 36 Step 3

37. Five Quality Review Steps Step 3 Step 3. Apply Criteria in Dimensions II–IV Examine the lesson/unit through the “lens” of each criterion Indicate each criterion met, record observations and feedback, and then rate 0–3 37

38. 38 Step 3: Dimension II

39. 39 Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Normed Response: Number and Operations in Base Ten The lesson/unit addresses reflects evidence of key shifts that are reflected in the CCSS:  Focus: Lessons and units targeting the major work of the grade provide an especially in-depth treatment, with especially high expectations. Lessons and units targeting supporting clusters have visible connection to the major work of the grade and are sufficiently brief. Lessons and units do not hold students responsible for material from later grades.  Coherence: The content develops through reasoning about the new concepts on the basis of previous understandings and provides opportunities for students to transfer knowledge and skills within and across domains and learning progressions.  Rigor : Requires students to engage with and demonstrate challenging mathematics with appropriate balance among the following —Application: Provides opportunities for students to independently apply mathematical concepts in real- world situations and problem solve with persistence, choosing and applying an appropriate model or strategy to new situations —Conceptual Understanding: Develops students’ conceptual understanding through tasks, brief problems, questions, multiple representations and opportunities for students to write and speak about their understanding. — Procedural Skill and Fluency : Expects, supports and provides guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately

40. 40 Step 3. Apply Criteria for Dimensions II – IV Criteria for Dimension II: Key Shifts in the CCSS The lesson/unit addresses key shifts in the CCSS:  Focus: Lessons and units targeting the major work of the grade provide an especially in-depth treatment, with especially high expectations. Lessons and units targeting supporting clusters have visible connection to the major work of the grade and are sufficiently brief. Lessons and units do not hold students responsible for material from later grades.  Coherence: The content develops through reasoning about the new concepts on the basis of previous understandings and provides opportunities for students to transfer knowledge and skills within and across domains and learning progressions.

41. 41  Rigor: Requires students to engage with and demonstrate challenging mathematics with appropriate balance among the following: * —Application: Provides opportunities for students to independently apply mathematical concepts in real-world situations and problem solve with persistence, choosing and applying an appropriate model or strategy to new situations —Conceptual Understanding: Develops students’ conceptual understanding through tasks, brief problems, questions, multiple representations and opportunities for students to write and speak about their understanding. —Procedural Skill and Fluency: Expects, supports and provides guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately. * Note: The three components of rigor are not considered/checked separately but only as required by the targeted content standards. Key words: APPROPRIATE BALANCE Step 3. Apply Criteria for Dimensions II – IV Criteria for Dimension II: Key Shifts in the CCSS Now, let’s try Step 3 –Dim II.

42. 42 The lesson/unit addresses reflects evidence of key shifts that are reflected in the CCSS:  Focus: … Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Normed Response: Number and Operations in Base Ten

43. 43 The lesson/unit addresses reflects evidence of key shifts that are reflected in the CCSS: Focus: This lesson does target a major standard for 4th grade. The depth that the lesson goes to with tying the place value build first to the algorithm of partial products is especially well done. The students are held accountable for only the work at their grade level. Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Normed Response: Number and Operations in Base Ten Sample

44. 44 The lesson/unit addresses reflects evidence of key shifts that are reflected in the CCSS:  Coherence: … Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Normed Response: Number and Operations in Base Ten

45. 45 Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Normed Response: Number and Operations in Base Ten Coherence: Coherence is addressed in the section titled Prior Knowledge. The standards are tied to grade three through 3.OA.B.5 and 3.NBT.A.3. There is no scaffold for a tie to grade 5. A suggestion for C-PALMS would be to add a section for writers to include coherence in the Overview box. Sample

46. 46  Rigor: Requires students to engage with and demonstrate challenging mathematics with appropriate balance among the following: *  Application: … Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Normed Response: Number and Operations in Base Ten * NOTE: Norming for this lesson began early in the life of the EQuIP review process. Earlier versions of the rubric allowed for separate checks for the three components of rigor.

47. 47 Rigor: Requires students to engage with and demonstrate challenging mathematics with appropriate balance among the following: * —Application: This lesson includes application problems that provide opportunities for students to independently apply mathematical concepts in real-world situations and problem solve with persistence. It prepares students to be able to choose and apply the appropriate model or strategy to new situations. The teacher guidance clarifies that the goal of application problems in this lesson is to move students to use an area model. To allow for application to mean more for the students, teachers are encouraged to substitute their student names in the problems. Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Normed Response: Number and Operations in Base Ten * NOTE: Norming for this lesson began early in the life of the EQuIP review process. Earlier versions of the rubric allowed for separate checks for the three components of rigor. Sample

48. 48  Rigor: Requires students to engage with and demonstrate challenging mathematics with appropriate balance among the following: * —Application: —Conceptual Understanding: The …. Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Normed Response: Number and Operations in Base Ten * NOTE: Norming for this lesson began early in the life of the EQuIP review process. Earlier versions of the rubric allowed for separate checks for the three components of rigor.

49. 49 Rigor: Requires students to engage with and demonstrate challenging mathematics with appropriate balance among the following: * —Application: —Conceptual Understanding: The lesson includes concept development that builds students’ understanding through the guided practice. Students are not allowed to use vocabulary shortcuts such as 14 times 12; instead, they are instructed to say 14 groups of 12 to ensure the development of conceptual understanding. Reinforcing the concept through vocabulary is a strength of this lesson. Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Normed Response: Number and Operations in Base Ten * NOTE: Norming for this lesson began early in the life of the EQuIP review process. Earlier versions of the rubric allowed for separate checks for the three components of rigor. Sample

50. * NOTE: Norming for this lesson began early in the life of the EQuIP review process. Earlier versions of the rubric allowed for separate checks for the three components of rigor. 50  Rigor: Requires students to engage with and demonstrate challenging mathematics with appropriate balance among the following: * —Application: —Conceptual Understanding: —Procedural Skill and Fluency: … Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Normed Response: Number and Operations in Base Ten

51. * NOTE: Norming for this lesson began early in the life of the EQuIP review process. Earlier versions of the rubric allowed for separate checks for the three components of rigor. 51 Rigor: Requires students to engage with and demonstrate challenging mathematics with appropriate balance among the following: * —Application: —Conceptual Understanding: —Procedural Skill and Fluency : The lesson supports the development of procedural skill and fluency by allowing students to practice the array model until they are comfortable. As students become comfortable using the procedure with the manipulatives, those are then removed and students are allowed to draw a picture. Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Normed Response: Number and Operations in Base Ten Sample

52. 52 Observations/Feedback and Rating The lesson…. Rating: ? Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Normed Response: Number and Operations in Base Ten

53. 53 Observations/Feedback and Rating This lesson allows and balances application, conceptual understanding, and fluency. There are opportunities for students to speak about their understanding as they explain their solutions, if the teacher chooses to allow students to do the explaining. The option for the teacher to explain is stated, which would lower the rigor of the lesson and the ability for the teacher to assess understanding. It might be helpful to provide some strategies for selecting students to explain their solutions. Students are also asked to explain to the teacher as they complete the practice problems. However, they are not asked to write about their understanding, other than to show the arrays and the partial products in their work. Rating: 2 (Meets many of the criteria in the dimension) Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Normed Response: Number and Operations in Base Ten Sample

54. 54 Compare Criterion-Based Checks, Observations/Feedback and Rating What is the pattern within our team in terms of the criteria we have checked? Do our observations and feedback reference the criteria and evidence (or lack of evidence) in the instructional materials? Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS Sample Lesson: Number and Operations in Base Ten

55. 55 Step 3: Dimension III

56. 56 The lesson/unit is responsive to varied student learning needs:  Includes clear and sufficient guidance to support teaching and learning of the targeted standards, including, when appropriate, the use of technology and media  Uses and encourages precise and accurate mathematics, academic language, terminology, and concrete or abstract representations (e.g., pictures, symbols, expressions, equations, graphics, models) in the discipline  Engages students in productive struggle through relevant, thought-provoking questions, problems and tasks that stimulate interest and elicit mathematical thinking  Addresses instructional expectations and is easy to understand and use  Provides appropriate level and type of scaffolding, differentiation, intervention and support for a broad range of learners Supports diverse cultural and linguistic backgrounds, interests and styles Provides extra supports for students working below grade level Provides extensions for students with high interest or working above grade level A longer unit or lesson should:  Recommends and facilitates a mix of instructional approaches for a variety of learners such as using multiple representations (including models), using a range of questions, checking for understanding, flexible grouping, pair-share, etc.  Gradually removes supports, requiring students to demonstrate their mathematical understanding independently  Demonstrates an effective sequence and a progression of learning where the concepts or skills advance and deepen over time  Expects, supports and provides guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten Now, let’s try Step 3 – Dim. III

57. 57 Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten The lesson/unit is responsive to varied student learning needs:  Includes clear and sufficient guidance to support teaching and learning of the targeted standards, including, when appropriate, the use of technology and media  Uses and encourages precise and accurate mathematics, academic language, terminology, and concrete or abstract representations (e.g., pictures, symbols, expressions, equations, graphics, models) in the discipline  Engages students in productive struggle through relevant, thought-provoking questions, problems and tasks that stimulate interest and elicit mathematical thinking  Addresses instructional expectations and is easy to understand and use

58. 58  Provides appropriate level and type of scaffolding, differentiation, intervention and support for a broad range of learners Supports diverse cultural and linguistic backgrounds, interests and styles Provides extra supports for students working below grade level Provides extensions for students with high interest or working above grade level Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten

59. 59 A unit or longer lesson should: longer lessons:  Recommends and facilitates a mix of instructional approaches for a variety of learners such as using multiple representations (including models), using a range of questions, checking for understanding, flexible grouping, pair-share, etc.  Gradually removes supports, requiring students to demonstrate their mathematical understanding independently  Demonstrates an effective sequence and a progression of learning where the concepts or skills advance and deepen over time  Expects, supports and provides guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten

60. 60 The lesson/unit is responsive to varied student learning needs:  Includes clear and sufficient guidance to support teaching and learning of the targeted standards, including, when appropriate, the use of technology and media The lesson… Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten

61. 61 The lesson/unit is responsive to varied student learning needs: Includes clear and sufficient guidance to support teaching and learning of the targeted standards, including, when appropriate, the use of technology and media The writer of this lesson includes clear directions for teaching the lesson along with guidance through the use of actual student examples (see an example; Array Picture 1-4;, etc.). The writer also warns teachers: "The Independent Practice may need to be completed on another day." This helps encourage teachers to slow down to make sure students understand the concept. Also included was the use of a segment from The Teaching Channel on "My Favorite No" to help teachers see and hear how to use this as a formative assessment. Document cameras or overhead projectors are the technology component of the lesson.. Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten Sample

62. 62 The lesson/unit is responsive to varied student learning needs:  Uses and encourages precise and accurate mathematics, academic language, terminology, and concrete or abstract representations (e.g., pictures, symbols, expressions, equations, graphics, models) in the discipline The language used …. Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten

63. 63 The lesson/unit is responsive to varied student learning needs: Uses and encourages precise and accurate mathematics, academic language, terminology, and concrete or abstract representations (e.g., pictures, symbols, expressions, equations, graphics, models) in the discipline. The use of vocabulary especially in relationship to place value is an important part of the lesson. The writer states, "Do not allow short cuts in vocabulary...." Also, the writer states, "As students begin to explain their strategies, encourage complete and specific explanations." Concrete representations are built through the use of base 10 blocks. Students are encouraged to move to pictures, "As students show they understand, you can let them draw the pieces rather than use the blocks." The use of equations is built into the concept after the picture is made, and the discussion occurs as to what each part of the product represents. Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten Sample

64. 64 The lesson/unit is responsive to varied student learning needs:  Engages students in productive struggle through relevant, thought-provoking questions, problems and tasks that stimulate interest and elicit mathematical thinking The lesson… Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten

65. 65 The lesson/unit is responsive to varied student learning needs: Engages students in productive struggle through relevant, thought-provoking questions, problems and tasks that stimulate interest and elicit mathematical thinking The writer states, "Let the children struggle, talk with one another, and create some possibilities. You will need to assess the frustration in the room. Offer hints or stop the exploration and discuss as a class, if you feel the students are too confused." Students are asked to be able to explain what each part of the array represents, why you add the partial products and not multiply, and compare/contrast arrays that have no numbers but physical models. It would be helpful to describe specific student behaviors that indicate "too confused" in order to help teachers unaccustomed to allowing students to struggle not to stop too early. Also, a more thorough discussion of how the guiding questions might be used during each stage of the lesson would be helpful.. Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten Sample

66. 66 The lesson/unit is responsive to varied student learning needs:  Addresses instructional expectations and is easy to understand and use Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten

67. 67 The lesson/unit is responsive to varied student learning needs: Addresses instructional expectations and is easy to understand and use Aspects needed in a quality lesson are used - whole group and individual practice. The one suggestion to make here is the idea that the students present their arrays OR the teacher shows; default to the students sharing so the teacher will know what a student understands and is able to do. Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten Sample

68. 68  Provides appropriate level and type of scaffolding, differentiation, intervention and support for a broad range of learners - Supports diverse cultural and linguistic backgrounds, interests and styles - Provides extra supports for students working below grade level - Provides extensions for students with high interest or working above grade level The lesson…. Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten

69. 69 Provides appropriate level and type of scaffolding, differentiation, intervention and support for a broad range of learners - Supports diverse cultural and linguistic backgrounds, interests and styles - Provides extra supports for students working below grade level - Provides extensions for students with high interest or working above grade level The differentiation is listed in the section titled Accommodations and Recommendations. Students who may be struggling, those who need ELL help and Extensions are given with a brief explanation of what to provide for each. Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten Sample

70. 70 A unit or longer lesson should: longer lessons:  Recommends and facilitates a mix of instructional approaches for a variety of learners such as using multiple representations (including models), using a range of questions, checking for understanding, flexible grouping, pair-share, etc.  Gradually removes supports, requiring students to demonstrate their mathematical understanding independently  Demonstrates an effective sequence and a progression of learning where the concepts or skills advance and deepen over time  Expects, supports and provides guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operation in Base Ten

71. 71 Observations/Feedback and Rating Rating: ? Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten

72. 72 Observations/Feedback and Rating The lesson is responsive to varied student learning needs. There is sufficient guidance to support the targeted standards and conceptual understanding is built into the lesson. This lesson allows for students to struggle through problems and mathematical thinking is elicited. Rating: 3 (Meets most to all of the criteria in the dimension) Step 3. Apply Criteria in Dimension III: Instructional Supports Normed Response: Number and Operations in Base Ten Sample

73. 73 Step 3. Apply Criteria in Dimension III: Instructional Supports Compare Criterion-Based Checks, Observations/Feedback and Rating What is the pattern within our team in terms of the criteria we have checked? Do our observations and feedback reference the criteria and evidence (or lack of evidence) in the instructional materials?

74. 74 Step 3: Dimension IV

75. 75 The lesson/unit regularly assesses whether students are mastering standards-based content and skills:  Is designed to elicit direct, observable evidence of the degree to which a student can independently demonstrate the targeted CCSS  Assesses student proficiency using methods that are accessible and unbiased, including the use of grade-level language in student prompts  Includes aligned rubrics, answer keys and scoring guidelines that provide sufficient guidance for interpreting student performance In addition, for units and longer lessons: and longer lessons:  Uses varied modes of curriculum-embedded assessments that may include pre-, formative, summative and self-assessment measures Step 3. Apply Criteria in Dimension IV: Assessment Normed Response: Number and Operations in Base Ten Now, let’s try Step 3 – Dim IV.

76. 76 The lesson/unit regularly assesses whether students are mastering standards-based content and skills:  Is designed to elicit direct, observable evidence of the degree to which a student can independently demonstrate the targeted CCSS  Assesses student proficiency using methods that are accessible and unbiased, including the use of grade-level language in student prompts Step 3. Apply Criteria in Dimension IV: Assessment Normed Response: Number and Operations in Base Ten

77. 77 The lesson/unit regularly assesses whether students are mastering standards-based content and skills: Is designed to elicit direct, observable evidence of the degree to which a student can independently demonstrate the targeted CCSS The assessments with this lesson are designed to elicit direct, observable evidence of the student independently mastering the targeted CCSS. This lesson includes a pre-assessment (labeled as a formative assessment) to assess prior knowledge. Clear directions are given for "My Favorite No" as well as sample student responses. Giving visual samples of work is a strength of this lesson. When teachers can visualize what the answers should look like it helps them check their own understanding of the material. One suggestion would be to include an estimate of the amount of time the teacher should spend on this section of the lesson. Problems are included for the independent practice and the summative assessment. Again, the writer encourages teachers to change the names to increase student motivation. Step 3. Apply Criteria in Dimension IV: Assessment Normed Response: Number and Operations in Base Ten Sample

78. 78 The lesson/unit regularly assesses whether students are mastering standards-based content and skills: Assesses student proficiency using methods that are accessible and unbiased, including the use of grade-level language in student prompts The assessments in this lesson are accessible and unbiased. Grade-level language is used by the teacher, and it is expected students will also use grade-level mathematics language to show understanding of the standard. By stating this expectation, the writer is getting students to understand the depth of the standard. Step 3. Apply Criteria in Dimension IV: Assessment Normed Response: Number and Operations in Base Ten Sample

79. 79  Includes aligned rubrics, answer keys and scoring guidelines that provide sufficient guidance for interpreting student performance In addition, for units and longer lessons:  Uses varied modes of curriculum-embedded assessments that may include pre-, formative, summative and self-assessment measures Step 3. Apply Criteria in Dimension IV: Assessment Normed Response: Number and Operations in Base Ten

80. 80 Includes aligned rubrics, answer keys and scoring guidelines that provide sufficient guidance for interpreting student performance Answer keys are included for the independent practice and the summative assessment. A suggestion for improvement would be to include the arrays on the answer sheets instead of just the answer to the multiplication problems. This would be beneficial for a new or struggling elementary math teacher. Some of the answer sheets have the answers in red; it would be beneficial if all the answer sheets did this, as well. In addition, for units and longer lessons:  Uses varied modes of curriculum-embedded assessments that may include pre-, formative, summative and self-assessment measures Step 3. Apply Criteria in Dimension IV: Assessment Normed Response: Number and Operations in Base Ten Sample

81. 81 Observations/Feedback and Rating Rating: ? Step 3. Apply Criteria in Dimension IV: Assessment Normed Response: Number and Operations in Base Ten

82. 82 Observations/Feedback and Rating During part of the formative assessment the teacher is checking students' work during the independent practice, it might save the teacher "racing around the room" if students had the opportunity to self or peer-assess during this phase of instruction. Rating: 3 (Meets most to all of the criteria in the dimension.) Step 3. Apply Criteria in Dimension IV: Assessment Normed Response: Number and Operations in Base Ten Sample

83. 83 Compare Criterion-Based Checks, Observations/Feedback and Rating What is the pattern within our team in terms of the criteria we have checked? Do our observations and feedback reference the criteria and evidence (or lack of evidence) in the instructional materials? Step 3. Apply Criteria in Dimension IV: Assessment Sample Unit: Number and Operations in Base Ten

84. 84 Step 4

85. Five Quality Review Steps Step 4 Step 4. Apply an Overall Rating and Provide Summary Comments Individually review ratings for Dimensions I–IV, adding/clarifying comments as needed Total dimension ratings and record an overall rating (E, E/I, R, N) based on total score Individually write summary comments for the overall rating on the Quality Review Rubric PDF 85

86. 86 Step 4. Apply an Overall Rating and Provide Summary Comments Overall Rating for the Lesson/Unit: E: Exemplar — Aligned and meets most to all of the criteria in dimensions II–IV (total 11–12) E/I: Exemplar if Improved — Aligned and needs some improvement in one or more dimensions (total 8–10) R: Revision Needed — Aligned partially and needs significant revision in one or more dimensions (total 3–7) N: Not Ready to Review — Not aligned and does not meet criteria (total 0–2) Individually consider the overall rating for this unit.

87. 87 Using Overall Ratings to Summarize Judgment Overall Rating for the Lesson/Unit: E: Exemplar — Aligned and meets most to all of the criteria in Dimensions II–IV (total 11–12) E/I: Exemplar if Improved — Aligned and needs some improvement in one or more dimensions (total 8–10) R: Revision Needed — Aligned partially and needs significant revision in one or more dimensions (total 3–7) N: Not Ready to Review — Not aligned and does not meet criteria (total 0–2)

88. Step 4. Provide Summary Comments Normed Response: Number and Operations in Base Ten 88 RATING:

89. Step 4. Provide Summary Comments Normed Response: Number and Operations in Base Ten 89 Sample Number and Operations in Base Ten 4 Rating: E This lesson is an example for writing individual lessons for the CCSS. The lesson has a specific standard it targets; within that standard it is narrowed for the lesson to allow for procedural fluency for the strategy of using arrays to build an area model. Vocabulary is used expertly to help students understand conceptually the standard for multiplying two-digit numbers by a two-digit number. If used by teachers with a limited understanding and experience in promoting productive struggle in students, additional guidance might be needed. C-PALMS might want to consider including in their Overview box the specified vocabulary contained within this lesson. Are Key Words used as a search engine to find lessons or is "key words" used as a term for vocabulary from the lesson that teachers should focus on? (The assumption by the reviewers is that Key Words are being used as a search engine.) Assessment focuses not only on the array being used to find the answer, but on the real-world context for the use of multiplication by students.

90. 90 Step 5

91. Five Quality Review Steps Step 5. Compare Overall Ratings and Determine Next Steps Note the evidence cited to arrive at final ratings, summary comments, and similarities and differences among raters. Recommend next steps for the lesson/unit and provide recommendations for improvement to developers/teachers. 91

92. 92 Step 5. Compare Summary Comments and Determine Next Steps Guiding questions to compare overall ratings and synthesize criterion-based observations and suggestions: How do our overall ratings compare? Does this example serve as a model of CCSS instruction? What are its strengths? Areas for improvement?

93. 93 The EQuIP Rubric for Mathematics – The 1-Page Version

94. 94 Let’s Practice!

95. In table groups: —Use the rubric to evaluate the lesson —Adding a two-digit and a one-digit number using ten frames

96. 96 Share your evaluation with others. How are they alike? Different?

97. 97 What resources do you currently use that you could evaluate using this rubric?

98. Use the rubric as a tool to support creation of new lessons and units. Use the rubric to review lesson templates and perhaps to create a new one. Use the rubric to review materials teachers are using. Support teachers by using the rubric to validate the materials they are using or have developed. Use the process to encourage discussion among grade level teams regarding the quality curriculum resources. For teachers who are creating lessons/units that will affect their evaluations, the process provides some very clear components for a quality lesson. How NC Participants Might Use the Quality Review Rubric and/or Review Process

99. 99 Explore what effective observations and criterion- based feedback look like Experience the process of reviewing instructional materials www.achieve.org/EQuIP EQuIP e-Learning Modules

100. 100 Reflection on Session Goals Did we … Develop your ability to use EQuIP criteria to provide observations about CCSS-aligned instructional materials and make suggestions for improvement? Develop a common understanding of the EQuIP quality review process? Develop a common understanding of the rating scale and descriptors for the four rubric dimensions and the rating categories and descriptors for overall ratings? Develop your ability to use EQuIP criteria, rating scales and rating descriptors to accurately rate instructional materials?

101. What questions do you have?

102. DPI Mathematics Section Kitty Rutherford Elementary Mathematics Consultant 919-807-3841 kitty.rutherford@dpi.nc.gov Denise Schulz Elementary Mathematics Consultant 919-807-3839 denise.schulz@dpi.nc.gov Johannah Maynor Secondary Mathematics Consultant 919-807-3842 johannah.maynor@dpi.nc.gov Lisa Ashe Secondary Mathematics Consultant 919-807-3934 lisa.ashe@dpi.nc.gov Dr. Jennifer Curtis K–12 Mathematics Section Chief 919-807-3838 jennifer.curtis@dpi.nc.gov Susan Hart Mathematics Program Assistant 919-807-3846 susan.hart@dpi.nc.gov

103. Follow Us! NC Mathematics www.facebook.com/NorthCarolinaMathematics @ncmathematics http://maccss.ncdpi.wikispaces.net

104. Achieve www.achieve.org/EQuIP 1400 16th Street, NW / Suite 510 Washington, DC 20036 104

105. For all you do for our students!