# (Educators Evaluating Quality Instructional Products)

## Presentation on theme: "(Educators Evaluating Quality Instructional Products)"— Presentation transcript:

(Educators Evaluating Quality Instructional Products)

The History Behind the Tool

Purpose Instructional Materials Guidance Lesson/Unit Alignment

The EQuIP Rubric Design

How States May Use The Rubric

EQuIP Rubric Guide for lesson plan/unit alignment

The Rubric Organizes Criteria That Describe Quality Lessons/Units
Criteria that define the rubric are organized to describe quality in four dimensions. Dimension #1 Alignment to the Depth of CCSS Dimension #2 Key Shifts of Focus in the CCSS Dimension #3 Instructional Supports Dimension #4 Assessment Dimension 1: Alignment to depth of CCSS Dimension 2: Key Shifts in the CCSS Dimension 3: Instructional Supports Dimension 4: Assessments: pre-assessment, self-assessment, formative, summative The descriptive criteria listed in each Dimension/column represent a high standard of quality which describes the characteristics one would find in an exemplary CCSS lesson or unit. Slide taken from Tri-State Quality Review Rubric & Process (Achieve)

CCSS Alignment is Four Dimensional

1st Step: Make a foldable Show only Dimensions I And IV

Backward Planning Goals Assessment Methods

Backwards Design/EQuIP Rubric

Dimension I: Alignment to the CCSS
Targets Standard(s) Content standards Practice Standards Balance Procedural Skill Conceptual Understanding Remind participants that the first dimension of the EQuIP Rubric is focused on alignment to the Common Core State Standards. In Mathematics there are some specific attributes that need to be present. Dimension I: Alignment Dimension I is non-negotiable. In order for the review to continue, a rating of 2 or 3 is required. If the review is discontinued, consider general feedback that might be given to developers regarding next steps. To apply the criteria in Dimension I it is helpful to ask the following clarifying questions regarding criteria 1, 2, and 3: For criterion 1 – Does the lesson/unit articulate alignment to a reasonable standard or set of standards? Do the assignments, tasks, and activities suggest that a standard or set of standards has been targeted for instruction? Does the lesson/unit make a distinction between targeted and supporting standards? Do the assignments and activities make sense given the standards listed? Does the lesson/unit address the targeted standards at the full depth defined in the standards? For criterion 2 – Which Standards for Mathematical Practice are identified? Which mathematical practices do the assignments and activities provide opportunity for students to demonstrate? Is a focused set of mathematical practices identified that are central to the lesson/unit? Are the mathematical practices tied closely to the content of the lesson or unit? For criterion 3 – Do the assignments and activities provide opportunities for students to practice mathematical procedures and deepen the emphasized concepts with a balance appropriate for the content and for the grade level?

Dimension II: Key Shifts in CCSS

Dimension III: Instructional Supports

Dimension IV: Assessment
Observable Evidence of Learning Assesses Proficiency Aligned Rubrics/Scoring Guides Remind participants that the fourth dimension of the rubric is focused on assessments. There are specific attributes that need to be present in this dimension: Dimension IV: Assessment To apply the criteria for Dimension IV it is helpful to ask the following questions regarding criteria 1, 2, and 3: For criterion 1 – Does the lesson/unit provide opportunities for students to independently demonstrate their understanding? For criterion 2 – Do students have multiple ways to show what they have learned? For criterion 3 – Is there evidence that the assessments produce a description of how close students have come to meeting expectations (e.g., annotated student work, descriptive rubrics/checklists)? Possible activities with this dimension: 1. Participants can highlight in yellow what teachers will feel comfortable and in blue what teachers will feel is challenging. 2. Participants can highlight the vocabulary within the rubric that teachers may need to discuss and come to consensus on what those words/phrases mean.

Lesson/Unit Analysis 1. Look at your unit/multi-day lesson

Rubric Activity Multi Day Lesson/ Unit Plan

Criteria- Dimension I The lesson/unit aligns with the letter and spirit of the CCSS: Targets a set of grade-level CCSS mathematics standard(s) to the full depth of the standards for teaching and learning. Standards for Mathematical Practice that are central to the lesson are identified, handled in a grade-appropriate way and well connected to the content being addressed. Presents a balance of mathematical procedures and deeper conceptual understanding inherent in the CCSS. We need to quickly refresh our memories of the criteria for each dimension before we discuss the unit. The slides that follow will show the normed response checks and comments.

Step 2 – Share Normed Responses - Dimension I First Grade – Exploring Two-Digit Numbers
The lesson/unit aligns with the letter and spirit of the CCSS: Targets a set of grade-level CCSS mathematics standard(s) to the full depth of the standards for teaching and learning. Standards for Mathematical Practice that are central to the lesson are identified, handled in a grade-appropriate way and well connected to the content being addressed. Presents a balance of mathematical procedures and deeper conceptual understanding inherent in the CCSS.

Step 2 – Share Normed Responses - Dimension I First Grade – Exploring Two-Digit Numbers

Step 2 – Share Normed Responses - Dimension I First Grade – Exploring Two-Digit Numbers
Standards for Mathematical Practice that are central to the lesson are identified, handled in a grade-appropriate way and well connected to the content being addressed. Observations and suggestions: All eight Standards for Mathematical Practice are cited in the introduction as being addressed within this 10-lesson unit. Then each lesson cites a somewhat shorter list of specific Mathematical Practices targeted, with from three to six Practices targeted in each lesson. This more specific targeting is a good idea but most teachers would find it challenging to focus on six Practices within one lesson. For example in Lesson 2.1 MP.2, 3, 4, 5, 6, and 7 are all cited as “emphasized” in the approximately one-hour lesson. It is likely that some of these are less central to the goals of the lesson than others. Developers should consider whether the lesson-level lists of Practices could be honed even further and whether listing all eight of the Practices in the introduction is at all helpful for teachers. Focusing only on those Practices that are CENTRAL to the set of lessons is ideal. To improve this unit, and the lessons within it, clear references and connections to the Mathematical Practices should be noted within activities. Such explicit references will help teachers understand when and how to emphasize various Practices. (Note: While some evidence of this criterion is apparent in these materials, there is not clear and substantial evidence that the criterion is met.) Same not as previous slide: This is a partial alignment with the criterion and the check v no-check issue is not as important as the commentary being in agreement.

Step 2 – Share Normed Responses - Dimension I First Grade – Exploring Two-Digit Numbers
Presents a balance of mathematical procedures and deeper conceptual understanding inherent in the CCSS Observations and suggestions: The ten lessons comprising this unit do a good job of building the procedural skills and the conceptual understanding of students with respect to 2-digit numbers and place value. Conceptual understanding is built using a variety of tools, including ten-frame mats, counters, snap cubes, and hundred boards. Students are expected to be able to represent numbers in a variety of ways and to compare numbers based on their understanding of the tens and ones digits. The lessons include questions throughout that will guide teachers in gauging both the procedural and conceptual understandings of students.

Criteria for Dimension II: Key Shifts in the CCSS
The lesson/unit addresses key shifts in the CCSS: Focus: Lessons and units targeting the major work of the grade provide an especially in-depth treatment, with especially high expectations. Lessons and units targeting supporting clusters have visible connection to the major work of the grade and are sufficiently brief. Lessons and units do not hold students responsible for material from later grades. Coherence: The content develops through reasoning about the new concepts on the basis of previous understandings and provides opportunities for students to transfer knowledge and skills within and across domains and learning progressions.

Criteria for Dimension II: Key Shifts in the CCSS
Rigor: Requires students to engage with and demonstrate challenging mathematics with appropriate balance among the following: Application: Provides opportunities for students to independently apply mathematical concepts in real-world situations and problem solve with persistence, choosing and applying an appropriate model or strategy to new situations. Conceptual Understanding: Provides opportunities for students to demonstrate conceptual understanding through challenging problems, questions, and writing and speaking about their understanding. Procedural Skill and Fluency: Expects, supports and provides guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately.

Step 2 – Share Normed Responses – Dimension First Grade – Exploring Two-Digit Numbers II
The lesson/unit addresses reflects evidence of key shifts that are reflected in the CCSS: 1. Focus: Lessons and units targeting the major work of the grade provide an especially in-depth treatment, with especially high expectations. Lessons and units targeting supporting clusters have visible connection to the major work of the grade and are sufficiently brief. Lessons and units do not hold students responsible for material from later grades. 2. Coherence: The content develops through reasoning about the new concepts on the basis of previous understandings and provides opportunities for students to transfer knowledge and skills within and across domains and learning progressions. 3. Rigor: Requires students to engage with and demonstrate challenging mathematics with appropriate balance among the following Application: Provides opportunities for students to independently apply mathematical concepts in real- world situations and problem solve with persistence, choosing and applying an appropriate model or strategy to new situations. Conceptual Understanding: Provides opportunities for students to demonstrate conceptual understanding through challenging problems, questions, and writing and speaking about their understanding. Procedural Skill and Fluency: Expects, supports and provides guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately. Notice that all of the three shifts are represented in this unit. This reinforces our decision to continue with the review. If this dimension was sorely lacking, we might go back and reconsider our 2-rating for Dim I.

Step 2 – Share Normed Responses - Dimension II First Grade – Exploring Two-Digit Numbers
Focus: One of the four critical areas cited in the CCSS for first grade students is developing understanding of whole number relationships and place value, including grouping in tens and ones, to This 10-lesson unit focuses on this major work of the grade, with respect to 2-digit numbers. At the unit level, reconsidering the inclusion of 1.NBT.1, which extends counting, reading and writing numerals, and number representation to 120, and 1.OA.6, which addresses addition and subtraction within 20 and fluency within 10, could sharpen the focus. Even though we checked this criteria, there are suggestions for improvement that echo what was said in Dim I.

Step 2 – Share Normed Responses - Dimension II First Grade – Exploring Two-Digit Numbers
Coherence: The lesson Overview and Background Information for each lesson indicate the prior knowledge that students will be expected to draw upon. For example, Lesson 1.2 cites prior knowledge needed as knowing number names and the count sequence and counting up to 19 objects---which are defined as expectations in the CCSS for Kindergarten. Each lesson within this unit also begins with an activity to engage students, building upon their prior knowledge. As Lesson 1.2 builds to work with larger numbers, it reinforces the students’ understanding that the teen numbers are composed of one ten and some leftovers—a CCSS for Kindergarten but one that is important as a strong foundation for first grade. Coherence within first grade is also evident since the first half of the year focuses on numbers 0-50, with the second half on 0-100, thereby providing students with an opportunity to build on previous understandings.

Step 2 – Share Normed Responses - Dimension II First Grade – Exploring Two-Digit Numbers
Rigor: The intent of this 10-lesson unit is to build conceptual understanding of place value with two digit numbers and a level of procedural skill so that students can represent 2-digit numbers in a variety of ways and also compare 2-digit numbers based on their understanding of place value. It is not the intent of this unit to engage students in real-world problem solving so application is not an element of rigor that should be taken into account in this instance. Students are given opportunities within the lessons to write or speak about their understandings. There are numerous references to student use of journals, and suggested questions probe students to do such things as identify and explain their strategies and explain how they know which number is larger/smaller. Important to note here that the BALANCE is not always EQUAL but rather APPROPRIATE for the content. Pay attention to the intended purpose of the instruction. For example if the primary purpose of a lesson is fluency, we may not expect to also see an emphasis on conceptual understanding. It may also be appropriate in a feedback cycle to remind the developers that applications involving of 2-digit numbers would probably be important at some point in the curriculum.

Criteria for Dimension III: Instructional Supports
The lesson/unit is responsive to varied student learning needs: Includes clear and sufficient guidance to support teaching and learning of the targeted standards, including, when appropriate, the use of technology and media. Uses and encourages precise and accurate mathematics, academic language, terminology, and concrete or abstract representations (e.g., pictures, symbols, expressions, equations, graphics, models) in the discipline. Engages students in productive struggle through relevant, thought- provoking questions, problems and tasks that stimulate interest and elicit mathematical thinking. Addresses instructional expectations and is easy to understand and use. Review the criteria. This dimension has too many criteria to be held on one slide. The slides that follow will divide them up into manageable sized groups.

Criteria for Dimension III: Instructional Supports
Provides appropriate level and type of scaffolding, differentiation, intervention and support for a broad range of learners. Supports diverse cultural and linguistic backgrounds, interests and styles. Provides extra supports for students working below grade level. Provides extensions for students with high interest or working above grade level. Review the criteria

Step 2 – Share Normed Responses - Dimension III First Grade – Exploring Two-Digit Numbers
The lesson/unit is responsive to varied student learning needs: Includes clear and sufficient guidance to support teaching and learning of the targeted standards, including, when appropriate, the use of technology and media. Uses and encourages precise and accurate mathematics, academic language, terminology, and concrete or abstract representations (e.g., pictures, symbols, expressions, equations, graphics, models) in the discipline. Engages students in productive struggle through relevant, thought-provoking questions, problems and tasks that stimulate interest and elicit mathematical thinking. Addresses instructional expectations and is easy to understand and use. Provides appropriate level and type of scaffolding, differentiation, intervention and support for a broad range of learners. Supports diverse cultural and linguistic backgrounds, interests and styles. Provides extra supports for students working below grade level. Provides extensions for students with high interest or working above grade level. Here are the checks for the normed response for the first part of the criteria.

Step 2 – Share Normed Responses - Dimension III First Grade – Exploring Two-Digit Numbers
Includes clear and sufficient guidance to support teaching and learning of the targeted standards, including, when appropriate, the use of technology and media. The lessons within this unit provide guidance to support teaching and learning, including some suggested links to online resources. However, more explicit guidance could be provided to support teachers. The lessons offer suggestions for questions teachers might use to probe student thinking but do not always include sample student responses. These would be helpful in some cases to guide teachers and their instruction. For example, the Explain section of Lesson 2.6 tells teachers—after the Get 100! game has been played by students—to discuss the game and various strategies with students, asking them what transpired during the game and what strategies they used. Sample student responses would be helpful, particularly for less experienced teachers. In some cases, there are references to materials (e.g., the Greater, Less, Equal Cover Up game board in Lesson 2.5), and those materials do not appear as attachments to the lesson. Complete sets of materials, or where to get such materials, would be useful supports for teachers. Notice that this is a partial match with some support but not enough for the normed response to include the check.

Step 2 – Share Normed Responses - Dimension III First Grade – Exploring Two-Digit Numbers
Uses and encourages precise and accurate mathematics, academic language, terminology, and concrete or abstract representations (e.g., pictures, symbols, expressions, equations, graphics, models) in the discipline. The lessons within this unit encourage precise and accurate language, terminology, and representations for both teachers and students. Each lesson overview notes specific vocabulary terms that are important in the lesson. Representations are key elements in many of the lessons, with students using ten-frame maps, hundred boards, snap cubes, and number sentences to represent 2-digit numbers. Engages students in productive struggle through relevant, thought-provoking questions, problems and tasks that stimulate interest and elicit mathematical thinking. The questions offered as suggestions to teachers, and many of the activities, appear to be ones that should be thought provoking and stimulate student interest, if handled appropriately in the classroom. Granted there are no real-world application problems but there are numerous activities and games that should prove to be interesting and thought-provoking for students.

Step 2 – Share Normed Responses - Dimension III First Grade – Exploring Two-Digit Numbers
Addresses instructional expectations and is easy to understand and use. While the lessons generally address the instructional expectations, as stated earlier there are some CCSS expectations listed as unit-level objectives (1.NBT.1 and 1.OA.6) that are not clearly addressed. The format and flow of the unit and lesson are clear and understandable but there are elements that could be improved upon to make the lessons easier to understand and use. Materials lists are sometimes incomplete or unclear. For example, the activities for Lesson 2.2 call for ten-frame cards and counters, while the Materials list only asks for number cards and 10-sided dice; and Lesson 2.7 depends upon Arrow Cards, which are not provided or adequately described. In some cases, activity sheets are referenced but not provided, and in one case (Lesson 1.1) the spinner referenced in the lesson (with words number, 1 more, and 1 less) does not match up with the one attached to the lesson (with words 1 more, 2 more, and 1 less). While this is a no-check criterion, the unit has some components that make the unit easy to understand and use. Deficits seem to outweigh assets in this case, however.

Step 2 – Share Normed Responses - Dimension III First Grade – Exploring Two-Digit Numbers
Provides appropriate level and type of scaffolding, differentiation, intervention and support for a broad range of learners. Each lesson plan contains a section called Plans for Individual Differences. This attempted attention to diverse learners is commendable but the level of detail provided to teachers often is not sufficient. No suggestions are given for working with students from diverse cultural or linguistic backgrounds. The interventions seem to generally call for using smaller numbers or smaller student groupings with struggling students. Suggestions for more adept students usually include using bigger numbers, which often means just moving them ahead to the next lesson. More detail and specific suggestions for addressing individual differences would be helpful to all teachers, but especially to inexperienced teachers. This is no-check is because of a lack of detail rather than complete oversight.

Criteria for Dimension III: Instructional Supports
A unit or lesson should: Recommends and facilitates a mix of instructional approaches for a variety of learners such as using multiple representations (including models), using a range of questions, checking for understanding, flexible grouping, pair-share, etc. Gradually removes supports, requiring students to demonstrate their mathematical understanding independently. Demonstrates an effective sequence and a progression of learning where the concepts or skills advance and deepen over time. Expects, supports and provides guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately. Review the criteria…

Step 2 – Share Normed Responses - Dimension III First Grade – Exploring Two-Digit Numbers
A unit or longer lesson should: Recommend and facilitate a mix of instructional approaches for a variety of learners such as using multiple representations (including models), using a range of questions, checking for understanding, flexible grouping, pair-share, etc. Gradually remove supports, requiring students to demonstrate their mathematical understanding independently. Demonstrate an effective sequence and a progression of learning where the concepts or skills advance and deepen over time. Expect, support and provide guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately. In the normed response this set of criteria are combined with a response written for all four together. In some reviews these are addressed separately. Either method is appropriate.

Step 2 – Share Normed Responses - Dimension III First Grade – Exploring Two-Digit Numbers
Summary of Observations and Suggestions for Improvement for Unit-Level Supports: Unit-level supports are generally strong. Collectively, the lessons provide a mix of instructional approaches, some of these involve various student-grouping strategies including small groups, student pairs, large group, and individual. Other approaches involve using various types of instructional tools and manipulatives—such as ten frame cards, hundred boards, snap cubes---and encouraging students to progress so they can respond with less reliance on manipulatives and tools. Lessons often transition students from doing larger group to small group or pair work, or vice versa. Sometimes it is not clear if students are being transitioned into individual work. For example, in the Elaborate section of Lesson 2.7 teachers give students a 2-digit number and ask them to say the number that is ten more. It is stated that this might be used as a summative assessment, but there is no clear indication whether this is intended as an individual summative activity. The sequence of lessons generally demonstrates a progression of learning (e.g., from using ten-frame mats and counters to using hundred boards). Accuracy is expected, as in Lesson 2.5 where students are asked to accurately mark an appropriate number. While procedural skill is expected in the lessons, the need for fluency (i.e. speed of procedural skills) does not appear to be applicable. The reason for NOT checking “gradual removal of supports” is bolded here.

Step 2 – Share Normed Responses - Dimension III First Grade – Exploring Two-Digit Numbers
The lesson/unit is responsive to varied student learning needs: Includes clear and sufficient guidance to support teaching and learning of the targeted standards, including, when appropriate, the use of technology and media. Uses and encourages precise and accurate mathematics, academic language, terminology, and concrete or abstract representations (e.g., pictures, symbols, expressions, equations, graphics, models) in the discipline. Engages students in productive struggle through relevant, thought-provoking questions, problems and tasks that stimulate interest and elicit mathematical thinking. Addresses instructional expectations and is easy to understand and use. Provides appropriate level and type of scaffolding, differentiation, intervention and support for a broad range of learners. Supports diverse cultural and linguistic backgrounds, interests and styles. Provides extra supports for students working below grade level. Provides extensions for students with high interest or working above grade level. A unit or longer lesson should: Recommend and facilitate a mix of instructional approaches for a variety of learners such as using multiple representations (including models), using a range of questions, checking for understanding, flexible grouping, pair-share, etc. Gradually remove supports, requiring students to demonstrate their mathematical understanding independently. Demonstrate an effective sequence and a progression of learning where the concepts or skills advance and deepen over time. Expect, support and provide guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately. This slide allows us to take a step back and look at the pattern of checks for the dimension as a whole. We see that the checks out number the no-checks, but that is less important than the weight of the no-checks. The missing components of teacher support are crucial to the overall quality of this unit and would require significant revision.

Criteria for Dimension IV: Assessment
The lesson/unit regularly assesses whether students are mastering standards-based content and skills: Is designed to elicit direct, observable evidence of the degree to which a student can independently demonstrate the targeted CCSS. Assesses student proficiency using methods that are accessible and unbiased, including the use of grade-level language in student prompts. Includes aligned rubrics, answer keys and scoring guidelines that provide sufficient guidance for interpreting student performance. A unit or longer lesson should: Uses varied modes of curriculum-embedded assessments that may include pre-, formative, summative and self-assessment measures. Review the criteria.

Step 2: Share Normed Responses - Dimension IV First Grade – Exploring Two-Digit Numbers
Dimension IV: Assessment The lesson/unit regularly assesses whether students are mastering standards-based content and skills: Is designed to elicit direct, observable evidence of the degree to which a student can independently demonstrate the targeted CCSS. Assesses student proficiency using methods that are unbiased and accessible to all students. Includes aligned rubrics or assessment guidelines that provide sufficient guidance for interpreting student performance. In addition, for units: Uses varied modes of assessment, including a range of pre-, formative, summative, and self-assessment measures. This is the normed response to the criteria. While there is very little attention paid in the unit to assessment, it is not true that there is NO attention paid. In this normed response the suggestions for varied modes of assessment is the only check.

Step 2: Share Normed Responses - Dimension IV First Grade – Exploring Two-Digit Numbers
Is designed to elicit direct, observable evidence of the degree to which a student can independently demonstrate the targeted CCSS. It is not clear that the assessments described in the lessons are designed to elicit direct, observable evidence of the degree to which a student can independently demonstrate understanding of the concepts required in the targeted CCSS. Questions to give formative feedback while observing students are prevalent in the Evaluation of Student Performance components of the lessons; sometimes it is noted that an activity sheet can be used for summative performance (as in Lesson 1.2) but even then it is not clear that this summative activity will provide insight as to whether a student can independently demonstrate understanding. It is also noted on occasion that student math journal work can be collected as a summative assessment (e.g., for Lesson 2.2) but the work in the math journal might be the result of individual work or group work. Assesses student proficiency using methods that are accessible and unbiased, including the use of grade-level language in student prompts. With no formal assessment instruments included with the lessons or unit, it is not possible to comment on accessibility or bias.

Step 2: Share Normed Responses - Dimension IV First Grade – Exploring Two-Digit Numbers
Includes aligned rubrics, answer keys and scoring guidelines that provide sufficient guidance for interpreting student performance. There are no rubrics, answer keys, or scoring guidelines provided to help teachers gauge and interpret student performance. In addition, for units and longer lessons: 4. Uses varied modes of curriculum-embedded assessments that may include pre-, formative, summative and self-assessment measures. All the lessons in this unit contain a section on Evaluation of Students, and the Overview of Lessons shows that this section is to describe both formative assessment (i.e., how the teacher can assess student learning during the lesson) and summative assessment (i.e., how the teacher can assess students’ understanding after the lesson). No mention is given to student self-assessment or to pre-assessment.

Where to Access Sample Lesson Plans/Units

EQuIP Resources Optional Lesson Plan Template Grades K-12