Building & Evaluating Rubrics 1 UL Louisville Writing Project KY ASCD Learn, Teach, Lead LWP Mini-Conference, 2014 Carol Franks, LWP, KYASCD, KDE Effectiveness Coach Rebecca Woosley, KDE Effectiveness Coach

Making Connections: Rubrics & Student Growth Goal Setting within the Teacher Professional Growth and Effectiveness System (TPGES) 2

Student Growth Goal-Setting Student growth goals (SGG) focus on needs of current students SGG focus on enduring skill(s) (ES) identified in the standards Teachers define proficiency for the ES using the grade level standards 3

Defining ENDURING Learning that ENDURES beyond a single test date, is of value in other disciplines, is relevant beyond the classroom (applying learning to new and unique situations) is worthy of embedded, course-long focus, may be necessary for the next level of instruction. Requires critical thinking (analyzing, creating and evaluating)

Rubrics and Student Growth Goal-Setting An analytic rubric can help teachers combine multiple sources of data to – determine a baseline score for goal-setting – determine if students met the goal at the end of the course, and – formatively assess, provide feedback, and adjust instruction along the way 5

Our Targets I can evaluate rubrics for quality. I can design a quality rubric congruent to expectations of grade level standards. I understand how rubric development and use supports the student growth goal- setting process. 6

Rubric Defined “Think of a fully developed rubric as a description of levels of understanding, proficiency, or quality along a continuum or scale.” --Jay McTighe “At their very best, rubrics are teaching tools that support student learning and the development of sophisticated thinking skills.” --Heide Goodrich Andrade 7

Purpose of Rubrics “ When the intended learning outcomes are best indicated by performances—things students would do, make, say, or write—then rubrics are the best way to assess them.” --Susan Brookhart “About the only kinds of schoolwork that do not function well with rubrics are questions with right or wrong answers.” --Susan Brookhart 8

Resources Jay McTighe’s Lumibook on PD360, Assessing What Matters Most (2013) (chapter 6) Stiggins, Arter, Chappuis & Chappuis, Classroom Assessment for Student Learning: Doing it Right – Using it Well (2006) (chapter 7) Brookhart, Susan (2013) How to Create and Use Rubrics for Formative Assessment and Grading Andrade, Heidi G. Using Rubrics to Promote Thinking and Learning(Feb 2000) Educational Testing Service, Creating & Recognizing Quality Rubrics CD (2006) Analytic Rubrics, DePaul University Teaching Commons 9

Holistic vs. Analytical Holistic Rubrics -provide an overall description of quality -used for summative assessment -performance levels used to identify a score -cannot be used for descriptive feedback 10

The Kentucky on-demand scoring rubric is a Holistic Rubric. 11

Holistic vs. Analytical Holistic Rubrics -provide an overall description of quality -used for summative assessment -no performance levels -cannot be used for descriptive feedback Analytical Rubrics -provide description of quality -used for formative and summative assessment -descriptive performance levels for each criteria -can be used for descriptive feedback 12

Scoring criteria Performance levels Descriptive language The LDC scoring rubric is an analytical Rubric. 13

Activity: What should a rubric do? At your table, look over the sample rubrics. List what a rubric should do. 14

What rubrics should do (CASL, pg.200) Define quality for ourselves. Describe quality for students. Make judgments more objective, consistent, accurate. Guide instructions Provide a common language. Promote descriptive feedback to students. Promote student self- assessment and goal- setting. Make expectations for students explicit. Eliminate bias. Focus teaching. Track student learning. 15

Purpose: Why use analytical rubrics? Analytical rubrics: Define quality Define proficiency at a level expected in standards Provide clear expectations Can be used for descriptive feedback to students Aid the formative process 16

Tool for Evaluating Instructional Rubrics 17

Activity: Applying the Evaluating Instructional Rubrics Tool You have: A LDC argumentative rubric Each criteria from the rubric tool on a strip of paper With your table team: Read one criteria strip at a time. Describe how the LDC rubric meets the criteria. 18

Congruency to Standards The description of proficiency is congruent with the level of rigor intended for the standard(s) being assessed. The criteria included in the rubric are important to defining all aspects of proficiency. 19

Reliability & Validity The rubric is appropriate for the skills/tasks it is used to assess; it can be used to assess what it is intended to assess. The rubric leads to the same or similar scores regardless of scorer. 20

A best practice: Inter-rater reliability With a team, check your understanding of the rubric Individually, score a set of student work Compare with team members to check the alignment of your scores Discuss any differences in scoring – Does the rubric need clarification? You should reach the same score regardless of scorer 21

Tool for Evaluating Instructional Rubrics 22

The Problem with Rubrics Divide the common problems at your table. Read independently. Share out by describing the problem and the “bottom line.” 23

Rubric Design: A Recursive Process 24

My example: From Enduring Skill to Draft Rubric Enduring Skill from Reading Anchor Standard 10: Read and comprehend text at the high end of grade-level complexity. 25 Note: You may choose more than one enduring skill. GL Standards Connections: RF 3.4a RI 3.1 RF 3.4c RI 3.2

My example: From Enduring Skill to Draft Rubric RF 3.4a Read grade-level text with purpose and understanding. RF 3.4c Use context to confirm or self-correct word recognition and understanding, rereading as necessary RI 3.1 Ask and answer questions to demonstrate understanding of a text, referring explicitly to the text as the basis for the answers RI 3.2 Determine the main idea of a text; recount the key details and explain how they support the main idea 26 GL Standards Connections: RF 3.4a RI 3.1 RF 3.4c RI 3.2

Below grade Read & comprehend below GL text Below GL text with prompting & support “ Retell details w/o explaining how they support the main idea Above Grade Independently read & comprehend above GL text... in above GL text.. and explain inferences... and summarize text.. and explain multiple main ideas of a text 27 Proficiency on grade level INDEPENDENTLY - Read & understand GL text. Read w/purpose. Use context for word recognition, to understand text. Reread to comprehend. Refer explicitly to text to answer questions. Determine main idea. Recount key ideas & details; explain how they support main idea.

Below grade Read & comprehend below GL text Below GL text with prompting & support “ Retell details w/o explaining how they support the main idea Above Grade Independently read & comprehend above GL text... in above GL text.. and explain inferences... and summarize text.. and explain multiple main ideas of a text 28 Proficiency on grade level INDEPENDENTLY - Read & understand GL text. Read w/purpose. Use context for word recognition, to understand text. Reread to comprehend. Refer explicitly to text to answer questions. Determine main idea. Recount key ideas & details; explain how they support main idea.

Below grade Read & comprehend below GL text Below GL text with prompting & support “ Retell details w/o explaining how they support the main idea Above Grade Independently read & comprehend above GL text... in above GL text.. and explain inferences... and summarize text.. and explain multiple main ideas of a text 29 Proficiency on grade level INDEPENDENTLY - Read & understand GL text. Read w/purpose. Use context for word recognition, to understand text. Reread to comprehend. Refer explicitly to text to answer questions. Determine main idea. Recount key ideas & details; explain how they support main idea.

Define the other Performance Levels Ensure parallel and concise language For below Proficiency, consider: – Previous grades’ expectations – What the standard assumes students already know – Continuum of learning/support for scaffolding – Language that represents less than quality What’s beyond proficiency? 30

Refine the Rubric Using Student Work Test your rubric with student work Continue to refine your rubric 31

A checkpoint for quality “The content has the “ring of truth”—your experience as a teacher confirms that the content is truly what you do look for when you evaluate the quality of a student performance or product. In fact, the rubric is insightful: it helps you organize your own thinking about what it means to perform well.” --Creating & Recognizing Quality Rubrics, Educational Testing Service 32

Engaging Students Students rewrite rubrics in student-friendly language Students use models to identify criteria for quality Students analyze poor models using the rubric and identify how to improve Students develop rubrics Students use rubrics to provide peer feedback 33

Engaging Students “ Effective rubrics show students how they will know to what extent their performance passes muster on each criterion of importance, and if used formatively can also show students what their next steps should be to enhance the quality of their performance.” --Susan Brookhart 34

Making Connections: Rubrics & Student Growth Goal Setting within the Teacher Professional Growth and Effectiveness System (TPGES) 35

Student Growth Goal-Setting Student growth goals (SGG) focus on needs of current students SGG focus on enduring skill(s) (ES) identified in the standards Teachers define proficiency for the ES using the grade level standards 36

Defining ENDURING Learning that ENDURES beyond a single test date, is of value in other disciplines, is relevant beyond the classroom (applying learning to new and unique situations) is worthy of embedded, course-long focus, may be necessary for the next level of instruction. Requires critical thinking (analyzing, creating and evaluating)

Rubrics and Student Growth Goal-Setting An analytic rubric can help teachers combine multiple sources of data to – determine a baseline score for goal-setting – determine if students met the goal at the end of the course, and – formatively assess, provide feedback, and adjust instruction along the way 38

Rubric Makers & Already Made Rubrics Essaytagger.com Rubistar.com iRubric.com Rubrics4teachers.com Delaware Dept of Education ELA rubrics 39

Other Rubric Resources Metarubrics – Rubrics for Rubrics CASL MetaRubric Educational Testing Service Rubric for Rubrics 40

Our Targets I can evaluate rubrics for quality. I can design a quality rubric congruent to expectations of grade level standards. I understand how rubric development and use supports the student growth goal- setting process. 41

What will you take away from this morning’s work with rubrics? 42

Carol Franks Rebecca Woosley 43