Taxonomic Levels And Rubrics MMann/SAS

Desired Outcomes An awareness of taxonomic levels and its purpose An awareness of the relationship of HCPS III benchmarks and taxonomic levels An opportunity to match benchmarks and tasks to the taxonomic levels An awareness of various types of performance assessment rubrics MMann/SAS

Why do I need to know the taxonomic levels? Aligning our instruction and assessment to the targets. Discuss with a partner – 1 minutes MMann/SAS

Alignment – congruence or match between curriculum, instruction and assessment Based on GLOs & HCPS III Instruction Implementation of the curriculum Assessment Multiple measures of proficiency of the STUDENT ACHIEVEMENT What How How Well Talking Points: We’ve defined each of the corners of the triangle. Alignment means that there is congruence (or a match) between the curriculum (or the work plan), instruction (or the work) and assessment (or the measure of the success of the work). MMann/SAS

(Prince-Baugh, 1997; Mitchell, 1998; Wishnick, 1989) Research on Aligning curriculum with standards and assessment shows a strong relationship to student achievement. (Prince-Baugh, 1997; Mitchell, 1998; Wishnick, 1989) MMann/SAS

Standards Implementation Process Model Student involvement throughout the process. Teacher collaboration throughout the process. Identify relevant content standards Determine acceptable evidence and criteria Determine learning experiences that will enable students to learn what they need to know and to do Teach and collect evidence of student learning Assess student work to inform instruction or use data to provide feedback Standards Implementation Planning Model. We are focusing on #1 (Identify relevant standards), #2 (Determine the acceptable evidence and criteria) and #5 (Assess student work to inform instruction or use data to provide feedback) in this module. Students should always be involved in each step of the implementation planning model. In assessment, you gather information about student learning to inform our teaching and to help students learn effectively. (Evaluation is a process to review the evidence to determine its value. This will be shared in the module on Standards based grading and reporting.) Teacher collaboration is essential in a standards-based education. Using professional learning communities to initiate discussions on the meaning of standards and benchmarks and what the evidence looks like helps teachers break the isolation of the classroom. Evaluate student work and make judgment on learning results and communicate findings Reteach, or repeat the process with the next set of relevant standards MMann/SAS Adapted from WestEd’s Learning from Assessment

Definition = the science or technique of classification All targets, curriculum, instruction, activities and assessments involve some level of thinking. Taxonomy Definition = the science or technique of classification MMann/SAS

Cognition Type or “cognitive demand” - generally refers to a taxonomy and reflects a classification of thinking rather than a sequential hierarchy. (understanding prior to application and analysis) Cognitive demand is determined by analyzing the context of the lesson. (What support is provided and what are the students being asked to do?) Talking Points: It is important to note that traditionally Bloom’s Taxonomy has been considered a sequential hierarchy in that each level was dependent on the previous level. For example, before one could apply certain content, one had to first comprehend it. With current brain research, we know that comprehension can be developed through the application process. Instead of referring to Bloom’s as levels of cognition, deep alignment references a class or type of cognition. The type of cognition is referred to as the cognitive demand in the Grade Level Expectations (GLEs). Let’s look back at our previous example. MMann/SAS

Adapt or adopt a systematic method for assigning performance expectations. McREL Taxonomy of objectives = a system for identifying distinct levels of difficulty. Bloom’s Guilford’s Three-Story Intellect Marzano’s MMann/SAS

Marzano’s Taxonomic Levels Level 1: Retrieval - recall, execution Level 2: Comprehension - integrating, representation, symbolizing Level 3: Analysis - matching, classifying, error analysis, generalizing, specifying Level 4: Utilization - decision making, problem solving,experimental inquiry, investigation MMann/SAS

Not used with performance standards, but part of taxonomy Level 5: Metacognitive System - goal setting, process monitoring, monitoring clarity, monitoring accuracy Level 6: Self System - examining importance, examining efficacy, examining emotional response, examining motivation. Adapted from Marzano (2001). “Designing A New Taxonomy of Educational Objectives”. Level 6: If an individual doesn’t perceive a specific piece of knowledge to be important at a personal level, he or she will probably not be highly motivate to learn it. Efficacy – If individuals do not believe they can change their level of competence relevant to a specific piece of knowledge they will probably not be motivated to learn it, even if they perceive it to be important. In UbD – use of a “hook” – (connected to the performance assessment). Motivator for unit. Level 5: Setting specific goals relative to one’s understanding of or skill at a specific type of knowledge. E.g. Example – World War… essential questions MAY help to set-up the context of the lesson and help students see the relevance to their own life. E.g. Today we are learning about World War II vs. Is war ever justifiable? MMann/SAS

The Three Systems and Knowledge New Task Self-System Decides to engage Continue current behavior Metacognitive System Sets goals and strategies Cognitive System Processes relevant information Knowledge From Marzano (2001). Designing a New Taxonomy of Educational Objectives

Knowledge Utilization Marzano’s Taxonomic Levels (Cognitive System) Level IV Level III Level II Level I Knowledge Utilization Analysis Comprehension Knowledge Retrieval Use ___ to determine ___ Judge the validity of ___ Use___ to solve Generate/Test hypotheses Analyze using evidence Investigate Compare/ contrast Differentiate Categorize Find what is common among Determine reasonableness of information Predict Determine what comes next/later Describe and explain Explain the concept Demonstrate and explain Diagram Illustrate/ describe how ___ is related to ___ Represent Recognize Select from a list Recall Give/Provide examples List Name Read Perform mathematical operation (by following a set algorithm) VERBS to help identify level Look at total benchmark for true understanding of benchmark level. For example – identify common idioms (i.e. out of the frying pan and into the fire…. Cat got your tongue…) Identify is at level 1 but the concept of idioms may not be at a retrieval level for students. The level of difficulty may be a level 3 task because students must understand understand figurative language vs. literal languaging, tie back to background knowledge, etc. MMann/SAS

MMann/SAS

Standards Implementation Process Model Evaluate Student Work and Make Judgment on Learning Results and Communicate Findings Ï Assess Student Work to Inform Instruction or Use Data to Provide Feedback Î Teach and Collect Evidence of Student learning Í Congruence of Content, Context & Cognitive Demand Determine Learning Experiences that will Enable Students to Learn what they need to Know & Do ¸ Everything “bounces” back to the standards. Work in groups of four or less. Assign roles (i.e. facilitator, recorder, reporter, timekeeper). Use handouts # 3 or #4, Grade Level Performance Indicators and note-taking forms. Use the six-step process as a tool to talk about standards-based issues and fill in as many sections on the note-taking form as possible based on what you see in the sample. Share conclusions with other groups. Share the big “Aha’s” with large group. Determine Acceptable Evidence and Criteria · Identify Relevant Standards  MMann/SAS

Let’s Identify Taxonomic Levels Retrieve (Marzano) Recognize, Recall, Execute Comprehension Integrate, Symbolize Analysis Match, Classify, Analyze, Generalize, Specify Knowledge Utilization Decision Making, Problem Solving, Inquire Experimentally, Investigate MMann/SAS

Level 1 - Knowledge Retrieval Topic Cells, Tissues, Organs, and Organ Systems Benchmark BS.4.1 Describe different cell parts and their functions. Level 1 - Knowledge Retrieval Topic The Universe Benchmark ES.8.10 Compare different theories concerning the formation of the universe. 4 Level 3 - Analysis MMann/SAS

Drill and Practice Work with your table group Read each card Group by taxonomic level MMann/SAS

Level of thinking helps determine the appropriate assessment method MMann/SAS

Describe the setting of the story. Analyze plot, setting, characterization, or conflict to interpret theme in a literary text. Describe the setting of the story. Explain how the author uses his characters to convey a message. Compare the plot of this story to the plot of the previous story. Chooses a literary element (e.g., conflict). Describes how the author treats this element in the story. Assess how this element relates to the theme. MMann/SAS

Knowing a taxonomy also helps in… scaffolding instruction. Create Compare Explain Identify MMann/SAS

Three Tiers of Skill and Assessment Work Thanks to Heidi Hayes Jacobs Drill & Practice Rehearsal & Scrimmage Authentic Performance Need for all 3 levels when addressing scaffolding Drill and practice - mastery of a skill Rehearsal -- practice putting the individual components; informal/formative assessment comes in to determine readiness/level of proficiency Authentic Performance – performance assessment e.g. Sports - isolating the skills – running, throwing, catching, passing, etc then combining the skills Actual game -- use the info to assess needs (need to shoot acurately, etc) MMann/SAS

The level of thinking in the benchmark is the level of thinking required to meet proficiency. MMann/SAS

Balanced Assessment Model MMann/SAS

Performance Assessment is an assessment (product or performance) based on observation and judgment about its quality. the activities, problems, projects, and assignments students are asked to perform. anything from a special task at the end of instruction as in a culminating event, or naturally occurring events during regular instruction. MMann/SAS

The Importance of Criteria “Teachers [frequently] ask the wrong question first … “What do we do?” - putting the focus immediately on designing tasks - when they need to ask, “What do we want kids to know and be able to do? How well? What does quality look like? [We} need to ask these questions very clearly first.” Mike Hibbard, Education Update, 38(4). p.5, ASCD, June, 1996. MMann/SAS

To Know Criteria Requires ... Being exposed to the criteria from the beginning of instruction. Having terms defined. (lots of details) Samples of strong and weak performances. Practice with feedback using the vocabulary of the criteria. Focused revision of work. Practice articulating the vocabulary for quality and applying it to many situations. Instruction consciously focused on subparts of the criteria. Judy Arter, ATI MMann/SAS

What is a Rubric? A scoring guide designed to provide constructive feedback to students Designed to show how important elements of a task would look in a progression from less well developed to exceptional along a continuum (Tomlinson, 2003). A Latin word that means “red.” MMann/SAS

A Rubric = Dimensions (essential qualities) + Continuum (Scale) + Descriptors of points on the scale + Work samples illustrating those points. MMann/SAS

Holistic Rubrics Holistic rubrics have one performance expectation description at each numerical level on the rubric. The product or performance is evaluated as a whole, and given a single score. Used “to obtain the overall impression of the quality of a performance or product.” (Wiggins and McTighe, 1999) MMann/SAS

Holistic Rubrics Quicker to write and to use. Summative because they evaluate work at the end of the process. Fails to communicate to students, especially low performing students, what their shortcomings are MMann/SAS

Holistic Rubric Example Fiction Writing Content Rubric 5 The plot, setting, and characters are developed fully and organized well. The who, what, where, when, and why are explained using interesting language and sufficient detail. 4 Most parts of the story mentioned in a score of 5 above are developed and organized well. A couple of aspects may need to be more fully or more interestingly developed. 3 Some aspects of the story are developed and organized well, but not as much detail or organization is expressed as in a score of 4. 2 A few parts of the story are developed somewhat. Organization and language usage need improvement. 1 Parts of the story are addressed without attention to detail or organization. MMann/SAS

Analytical Rubrics Use multiple descriptors for each criterion evaluated. Type of “task analysis” where teachers award points on a criterion-by-criterion basis. Described as teaching rubrics because their design helps students improve their own performance. MMann/SAS

Analytic Rubric Example Fiction Writing Content Rubric Criteria 4 3 2 1 PLOT: "What" and "Why" Both plot parts are fully developed. One of the plot parts is fully developed and the less developed part is at least addressed. Both plot parts are addressed but not fully developed. Neither plot parts are fully developed. SETTING: "When" and "Where" Both setting parts are fully developed. One of the setting parts is fully developed and the less developed part is at least addressed. Both setting parts of the story are addressed but not fully developed. Neither setting parts are developed. CHARACTERS: "Who" described by behavior, appearance, personality, and character traits The main characters are fully developed with much descriptive detail. The reader has a vivid image of the characters. The main characters are developed with some descriptive detail. The reader has a vague idea of the characters. The main characters are identified by name only None of the characters are developed or named. MMann/SAS

Holistic or Analytical Trait Use : Quick snapshot of overall status When speed of scoring is important Simple products or performances Disadvantages: 2 students can get same score for different reasons Can’t identify strengths & weaknesses Not useful for students Analytical Use: Planning instruction - show relative strengths & weaknesses Details to teach student quality Detailed feedback Precision more important that speed: Disadvantages: Scoring is slower Takes longer to learn MMann/SAS

Descriptive Terms for Differences in Degree Degrees of Understanding Degrees of Frequency Degrees of Effectiveness Degrees of Independence Degrees of Accuracy Degrees of Clarity MMann/SAS

Descriptive Terms for Differences in Degrees Understanding Frequency Accuracy Clarity thorough/ complete consistently completely accurate exceptionally clear substantial generally generally accurate generally clear partial/ incomplete sometimes inaccurate lacks clarity misunder-standing rarely major inaccuracies unclear MMann/SAS

Options for Selecting Rubrics Create your own - build from scratch Adopt - use an existing rubric Adapt - Modify or combine existing rubrics Reword parts Drop or change one or more scales Omit irrelevant criteria “Mix” and Match” rubrics Change a holistic rubric into an analytic rubric Modify for different grade levels MMann/SAS

Guidelines for Rubrics Rubrics are effective when teachers utilize the following criteria: Use specific numbers like “2” or “3 or more” rather than vague words like “some,” “many,” or “few.” Use specific descriptors, rather than general descriptors like “good” or “excellent.” Use the vocabulary of the standards and benchmarks. State clear expectations for work so that all teachers, students, and parents know the criteria for quality and the requirements for earning a grade. Burke, 2006 MMann/SAS

Resources Anderson, L.,Krathwohl, D. et al. (2001). A Taxonomy for Learning, Teaching and Assessing. New York: Longman. Curriculum Associates:Assessing Levels of Comprehension. Lewin, L. & Shoemaker, B.J. (1998). Great Performances. Virginia: ASCD. Marzano, R.J. (2001). Designing a New Taxonomy of Educational Objectives. Thousand Oaks: Corwin Press. Popham, W. J. (2002). Classroom Assessment. Boston: Allyn & Bacon. Stiggins, R.J. et al. (2004). Classroom Assessment for Student Learning. Portland: ATI. Wahlstrom, D. (2002). Designing & Using High- Quality Paper-and-Pencil Tests. Virginia: Successline. www.k12.wa.us/CurriculumInstruct/ pubdocs/WERA/WERA2005_Webversion.pp http://www.stedwards.edu/cte/content/view/1536/49/ MMann/SAS