1 Standards Scaling– Teacher Leaders LMS Team March 21, 2014
2 Entry Task What do you already know about DOK? What do you want to learn?Complete DOK KWL chart
3 “Clearly a better method for developing and scoring assessments is needed – one that ensures that the scale (the size of an inch) stays the same from one assessment to the next and that a teacher applies the same logic to scoring of each assessment.”Marzano, Robert J. Formative Assessment and Standards-Based Grading 2010
4 Focus TodayWhat do we mean by “Depth of Knowledge” (DOK) as defined by Norman WebbHow is the “Depth of Knowledge” learning progression a tool in using the construction of a proficiency scale to “unpack” clusters/ standards?How can a deeper understanding of DOK help design Success Time tutorials and enrichment activities?
5 Norman Webb’s Depth of Knowledge Adapted from the model used by Norm Webb, University of Wisconsin, to align standards with assessmentsThe degree of depth or complexity of knowledge reflected in the content standards and assessmentsHow deeply a student needs to understand the content for a given response/assessment.
6 Webb’s Depth of Knowledge (DOK) Is a scale of cognitive demand which aligns standards with assessments.DOK addresses the content being assessed and the depth to which we expect students to demonstrate understanding of that content.DOK is a reference to the complexity of mental processing that must occur to answer a question, perform a task, or generate a product.DOK is about cognitive complexity, not difficulty.
7 Have teachers pause to note observations regarding this graphic.
11 Recall and Reproduction: Level 1 DOK 1 requires recall of information, such as a fact, definition, term, or performance of a simple process or procedure.Locate or recall facts explicitly found in text.
12 Level 1 Recall-Reproduction Recall or recognize a fact, term, or property.Represent in words or diagrams a scientific concept or relationship.Provide or recognize a standard scientific representation for simple phenomena.Perform a routine procedure such as measuring length.Identify familiar forces (e.g. pushes, pulls, gravitation, friction, etc.)Identify objects and materials as solids, liquids, or gases
13 Skills/Concepts: Level 2 DOK 2 includes the engagement of some mental processing beyond recalling or reproducing a response. Items require students to make some decisions as to how to approach the question or problem.These actions imply more than one mental or cognitive process/step.Identify and summarize the major events, problem, solution, conflicts in literary text.Explain the cause-effect of historical events.
14 Level 2- Skill and Concepts Specify and explain the relationship among facts, terms, properties, and variables.Identify variables, including controls, in simple experiments.Distinguish between experiments and systematic observations.Describe and explain examples and non-examples of science concepts.Select a procedure according to specified criteria and perform it.Formulate a routine problem given data and conditions.Organize, represent, and interpret data.
15 Strategic Thinking: Level 3 DOK 3 requires deep understanding as exhibited through planning, using evidence, and more demanding cognitive reasoning. The cognitive demands at Level 3 are complex and abstract.An assessment item that has more than one possible answer and requires students to justify the response they give would most likely be a Level 3.Explain, generalize or connect ideas, using supporting evidence from a text or source.
16 Level 3- Strategic Thinking Identify research questions and design investigations for a scientific problem.Design and execute an experiment or systematic observation to test a hypothesis or research question.Develop a scientific model for a complex situation.Form conclusions from experimental data.Cite evidence that living systems follow the Laws of Conservation of Mass and Energy.Explain how political, social, and economic concerns can affect science, and vice versa.Create a conceptual or mathematical model to explain the key elements of a scientific theory or concept.Explain the physical properties of the Sun and its dynamic nature and connect them to conditions and events on Earth.Analyze past, present, and potential future consequences to the environment resulting from various energy production technologies
17 Extended Thinking: Level 4 DOK 4 requires high cognitive demand and is very complex. Students are expected to make connections—relate ideas within the content or among content areas—and have to select or devise one approach among many alternatives on how the situation can be solved.Due to the complexity of cognitive demand, DOK 4 often requires an extended period of time.Gather, analyze, organize, and interpret information from multiple (print and non-print sources) to draft a reasoned report.
18 Level 4- Extended Thinking Based on provided data from a complex experiment that is novel to the student, deduce the fundamental relationships among several variables.Conduct an investigation, from specifying a problem to designing and carrying out an experiment, to analyzing its data and forming conclusions.Explain how a particular scientific theory (e.g., evolution, plate tectonics, atomic theory, etc.) is supported by evidence from multiple lines of inquiry.Produce a detailed report of a scientific experiment or systematic observation and infer conclusions based upon evidence obtained.Write a detailed history of the development of an important scientific concept (e.g., atomic theory, gravitation) and explain how current conceptions developed from prior ones
19 Depth of Knowledge Recall— Identify this utensil. From: Lois BarnesSREB/HSTWRecall— Identify this utensil.Concept— Explain the function of the fork.Strategic— Identify two examples of when a fork would not be the best utensil for a type of food and explain why.Extended— Design an investigation to determine the optimal number and length of tines for a salad fork.This example is meant to provide a simple, academic example of the difference between the Webb levels. It is meant to mirror the basic labels provided for each level on the first Webb slide.Level 1 – Recognizing and identifying a fork is at the factual recall level of understanding and application of the “concept” of a utensil.Level 2 - Requires a relatively simple level of understanding of how a fork functions and the skill of translating that understanding into an explanation.Level 3 - Requires some analysis and connecting to the understanding of forks and food to generate a conclusion and explanation for a real situation.Level 4 – Requires a deeper and more involved application of the understanding of forks as part of a complex investigation.
20 Depth of Knowledge Recall— Identify the type of tree. Concept— Explain the function of the leaves.Strategic— Explain how a drought might affect the growth of the tree.Extended— Design an investigation of seedling growth to determine the best fertilizer for this type of tree.This example is meant to provide a simple, academic example of the difference between the Webb levels. It is meant to mirror the basic labels provided for each level on the first Webb slide.Level 1 – Recognizing and identifying a type of tree (coniferous, deciduous, etc.) is at the factual recall level of understanding and application of the “concept” of a tree.Level 2 - Requires a relatively simple level of understanding of how a tree functions and the skill of translating that understanding into an explanation.Level 3 - Requires some analysis and connecting to the understanding of trees to generate a conclusion and explanation for a real situation.Level 4 – Requires a deeper and more involved application of the understanding of trees as part of a complex investigation.From: Lois BarnesSREB/HSTW
21 FROM TEACHER TO STUDENT IN FOCUS DOK 1 – Teacher drivenDOK 2 – Teacher provided optionsDOK 3 – StudentDOK 4 - Student
22 Card Sort Activity? NEED NEW SORT Working with an elbow partner label each task/prompt with the appropriate DOK level and justify your decision.
23 Cognitive Rigor Matrix–Switch to ELA This matrix from the Smarter Balanced Content Specifications for Literacy draws from both Bloom’s (revised) Taxonomy of Educational Objectives and Webb’s Depth-of-Knowledge Levels below.The depth of knowledge levels defined by Smarter Balanced are informed by the Cognitive Rigor matrix. The Cognitive Rigor matrix combines two common taxonomies that categorize levels of cognition and shows how the Smarter Balanced depth of knowledge categories relate to these taxonomies. Here, the concept of remembering information only relates to the first depth of knowledge level. In contrast, evaluation does not relate to either of the first two levels of depth of knowledge. This table, which can be found in the Smarter Balanced Content Specifications for Mathematics, is a useful aid for guiding the development of items at different depth of knowledge levels.Now let’s examine the content specifications.
24 DOK Snapshot DOK is a scale of cognitive demand DOK requires looking at the assessment item/standard not student work to determine the level.DOK is about the item/standard not the studentThe context of the item/standard must be considered to determine the DOK level not just a look at what verb was chosen.DOK is lowered when too much information is givenDOK is not about difficulty but how much thinking is required for the student to complete the prompt/taskDOK is not an exact science
25 Proficiency reflects DOK level specific to a standard Recall/ReproductionDOK 1Skills/ ConceptsDOK 2DOK 3Strategic ThinkingExtended ThinkingDOK 4Proficiency reflects DOK level specific to a standardCognitive Demand Activity- mini - scaling
26 Assessment Literacy Each item aligned to a standard Enough items to represent standardItems grouped by standardEnough items at different DOK levelsBest “type” of assessment item to measure levelStrength of item to distinguish both correctness and misconceptions for instruction
27 - using scaling as foundation Plan- using scaling as foundationDOK scaffolding for Tutorial DeignDOK 3 and 4 collaboration items for enrichment
28 Course Design Basics Select course standards These are the priority standards for the course – the ones students will have multiple opportunities to reach standard on – shift to CCSS and NGSS alignmentScale standardsDevelop summative assessments – “body of evidence”Leveled prompts/itemsBackwards design an instructional plan: include levels of rigor, quality assignment development, intentional text selection, frequent checks for understanding and student-involved progress monitoringInterventions and enrichmentsWithin classAs a buildingDetermine reporting and grading based on the instructional plan
29 Curriculum Portal Insight Plan Now imports gradebook information Teachers could enter common assessment information that can be extracted through InsightTraining resourcesCoaching sessionsTraining in Excel pivot tables and Macro to add efficiency parallel to building Insight use Proficiency