1. Standards Scaling– Teacher Leaders LMS Team
March 21, 2014

2. Entry Task What do you already know about DOK?
What do you want to learn?
Complete DOK KWL chart

3. “Clearly a better method for developing and scoring assessments is needed – one that ensures that the scale (the size of an inch) stays the same from one assessment to the next and that a teacher applies the same logic to scoring of each assessment.”
Marzano, Robert J. Formative Assessment and Standards-Based Grading 2010

4. Focus Today
What do we mean by “Depth of Knowledge” (DOK) as defined by Norman Webb
How is the “Depth of Knowledge” learning progression a tool in using the construction of a proficiency scale to “unpack” clusters/ standards?
How can a deeper understanding of DOK help design Success Time tutorials and enrichment activities?

5. Norman Webb’s Depth of Knowledge
Adapted from the model used by Norm Webb, University of Wisconsin, to align standards with assessments
The degree of depth or complexity of knowledge reflected in the content standards and assessments
How deeply a student needs to understand the content for a given response/assessment .

6. Webb’s Depth of Knowledge (DOK)
Is a scale of cognitive demand which aligns standards with assessments.
DOK addresses the content being assessed and the depth to which we expect students to demonstrate understanding of that content.
DOK is a reference to the complexity of mental processing that must occur to answer a question, perform a task, or generate a product.
DOK is about cognitive complexity, not difficulty.

7. Have teachers pause to note observations regarding this graphic.

8. Talk about what you observe here

9. DOK Levels 4: Extended Thinking 1: Recall 3: Strategic Thinking 2:
Organize, Group
Show
Guess or predict
Compare
What caused...
Construct
Explain
Investigate
Edit or change
Comment on…
Who? What? When? Where? Why?
Find, Tell
State, Draw
Criticize
Prove
Connect
Design
Analyze
Create
4: Extended Thinking
1: Recall 2:
Skill /Concept
3: Strategic Thinking
DOK Levels

10. Pause to process.

11. Recall and Reproduction: Level 1
DOK 1 requires recall of information, such as a fact, definition, term, or performance of a simple process or procedure.
Locate or recall facts explicitly found in text.

12. Level 1 Recall-Reproduction
Recall or recognize a fact, term, or property.
Represent in words or diagrams a scientific concept or relationship.
Provide or recognize a standard scientific representation for simple phenomena.
Perform a routine procedure such as measuring length.
Identify familiar forces (e.g. pushes, pulls, gravitation, friction, etc.)
Identify objects and materials as solids, liquids, or gases

13. Skills/Concepts: Level 2
DOK 2 includes the engagement of some mental processing beyond recalling or reproducing a response. Items require students to make some decisions as to how to approach the question or problem.
These actions imply more than one mental or cognitive process/step.
Identify and summarize the major events, problem, solution, conflicts in literary text.
Explain the cause-effect of historical events.

14. Level 2- Skill and Concepts
Specify and explain the relationship among facts, terms, properties, and variables.
Identify variables, including controls, in simple experiments.
Distinguish between experiments and systematic observations.
Describe and explain examples and non-examples of science concepts.
Select a procedure according to specified criteria and perform it.
Formulate a routine problem given data and conditions.
Organize, represent, and interpret data.

15. Strategic Thinking: Level 3
DOK 3 requires deep understanding as exhibited through planning, using evidence, and more demanding cognitive reasoning. The cognitive demands at Level 3 are complex and abstract.
An assessment item that has more than one possible answer and requires students to justify the response they give would most likely be a Level 3.
Explain, generalize or connect ideas, using supporting evidence from a text or source.

16. Level 3- Strategic Thinking
Identify research questions and design investigations for a scientific problem.
Design and execute an experiment or systematic observation to test a hypothesis or research question.
Develop a scientific model for a complex situation.
Form conclusions from experimental data.
Cite evidence that living systems follow the Laws of Conservation of Mass and Energy.
Explain how political, social, and economic concerns can affect science, and vice versa.
Create a conceptual or mathematical model to explain the key elements of a scientific theory or concept.
Explain the physical properties of the Sun and its dynamic nature and connect them to conditions and events on Earth.
Analyze past, present, and potential future consequences to the environment resulting from various energy production technologies

17. Extended Thinking: Level 4
DOK 4 requires high cognitive demand and is very complex. Students are expected to make connections—relate ideas within the content or among content areas—and have to select or devise one approach among many alternatives on how the situation can be solved.
Due to the complexity of cognitive demand, DOK 4 often requires an extended period of time.
Gather, analyze, organize, and interpret information from multiple (print and non-print sources) to draft a reasoned report.

18. Level 4- Extended Thinking
Based on provided data from a complex experiment that is novel to the student, deduce the fundamental relationships among several variables.
Conduct an investigation, from specifying a problem to designing and carrying out an experiment, to analyzing its data and forming conclusions.
Explain how a particular scientific theory (e.g., evolution, plate tectonics, atomic theory, etc.) is supported by evidence from multiple lines of inquiry.
Produce a detailed report of a scientific experiment or systematic observation and infer conclusions based upon evidence obtained.
Write a detailed history of the development of an important scientific concept (e.g., atomic theory, gravitation) and explain how current conceptions developed from prior ones

19. Depth of Knowledge Recall— Identify this utensil.
From: Lois Barnes
SREB/HSTW
Recall— Identify this utensil.
Concept— Explain the function of the fork.
Strategic— Identify two examples of when a fork would not be the best utensil for a type of food and explain why.
Extended— Design an investigation to determine the optimal number and length of tines for a salad fork.
This example is meant to provide a simple, academic example of the difference between the Webb levels. It is meant to mirror the basic labels provided for each level on the first Webb slide.
Level 1 – Recognizing and identifying a fork is at the factual recall level of understanding and application of the “concept” of a utensil.
Level 2 - Requires a relatively simple level of understanding of how a fork functions and the skill of translating that understanding into an explanation.
Level 3 - Requires some analysis and connecting to the understanding of forks and food to generate a conclusion and explanation for a real situation.
Level 4 – Requires a deeper and more involved application of the understanding of forks as part of a complex investigation.

20. Depth of Knowledge Recall— Identify the type of tree.
Concept— Explain the function of the leaves.
Strategic— Explain how a drought might affect the growth of the tree.
Extended— Design an investigation of seedling growth to determine the best fertilizer for this type of tree.
This example is meant to provide a simple, academic example of the difference between the Webb levels. It is meant to mirror the basic labels provided for each level on the first Webb slide.
Level 1 – Recognizing and identifying a type of tree (coniferous, deciduous, etc.) is at the factual recall level of understanding and application of the “concept” of a tree.
Level 2 - Requires a relatively simple level of understanding of how a tree functions and the skill of translating that understanding into an explanation.
Level 3 - Requires some analysis and connecting to the understanding of trees to generate a conclusion and explanation for a real situation.
Level 4 – Requires a deeper and more involved application of the understanding of trees as part of a complex investigation.
From: Lois Barnes
SREB/HSTW

21. FROM TEACHER TO STUDENT IN FOCUS
DOK 1 – Teacher driven
DOK 2 – Teacher provided options
DOK 3 – Student
DOK 4 - Student

22. Card Sort Activity? NEED NEW SORT
Working with an elbow partner label each task/prompt with the appropriate DOK level and justify your decision.

23. Cognitive Rigor Matrix–Switch to ELA
This matrix from the Smarter Balanced Content Specifications for Literacy draws from both Bloom’s (revised) Taxonomy of Educational Objectives and Webb’s Depth-of-Knowledge Levels below.
The depth of knowledge levels defined by Smarter Balanced are informed by the Cognitive Rigor matrix. The Cognitive Rigor matrix combines two common taxonomies that categorize levels of cognition and shows how the Smarter Balanced depth of knowledge categories relate to these taxonomies. Here, the concept of remembering information only relates to the first depth of knowledge level. In contrast, evaluation does not relate to either of the first two levels of depth of knowledge. This table, which can be found in the Smarter Balanced Content Specifications for Mathematics, is a useful aid for guiding the development of items at different depth of knowledge levels.
Now let’s examine the content specifications.

24. DOK Snapshot DOK is a scale of cognitive demand
DOK requires looking at the assessment item/standard not student work to determine the level.
DOK is about the item/standard not the student
The context of the item/standard must be considered to determine the DOK level not just a look at what verb was chosen.
DOK is lowered when too much information is given
DOK is not about difficulty but how much thinking is required for the student to complete the prompt/task
DOK is not an exact science

25. Proficiency reflects DOK level specific to a standard
Recall/
Reproduction
DOK 1
Skills/ Concepts
DOK 2
DOK 3
Strategic Thinking
Extended Thinking
DOK 4
Proficiency reflects DOK level specific to a standard
Cognitive Demand Activity- mini - scaling

26. Assessment Literacy Each item aligned to a standard
Enough items to represent standard
Items grouped by standard
Enough items at different DOK levels
Best “type” of assessment item to measure level
Strength of item to distinguish both correctness and misconceptions for instruction

27. - using scaling as foundation
Plan
- using scaling as foundation
DOK scaffolding for Tutorial Deign
DOK 3 and 4 collaboration items for enrichment

28. Course Design Basics Select course standards
These are the priority standards for the course – the ones students will have multiple opportunities to reach standard on – shift to CCSS and NGSS alignment
Scale standards
Develop summative assessments – “body of evidence”
Leveled prompts/items
Backwards design an instructional plan: include levels of rigor, quality assignment development, intentional text selection, frequent checks for understanding and student-involved progress monitoring
Interventions and enrichments
Within class
As a building
Determine reporting and grading based on the instructional plan

29. Curriculum Portal Insight Plan Now imports gradebook information
Teachers could enter common assessment information that can be extracted through Insight
Training resources
Coaching sessions
Training in Excel pivot tables and Macro to add efficiency parallel to building Insight use Proficiency

30. Questions?