1. Webb’s Depth of Knowledge
No Child Left Behind requires assessments to “measure the depth and breadth of the state academic content standards for a given grade level” ( US Department of Education, 2003, p.12)
Focus on the leveling-why is this important?

2. Webb’s Depth of Knowledge
The Depth-of-Knowledge (DOK) was created by Norman Webb from the Wisconsin Center for Education Research. 

3. Depth of Knowledge
…measures the degree to which the knowledge elicited from students on the assessment is as complex as what students are expected to know and do as stated in the standards.

4. Depth of Knowledge is NOT.....
….about verbs. Verbs are not always used appropriately.
…about "difficulty"
…about the student or level of difficulty for the student - it requires looking at the assessment item not student work in order to determine the level.

5. Depth of Knowledge is…
…about what FOLLOWS the verb. What comes after the verb is more important than the verb itself.
…about the complexity of mental processing that must occur to answer a question.

6. Why is DOK Important?
Teachers can no longer rely on "tell-me-what-I told-you" assessments; new assessments must be created requiring students to demonstrate and apply what they have learned.

7. Should DOK Change the Way I Teach?
Instruction, assignments, and classroom assessment must incorporate the same expectations. DOK levels for a targeted objective must mirror the DOK level for the assessment. Teachers need to examine if their required student work and activities are keeping students engaged in activity or engaged in learning. Not all activities help students learn.

8. DOK- What it is and what it is not?
A common language for discussing knowledge complexity
A tool for alignment
A way to “tune” common assessments
A conversation starter about content
A part of reflective teaching
Most state/national tests will have DOKs 1 & 2 with few DOK 3; however, the PARCC test in will have DOK 4.
A state mandate
A silver bullet
Based on verbs
A taxonomy
A wheel
DOK is not an exact science.
DOK is not about difficulty but more about the cognitive demand needed to meet the standard.

9. The Depth of Knowledge is NOT determined by the verb, but the context in which the verb is used and the depth of thinking required.

10. Not sanctioned or authorized by
Norman Webb
DOK is NOT just about the verb!

11. DOK LEVEL REVIEW
LEVEL ONE - RECALL
Recall of a fact, information, or procedure
LEVEL TWO – SKILL/CONCEPT
Use information or conceptual knowledge
LEVEL THREE – STRATEGIC THINKING
Reasoning, developing a plan, more complex and abstract, students must justify responses
LEVEL FOUR – EXTENDED THINKING
Requires an investigation, collection of data and analysis of results; often occurs over an extended period of time

13. Language Arts Examples
Describe the characteristics of a short story.
Describe the difference between a short story and a novel.
Describe how the author of a short story must be cognizant of how much space he actually has to develop the elements of the story because he is confined by that space.
DOK 1- Simple recall
DOK 2- Requires cognitive processing to determine the differences in the two genres
DOK 3- Requires deep understanding of the elements of a short story and the definition of the short story genre

14. Social Studies Standards
SSUSH1 The student will describe European settlement in North America during the 17th century.
SSUSH9 e. Describe the significance of the Emancipation Proclamation
SSUSH9 b. Describe President Lincoln’s efforts to preserve the Union as seen in his second inaugural address and the Gettysburg speech and in his use of emergency powers, such as his decision to suspend habeas corpus.

15. All standards use the verb “Explain”
Science Standards
All standards use the verb “Explain”
SB1d: Explain the impact of water on life processes (i.e. osmosis, diffusion).
SB1b: Explain how enzymes function as catalysts in living cells.
SB1a: Explain the role of cell organelles for both prokaryotic and eukaryotic cells, including the cell membrane, in maintaining homeostasis and cell reproduction

16. DOK Level 1 Examples List animals that survive by eating other animals
Locate or recall facts found in text
Describe physical features of places
Determine the perimeter or area of rectangles given a drawing or labels
Identify elements of music using musical terminology
Identify basic rules for participating in simple games and activities

17. DOK Level 2 Examples Compare desert and tropical environments
Identify and summarize the major events, problem, solution conflicts in literary text
Explain the cause-effect of historical events
Predict a logical outcome based on information in a reading selection
Explain how good work habits are important at home, school and on the job.
Classify plane and three dimensional figures
Describe various styles of music

18. DOK Level 3 Examples
Compare consumer actions and analyze how these actions impact the environment
Analyze or evaluate the effectiveness of literary elements ( e.g. characterization, setting, point of view, conflict and resolution)
Solve a multiple-step problem and provide support with a mathematical explanation that justifies the answer
Develop a scientific model for a complex idea
Propose and evaluate solutions for an economic problem
Create a dance that represents the characteristics of a culture

19. DOK Level 4 Examples
Gather, analyze, organize, and interpret information from multiple sources to draft a reasoned report
Analyzing author’s craft (e.g. style, bias, literary techniques, point of view)
Create an exercise plan applying the FITT Principle(frequency, intensity time and type)
Analyze and explain multiple perspectives or issues within or across time periods, events or cultures
Write and produce an original play

20. DOK TASKS
As a table group, you will DOK tasks. You’ll need a set of task cards.
Deal the DOK task cards out to members at the table.
Lay out the 4 “DOK Example” headers on the table.
First person places one of the task cards under the appropriate header, explaining the rationale for the placement.
The table group confirms the placement or comes to consensus for another placement---be sure to articulate the rationale for the placement using DOK rubrics.
Continue and repeat the process until all tasks have been DOK’ed.

21. DOK 1 Tasks DOK 2 Tasks DOK 3 Tasks DOK 4 Tasks
Recall elements and details of story structure, such as sequence of events, character, plot and setting.
Formulate a routine problem given data and conditions.
Support ideas with details and examples.
Conduct a project that requires specifying a problem, designing and conducting an experiment, analyzing its data, and reporting results/ solutions.
Conduct basic mathematical calculations.
Identify and summarize the major events in a narrative.
Use voice appropriate to the purpose and audience.
Apply mathematical model to illuminate a problem or situation.
Label locations on a map.
Solve routine multiple-step problems.
Identify research questions and design investigations for a scientific problem.
Analyze and synthesize information from multiple sources.
Represent in words or diagrams a scientific concept or relationship.
Describe the cause/effect of a particular event.
Apply a concept in other contexts.
Describe and illustrate how common themes are found across texts from different cultures.
Describe the features of a place or people.
Identify patterns in events or behavior.
Develop a scientific model for a complex situation.
Design a mathematical model to inform and solve a practical or abstract situation.
Perform routine procedures like measuring length or using punctuation marks correctly.
Use context cues to identify the meaning of unfamiliar words.
Determine the author’s purpose and describe how it affects the interpretation of a reading selection.
Organize, represent and interpret data.

22. DOK Level 4 Extended Thinking Time alone is not the distinguishing factor
Task
Thinking
Collecting data sample over several months
Recall
Organizing the data in a chart
Skills/Concepts
Using this chart to make and justify prediction
Strategic Thinking
Developing a generalized model from this data and applying it to a new situation
Extended Thinking

23. Subject Depth of Knowledge Level 1 Level 2 Level 3 Level 4
English Language Arts
Requires students to recall, observe, question or represent facts or simple skills or abilities. Requires only surface understanding of text often verbatim recall or slight paraphrasing. Use conventions of Standard English.
Examples:
Support ideas by reference to specific details in text
Use dictionary to find meaning
Use punctuation marks correctly
Identify figurative language in passage
Identify correct spelling or meaning of words
Requires processing beyond recall and observation. Requires both comprehension and subsequent processing of text. Involves ordering, classifying text as well as identifying patterns, relationships and main points. Connect ideas using ideas using simple organizational structures. Requires some scrutiny of text.
Use contextual clues to identify unfamiliar words
Predict logical outcome
Construct or edit compound or complex sentences
Identify and summarize main points
Apply knowledge of conventions of standard American English
Compose accurate summaries
Requires students to go beyond text. Requires students to explain, generalize and connect ideas. Involves inference, prediction, elaboration and summary. Requires students to support positions using prior knowledge and to manipulate themes across passages. Students develop compositions with multiple paragraphs.
Determine effect of author’s purpose on text elements
Summarize information from multiple sources
Critically analyze literature
Edit writing to produce logical progression
Compose focused, organized, coherent, purposeful prose
Requires extended higher order processing. Typically requires extended time to complete task, but time spent not on repetitive tasks. Involves taking information from one text/passage and applying this information to a new task. May require generating hypotheses and performing complex analyses and connections among texts. Examples:
Analyze and synthesize information from multiple sources
Examine and explain alternative perspectives across sources
Describe and illustrate common themes across a variety of texts
Create compositions that synthesize, analyze, and evaluate

24. Mathematics Subject Depth of Knowledge Level 1 Level 2 Level 3 Level 4
Requires students to recall or observe facts, definitions, or terms. Involves simple one-step procedures. Involves computing simple algorithms (e.g., sum, quotient). Examples:
Recall or recognize a fact, term or property
Represent in words, pictures or symbols in a math object or relationship
Perform routine procedure like measuring
Requires students to make decisions of how to approach a problem. Requires students to compare, classify, organize, estimate or order data. Typically involves two-step procedures.
Examples:
Specify and explain relationships between facts, terms, properties or operations
Select procedure according to criteria and perform it
Solve routine multiple-step problems
Requires reasoning, planning or use of evidence to solve problem or algorithm. May involve activity with more than one possible answer. Requires conjecture or restructuring of problems. Involves drawing conclusions from observations, citing evidence and developing logical arguments for concepts. Uses concepts to solve non-routine problems. Examples:
Analyze similarities and differences between procedures
Formulate original problem given situation
Formulate mathematical model for complex situation
Requires complex reasoning, planning, developing and thinking. Typically requires extended time to complete problem, but time spent not on repetitive tasks. Requires students to make several connections and apply one approach among many to solve the problem. Involves complex restructuring of data, establishing and evaluating criteria to solve problems. Examples:
Apply mathematical model to illuminate a problem, situation
Conduct a project that specifies a problem, identifies solution paths, solves the problem, and reports results
Design a mathematical model to inform and solve a practical or abstract situation

25. Social Studies/History
Subject
Depth of Knowledge
Social Studies/History
Level 1
Recall of Information
Level 2
Basic Reasoning
Level 3
Application
Level 4
Extended Reasoning
Level 1 asks students to recall facts, terms, concepts, trends, generalizations and theories or to recognize or identify specific information contained in graphics. This level generally requires students to identify, list, or define. The items at this level usually ask the student to recall who, what, when and where. Items that require students to “describe” and “explain” could be classified at Level 1 or 2 depending on what is to be described and explained. A Level 1 “describe or explain” would recall, recite or reproduce information. Items that require students to recognize or identify specific information contained in maps, charts, tables, graphs or drawings are generally level 1 Recall or recognize an event, map or document.
EXAMPLES:
Describe the features of a place or people.
Identify key figures in a particular context.
Level 2 includes the engagement of some mental processing beyond recalling or reproducing a response. This level generally requires students to contrast or compare people, places, events and concepts; convert information from one form to another; classify or sort items into meaningful categories ; describe or explain issues and problems, patterns , cause and effect, significance or impact, relationships, points of view or processes. A Level 2 “describe or explain” would require students to go beyond a description or explanation of recalled information to describe or explain a result or “how” or “why.”
Describe the causes/effects of particular events.
Identify patterns in events or behavior.
Categorize events or figures into meaningful groupings
Level 3 requires reasoning, using evidence, and a higher level of thinking than the previous two levels. Students would go beyond knowing “how and why” to justifying the “how and why” through application and evidence. The cognitive demands at Level 3 are more complex and more abstract than Levels 1 or 2. Items at Level 3 include drawing conclusions; citing evidence; using concepts to explain “how and why;” using concepts to solve problems; analyzing similarities and differences in issues and problems; proposing and evaluating solutions to problems; recognizing and explaining misconceptions or making connections across time and place to explain a concept or big idea.
Analyze how changes have affected people or places.
Apply concept in other contexts.
Form alternate conclusions
Level 4 requires addition of planning, investigating, or developing that will most likely require an extended period of time. At this level the cognitive demands should be high and the work should be very complex. Students should be required to connect and relate ideas and concepts within the content area or among content areas in order to be at this highest level. Task or product oriented. Students may analyze and synthesize information from multiple sources, examine and explain alternative perspectives across a variety of sources and/or describe and illustrate how common themes and concepts are found across time and place. Predictions with evidence as support, a logical argument, or plan and develop solutions to problems.
Research, define and describe situation and provide alternative solutions.
Describe, define and illustrate common social, historical, or geographical themes and how they interrelate.

26. Science Subject Depth of Knowledge Level 1 Level 2 Level 3 Level 4
(Recall and Reproduction) Requires the recall of information, such as a fact, definition, term, or a simple procedure, as well as performance of a simple science process or procedure. Level 1 only requires students to demonstrate a rote response, use a well-known formula, follow a set procedure (like a recipe), or perform a clearly defined series of steps. Examples:
Recall or recognize a fact, term, or property.
Represent in words or diagrams a scientific concept or relationship.
Provide or recognize a standard scientific representation for simple phenomenon.
Perform a routine procedure, such as measuring length.
(Skills and Concepts)
Includes the engagement of some mental processing beyond recalling or reproducing a response. Level 2 activities include making observations and collecting data; classifying, organizing, and comparing data; and organizing and displaying data in tables, graphs, and charts. Items require students to make some decisions as to how to approach the question or problem. Examples:
Specify and explain the relationship between facts, terms, properties, or variables.
Describe and explain examples and non-examples of science concepts.
Select a procedure according to specified criteria and perform it.
Formulate a routine problem, given data and conditions.
Organize, represent, and interpret data.
(Strategic Thinking)
Requires reasoning, planning, using evidence, and a higher level of thinking than the previous two levels. The cognitive demands at Level 3 are complex and abstract. The complexity does not result only from the fact that there could be multiple answers, a possibility for both Levels 1 and 2, but because the multi-step task requires more demanding reasoning. Examples:
Identify research questions and design investigations for a scientific problem.
Solve non-routine problems.
Develop a scientific model for a complex situation.
Form conclusions from experimental data.
(Extended Thinking)
Requires complex reasoning, experimental design and planning, and probably will require an extended period of time either for the science investigation required by an objective, or for carrying out the multiple steps of an assessment item.
Examples:
Based on data provided from a complex experiment that is novel to the student; deduct the fundamental relationship between several controlled variables.
Conduct an investigation, from specifying a problem to designing and carrying out an experiment, to analyzing its data and forming conclusions.

27. Increased Cognitive Demand
Describe

28. The level of a DOK item is determined by the task (defined by complex thinking and reasoning skills), not grade level or ability of the student. 
Therefore, the DOK of the task does not change with grade or ability of the student.  
Verbs alone do not determine the DOK’s level of an assessment task.  DOK’s focus is on how deeply students need to know content for a given response.
DOK and Assessments

29. DOK and Assessments
Multiple-choice questions can be written at a DOK 3 or 4 level; however, to design a question in this format is difficult.  An Item at DOK level 3 or 4 requires complex reasoning, strategic and extended thinking about the concepts of the content and a real world context, and especially at a level 4 that requires research, investigation and application often over an extended period of time.
"There are six dimensions to the alignment process and depth (DOK) was only one.  The U.S. Department of Education issued guidelines that include six dimensions important for making judgments about the alignment between state standards and assessments.  These dimensions include comprehensiveness, content and performance match, emphasis, depth, consistency with achievement standards and clarity for users.
DOK levels are not related to the score points.