1. Understanding Depth of Knowledge
Lou Maynus, Mathematics Coordinator WVDE

2. Depth of Knowledge (DOK)
Adapted from the model used by Norm Webb, University of Wisconsin, to align standards with assessments
Used by the Council of Chief State School Officers (CCSSO) for assessment alignment

3. Why Depth of Knowledge?
No Child Left Behind (NCLB) requires assessments to “measure the depth and breadth of the state academic content standards for a given grade level” (U.S. Department of Education, 2003, p. 12)

4. Why Depth of Knowledge?
Mechanism to ensure that the intent of the standard and the level of student demonstration required by the standard matches the assessment items (required under NCLB)
Provides cognitive processing ceiling (highest level students can be assessed) for item development

5. What is Depth of Knowledge?
The degree of depth or complexity of knowledge reflected in the content standards and assessments
How deeply a student needs to understand the content for a given response/assessment
It is essential all educators recognize the Depth of Knowledge in the revised CSOs. We need to understand the DOK of the CSOs to comprehend what we want kids to know, understand, and be able to do. If teachers don’t understand the DOK of the CSOs, they won’t be able to effectively instruct or assess. They won’t be able to prepare students for the DOK the WESTEST.

6. Webb’s DOK Levels Level 1 Recall and Reproduction
Level 2 Skills & Concepts
Level 3 Strategic Thinking
Level 4 Extended Thinking

7. Depth of Knowledge Levels
Level 1: Recall Recall, recognition; skill, behavior or sequence of behaviors learned through practice and easily performed Level 2: Skill/Concept Engagement of some mental processing beyond recalling; the use of information or conceptual knowledge; requires making some decisions regarding how to approach a question or problem Level 3: Strategic Thinking More sophisticated reasoning and analysis; deep understanding; students are required to solve problems & draw conclusions Level 4: Extended Thinking Requires integration of knowledge from multiple sources and ability to represent knowledge in a variety of ways; usually requires work over an extended period of time
The DOK has 4 levels. Depth of Knowledge, or DOK, is a way to think about content complexity, not content difficulty. That is the first thing with which people must come to understanding. Complexity is different from difficulty. For example, if students have not seen the word or content before, it might be difficult for them, but it is not complex.
Level 1 DOK is recall and recognition.
Level 2 – is about using a skill or a concept, i.e. Paraphrase. Conceptual understanding generally refers to the integration and application of concepts and other ideas within a content area. Procedural understanding denotes knowledge about skills and sequence of steps, when and how these should be used appropriately, and their efficient and accurate applications.
Level 3 DOK requires strategic thinking. Analysis and other examples are given here. Non-routine problem solving like in reading and determining author’s purpose is Level 3.
Level 4 DOK requires extended thinking usually requires work over a period of time, including gathering information, analyzing findings, preparing reports, and presenting findings.
The new rigor of the WESTEST means it contains more levels 2 & 3 items.

8. Understanding Depth of Knowledge
DOK is about intended outcome, not difficulty.
DOK is a reference to the complexity of mental processing that must occur to answer a question, perform a task, or generate a product.
(Show House Video-clip)
We must understand that DOK is not about the difficulty of the objective, rather, it is about the intended outcome or the complexity of the mental processing that must occur in order to get to the intended outcome. Consider the example of adding This is easy. Students can do this by rote. It’s a DOK 1.
Increasing the difficulty of the problem by asking a student to add 4, ,530 does not increase the complexity. To solve this problem requires a child to recall the sequence of steps. The level of cognitive difficulty is still a DOK 1. It’s still recall. Knowing the rules of adding larger numbers - increasing the difficulty of the addition problem - does not affect the intended outcome. We still want the student to be able to add, just larger numbers.

9. Understanding Depth of Knowledge
Difficulty is a reference to how many students answer a question correctly:
How many of you know the definition of exaggerate?
DOK 1 - recall
If all of you know the definition, this question is an easy question.
How many of you know the definition of prescient?
If most of you do not know the definition, this question is a difficult question.
Copyright © 2007 Mississippi Department of Education
Let’s look at another example of how the level of difficulty can be increased without increasing the level DOK.
Raise your hand if you know the definition of exaggerate.
Raise your hand if you know the definition of prescient.
A common confusion is that in order to make an objective more difficult, we must increase the level of DOK. This is simply NOT the case. The questions are asking the student to recall information…one definition might be considered less likely to be known that the other, but the objective remains the same…provide a definition. (RECALL). We can raise the level of difficulty WITHOUT raising the DOK level of the objective.
Prescient: press-ee-ant, preshent, pree-see-ent: perceiving the significance of events before they occur. To know beforehand.

10. Understanding Depth of Knowledge
Words like explain or analyze have to be considered in context.
“Explain to me where you live” does not raise the DOK of a simple rote response.
Even if the student has to use addresses or landmarks, the student is doing nothing more than recalling and reciting.
Copyright © 2007 Mississippi Department of Education
Even though the verbs (or action words) are listed on graphics explaining DOK, it is not the verb that is critically important, rather it is the words that follow the action. What is the objective really asking the student to do? In the above example, the word “Explain” may be perceived as a higher level objective, however, it simply asks the student to recall information.

11. Depth of Knowledge Levels
Level 1: Recall Recall, recognition; skill, behavior or sequence of behaviors learned through practice and easily performed Level 2: Skill/Concept Engagement of some mental processing beyond recalling; the use of information or conceptual knowledge; requires making some decisions regarding how to approach a question or problem Level 3: Strategic Thinking More sophisticated reasoning and analysis; deep understanding; students are required to solve problems & draw conclusions Level 4: Extended Thinking Requires integration of knowledge from multiple sources and ability to represent knowledge in a variety of ways; usually requires work over an extended period of time
Use this slide as participations work through the word sort….
Explain that they have four levels of DOK within the content areas. Partners will sort according to DOK levels.
When everyone finishes their sort…show the next few slides so they can check their answers.

12. DOK Level 1 Examples
List animals that survive by eating other animals.
Locate or recall facts explicitly found in text
Describe physical features of places
Determine the perimeter or area of rectangles given a drawing or labels
Identify elements of music using musical terminology
Identify basic rules for participating in simple games and activities
Recall and reproduction Level 1
DOK 1 requires recall of information, such as facts, definition, term or performance of a simple process or procedure
Answering a Level 1 item can involve following a simple, well-known procedure or formula. Simple skills and abilities or recall characterize DOK 1.

13. DOK Level 2 Examples Compare desert and tropical environments
Identify and summarize the major events, problem, solution, conflicts in literary text
Explain the cause-effect of historical events
Predict a logical outcome based on information in a reading selection
Explain how good work habits are important at home, school, and on the job.
Classify plane and three dimensional figures
Describe various styles of music
DOK 2 includes the engagement of some mental processing beyond recalling or reproducing a response. Items require students to make some decisions as to how to approach the question or problem.
These actions imply more than one mental or cognitive process/step.

14. DOK Level 3 Examples
Compare consumer actions and analyze how these actions impact the environment
Analyze or evaluate the effectiveness of literary elements (e.g. characterization, setting, point of view, conflict and resolution, plot structures)
Solve a multiple-step problem and provide support with a mathematical explanation that justifies the answer
Level 3 Strategic thinking
DOK 3 requires deep understanding as exhibited through planning, using evidence, and more demanding cognitive reasoning. The cognitive demands at Level 3 are complex and abstract.
An assessment item that has more than one possible answer and requires students to justify the response they give would most likely be a Level 3.

15. DOK Level 3 Examples Develop a scientific model for a complex idea
Propose and evaluate solutions for an economic problem
Explain, generalize or connect ideas, using supporting evidence from a text or sourc
Create a dance that represents the characteristics of a culture

16. DOK Level 4 Examples
Gather, analyze, organize, and interpret information from multiple (print and non print sources) to draft a reasoned report
Analyzing author’s craft (e.g., style, bias, literary techniques, point of view)
Create an exercise plan applying the “FITT (Frequency, Intensity, Time, Type) Principle
DOK 4 requires high cognitive demand and is very complex. Students are expected to make connections – related ideas within the content or among content areas – and have to select or devise one approach among many alternatives on how the situation can be solved.
Due to the complexity of cognitive demand, DOK 4 often requires an extended period of time. However, extended time alone, is not a distinguishing factor.

17. DOK Level 4 Examples
Analyze and explain multiple perspectives or issues within or across time periods, events, or cultures
Specify a problem, identify solution paths, solve the problem, and report the results
Write and produce an original play

18. Caution!
The Depth of Knowledge is NOT determined by the verb, but the context in which the verb is used and the depth of thinking required.

19. Same VERB – three DOK Levels Science
DOK? – Describe a model that you might use to represent the relationships that exist within the rock cycle
DOK ? – Describe the differences between metamorphic and igneous rocks
DOK ? – Describe three characteristics of metamorphic rocks
DOK 3 (requires deep understanding of rock cycle and a determination of how best to represent it.
DOK 2 (requires cognitive processing to determine the differences in the two types of rocks)
DOK 1 (simple recall)

20. Same VERB – three DOK Levels Social Studies
DOK ?  examine the amendment process of the West Virginia Constitution, give examples of amendments and explain why they occurred. 
DOK ?   explain how nations benefit when they resolve conflicts peacefully.        
DOK ? Use correct geographic terminology to explain direction, location, time zones, physical features of the earth.
DOK 3
DOK 2
DOK 1

21. Same VERB – three DOK levels Secondary Mathematics
DOK ? Explain what family of functions would best model a particular real-world situation.
DOK ? Explain the difference between a linear and a quadratic function.
DOK ? Explain how the “vertical line test is useful in determining whether or not a given graph is representing a function.
DOK 3
DOK 2
DOK 1

22. Same VERB – three DOK levels Elementary Mathematics
DOK ? Determine the formula for the area of a rectangle and explain reasoning through modeling
DOK ? Demonstrate and model multiplication (repeated addition, arrays) and division (repeated subtraction, partitioning)
DOK ? Model, describe, and draw – lines, rays, angles including right, obtuse, and acute angles.
DOK 3
DOK 2
DOK 1

23. Practice Activity: Sample One
M.O Read and interpret information represented on a circle graph.
What is the DOK level? Answer is level two. What makes this a DOK 2 – not a DOK 1?
(the word interpret) We can make it a level one by removing the word “interpret”.

24. Practice Activity: Sample Two
Solve one-step linear equations and inequalities with one variable, interpret the solution or solutions in the context from which they arose, and verify the reasonableness of the results.
A one-step linear equation sample might be: 20 = 3X X= 6 is not a reasonable answer for this sample equation.
This objective is an example of Level 3. The expectation expressed in this objective is that students will not only solve and verify the results. This will require students to do some reasoning in order to interpret the solution and could be fairly complex depending on the context. If students were only required to solve linear equations and verify solutions, then the expectation would be Level 2.

25. Practice Activity: Sample Three
Design a statistical experiment to study a problem and communicate the outcomes.
This is a DOK 4 because to plan a statistical experiment, a student must define the problem and develop a procedure for solving it. This involves identifying the correct statistical model, applying the model to data and communicating the outcome of the selected model. The student must interpret finding and make reasonable and rationed inferences from obtained data. This represents a complex, multistep reasoning task.

26. DOK Levels Can Be Cumulative
An item/standard written to DOK 3 often contains DOK 1 and DOK 2 level demands.
This goes into the deconstruction aspects of looking at our CSOs. However, remember that the assessment items are written at the highest level of DOK within the CSO. We rate the CSO at the highest level as well.

27. Remember… Depth of Knowledge (DOK) is a scale of cognitive demand.
DOK requires looking at the assessment item/standard – not student work – in order to determine the level. DOK is about the item/standard – not the student.
The context of the assessment item/standard must be considered to determine the DOK – not just a look at what verb was chosen.

28. WVDE Office of Instruction Lou Maynus Mathematics Coordinator lmaynus@access.k12.wv.us 304-558-5325