1. DEPTH OF KNOWLEDGE (DOK) What are students being asked to do? How complex is the student thinking?

2. Depth of Knowledge (DOK) is A scale of cognitive demand (thinking) to align standards with assessments Descriptive of the task students are being asked to do Based on the research of Norman Webb, University of Wisconsin Center for Education Research and the National Institute for Science Education  Not the same as difficulty

3.  To align instructional activities with the cognitive rigor needed to master the standards  Is used by Smarter Balanced to indicate the complexity of the assessment task/item  Will be used to develop new NGSS assessments  Applicable to all subject areas and at all grade levels

4. Varying Level of Cognitive Demand ELA Retell or summarize Little Red Riding Hood in your own words. What is your opinion about the intelligence of the wolf (in Little Red Riding Hood)? Justify using details/evidence from the story.

5. Varying Level of Cognitive Demand Mathematics Is 4/5 closer to 1 than 5/4? 4/5 is closer to 1 than 5/4. Show why this is true on a number line.

6. How are the new assessments changing? MATHEMATICSELA / LITERACY DOK 3DOK 4DOK 3DOK 4 Current Assessments<2%0%20%2% New SBAC Assessments49%21%43%25% Yuan & Le (2012); Herman & Linn (2013), from Linda Darling-Hammond Assembly Testimony, 3.6.13 Smarter Balanced Depth of Knowledge Goals for Items

7. Write a definition of Cognitive Rigor on your DOK note-taking guide. Read “Worldwide Loss of Bees”– develop one basic question and one “rigorous” question. Share your questions with a partner

8. Rating Depth of Knowledge (DOK)

9. In your classroom: What part of your classroom instruction is spent in each DOK Level? Which DOK levels are assessments focused on? What lessons / activities engage students in the different DOK Levels? What DOK level would you assign your “Ant and The Grasshopper” questions?

10. Webb’s 4 Levels of Cognitive Complexity DOK 1: Recall & Reproduction DOK 2: Basic Application of Skills/Concepts DOK 3: Strategic Thinking and Reasoning DOK 4: Extended Thinking

11. LEVEL 1: RECALL Requires recall of information, such as a fact, definition, term, or simple procedure and/or performing simple procedures. Involves only basic initial comprehension, not an analysis or interpretation. Students are to work with specific facts or definitions. The activity requires a shallow understanding of the topic or text.

12. LEVEL 2: Skills and Concepts Includes the engagement of some mental processing beyond recalling or reproducing a response Items require students to make some decisions as to how to approach the question or problem Actions imply more than one mental or cognitive process/step

13. Recall vs. Skills & Concepts MATH How many minutes did it take Selena to do 3 problems? Selena continues to work at the same rate. How many problems will she do in 40 minutes? Selena’s

14. Recall vs. Skills & Concepts Science Identify parts of the cell Distinguish between a plant cell and an animal cell

15. LEVEL 3: Strategic Thinking Requires deep understanding exhibited through planning, using evidence, and more demanding cognitive reasoning The cognitive demands are complex and abstract An assessment item that has more than one possible answer and requires students to justify the response would most likely be a Level 3

16. 3. Strategic Thinking Mathematics Solve a multi-step problem, and provide support with a mathematical explanation that justifies the answer. Beto and Jose were playing a game. At the end of each game, the loser gave the winner a penny. After a while, Jose won 3 games and Beto had 3 more pennies than he did when he began. How many games did they play.

17. 3. Strategic Thinking A species may become extinct if environmental changes occur and the species does not adapt quickly enough to the changes. 1. Identify an environmental change that might cause a species to become extinct and identify a species that would likely be affected by such a change. 2. Describe how extinction of one species can affect other organisms in the ecosystem

18. Extended Thinking: Level 4 Requires high cognitive demand and is very complex Students are expected to make connections, relate ideas within the content or among content areas, and select or devise one approach among many alternatives on how the situation can be solved Due to the complexity of cognitive demand, DOK 4 often requires an extended period of time

19. Extended Thinking ELA/Science Construct an explanation based on evidence for how the availability of natural resources, occurrence of natural hazards, and changes in climate have influenced human activity. (NGSS HS-ESS3-1.)

20. Extended Thinking Science Evaluate competing design solutions for maintaining biodiversity and ecosystem services.* (NGSS MS-LS2-5.)

21. TaskThinkingDOK Level Collecting data samples over several months Recall1 Organizing data in a chartSkills / Concept2 Using charts and graphs to make predictions and justify conclusions with evidence Strategic Thinking 3 Developing a generalized model from data, graphs and analysis and applying it to a new situation or to solve a new problem Extended thinking 4

22. DOK an Introduction Take a few minutes to read through the descriptions or bullets for the levels of DOK 1-4 in Science How does your original definition of Cognitive Rigor Change?

23. Rate the Examples Level of DOK Compare and contrast desert and tropical climates. After observing a creek bed over time and analyzing data, solve a problem to improve the ecosystem. Use a glossary to define the term evaporation Explain the cause- effect relationship of dropping an object from different heights Try it!

24. Rate the Examples Level of DOK Identify the elements in Nitric Acid Evaluate the validity and reliability of claims in published materials of the effects that different frequencies of electromagnetic radiation have when absorbed by matter. Construct an explanation for the anatomical similarities and differences among modern organisms and between modern and fossil organisms to infer evolutionary relationships. Try it!

25. Rate the Level of DOK Draw and label a diagram that shows how a mid-ocean ridge forms and describe two possible consequences that the formation of mid-ocean ridges has on other locations on Earth Complete research regarding Organic, GMO and other agricultural practices. Compare legislation from around the world. Create a document with your point of view. Determine the average speed of an object when given the distance it travels in a specific amount of time Explain the process of radioactive decay Try it!

26. The depth of knowledge level is NOT determined by the verb, but by the context in which the verb is used and the complexity of thinking required. Depth of Knowledge

27. Same Verb – 3 Different Levels of DOK DOK 1- Describe three characteristics of metamorphic rocks. (Requires simple recall) DOK 2- Describe the difference between metamorphic and igneous rocks. (Requires cognitive processing to determine the differences in the two rock types) DOK 3- Describe a model that you might use to represent the relationships that exist within the rock cycle. (Requires deep understanding of rock cycle and a determination of how best to represent it) Caution 1

28. Sort the problems by DOK Levels Choose a set of cards With your partner, sort the 8 cards based on the DOK level Compare your results with your table partners

29. DOK is about intended outcome not about Difficulty DOK is a reference to the complexity of mental processing that must occur to answer a question, perform a task, or generate a product. Adding is a mental process Knowing the rule for adding is the intended outcome that influences the DOK Once someone learns the “rule” of how to add, 4 + 4 is DOK 1 and is also easy Adding 4,678,895 + 9,578,885 is still a DOK 1 but may be more “difficult”

30. Points to Consider DOK 1 + DOK 1 + DOK 1 = 1 Depths of knowledge classification is based on the task, not the student DOK is different from task/item difficulty DOK is not a grade level indicator DOK ratings aid in alignment of standards and assessment, and therefore instruction

31. How Many of Each Type of Problem? Colorado Department of Education

32. Creating DOK Questions and Activities at all levels

33. DOK: Questions at lower levels are usually more appropriate for: Evaluating students’ preparation and comprehension Diagnosing students’ strengths and weaknesses Reviewing and/or summarizing Usually questions at upper DOK levels are appropriate for: Encouraging students to think deeply and critically Problem solving Encouraging Discussions Stimulating students to seek information on their own From Karin Hess

34. DOK 1, 2, 3, 4 Level 1— Identify the tree Level 2— Explain the function of the leaves Level 3— Explain how a drought might affect the growth of the tree Level 4— Design an investigation of seedling growth to determine the best fertilizer for this type of tree. Oak Tree

35. DOK 1, 2, 3 and 4 Level 1— Identify this utensil Level 2— Level 3— Level 4—

36. Reflect on your learning… Revisit your definition of rigor – has it changed or been refined? How? What is one way you might apply these ideas to your work? How might you shift your classroom instructional and assessment practices? What existing materials could you examine for a range of cognitive rigor? From Karin Hess

37. Questions and Thoughts What questions and thoughts do you still have regarding DOK?