1. Introduction to Depth of Knowledge
Jamie Marantz
Director
Core Learning
Dublin Unified School District Presentation

2. Dublin Unified School District Presentation
Learning Targets
I will be able to evaluate assessment questions using Webb’s Depth of Knowledge Levels.
Dublin Unified School District Presentation

3. Why is Depth of Knowledge (DOK) Important?
Use it to ensure that instructional and assessment activities match the cognitive rigor required to master the standards.
Dublin Unified School District Presentation

4. Dublin Unified School District Presentation
DOK is NOT…
… about the ‘TYPE’ of thinking
Verbs (i.e. Bloom’s) such as analyzing, comprehending, recalling
…about the ‘DIFFICULTY’
Difficulty is about how many students answer a question correctly.
Dublin Unified School District Presentation

5. Same Verb does Not Equal Same TYPE or DEPTH of Thinking
While verbs may point to a DOK level, it is what comes after the verb that is the best indicator of the rigor/DOK level.
DOK 1- Describe three characteristics of plant cells
DOK 2- Describe the difference between plant and animal cells.
DOK 3- Describe a model that you might use to represent a key relationship between plant and animal cells based on their functions.
Dublin Unified School District Presentation

6. DOK is not about difficulty...
Difficulty refers to how many students answer a question correctly.
DOK 1 – recall:
“How many of you know what ‘angry’ means?”
If you know, it’s an easy question.
“Who knows what ‘prescient’ means?”
If most do not know, then you have a ‘hard’ or difficult seeming question.
Dublin Unified School District Presentation

7. DOK is about COMPLEXITY
DOK refers to the complexity of the mental processing students must perform to complete a task, answer a question or create a product.
How deeply do you have to know the content to complete the task.
Instruction and assessment activities must reflect the DOK level of the objective or intended learning outcome.
Dublin Unified School District Presentation

8. Dublin Unified School District Presentation
DOK Levels
DOK-1 : Recall, Remember and Reproduction
DOK-2 : Basic Application of Skills & Concepts
DOK-3 : Strategic Thinking
DOK-4 : Extended Thinking
Dublin Unified School District Presentation

9. Level 1- Recall and Reproduction
DOK 1 requires recall of information, such as a fact, definition, term, or performance of a simple process or procedure.
You could memorize something simple, or complex
DOK 1 can involve perform a routine procedure.
Dublin Unified School District Presentation

10. Dublin Unified School District Presentation
Level 1 Examples
Find facts explicitly stated in a text.
Describe the components of a group.
Identify elements from the periodic table from given facts.
Identify basic rules for performing subtraction of double digit numbers.
Dublin Unified School District Presentation

11. Level 2 – Skills and Concepts
DOK 2 requires
Organizing of information in some way
Using some mental processing like determining relationships between items beyond recalling
Items can require students to make some decisions as to how to approach the question or problem.
These actions imply more than one cognitive process/step.
For DOK 1 + 2, there’s usually one ‘right’ answer.
Dublin Unified School District Presentation

12. Dublin Unified School District Presentation
Level 2 - Examples
Explain the cause-effect of a collision using Newton’s 2nd Law.
Distinguish fact from opinion in a text.
Compare tundra and arctic seasons. Identify and summarize the major events, problem, solution, conflicts in a literary text.
Dublin Unified School District Presentation

13. Level 3 – Strategic Thinking
DOK 3 questions can have multiple ways to get to the right answer or multiple right answers.
DOK 3 requires deep understanding as exhibited through reasoning, planning, and more demanding cognitive reasoning. The cognitive demands are often complex and/or abstract.
DOK 3 questions often have more than one possible answer and often require students to justify the response they give using and explaining evidence for their reasoning.
Dublin Unified School District Presentation

14. Dublin Unified School District Presentation
Level 3 - Examples
Create a children’s story that represents the characteristics of a culture
Develop a scientific model for the results of our experiment
Explain, the most significant effect of WWII on the nations of Europe
Propose and evaluate solutions for a social problem
Dublin Unified School District Presentation

15. Level 4 – Extended Thinking
DOK 4 requires high cognitive demand and is very complex. Students are expected to make connections—relate ideas within the content or among content areas—and have to select or devise one approach among many alternatives on how the situation can be solved.
Due to the complexity of cognitive demand, DOK 4 often requires an extended period of time.
Dublin Unified School District Presentation

16. Dublin Unified School District Presentation
Level 4 – Examples
Gather, analyze, organize, and interpret information from multiple (print and non print sources) to draft a reasoned report.
Write a report that illustrates how multiple themes (historical, geographical, social) may be interrelated.
Develop generalizations of the results obtained or strategies we used and apply them to this new problem.
Dublin Unified School District Presentation

17. DOK levels can be cumulative
An item/standard written to DOK 3 often contains DOK 1 and DOK 2 level demands.
Dublin Unified School District Presentation

18. Assigning Depth of Knowledge to Assessments Activity
Read each of the following assessment Items and Assign a Depth of Knowledge level to each Question and then compare with others. Justify your reasoning.
Use language structures (Pre/suffix) or word relationships (antonym/synonym) to determine word meanings.
Analyze/interpret author’s craft (literary devices, viewpoint, potential bias) to critique the text.
Explain the relationship between the action and resolution of the story.
Dublin Unified School District Presentation

19. Transitioning to Next Generation Assessments
Ingrid Roberson
Professional Expert
Assessment

20. Dublin Unified School District Presentation
Learning Targets
I will be able to assess the Depth of Knowledge required of individual questions.
I will be able to identify key differences between the California Standards Test (CST) and SBAC.
Dublin Unified School District Presentation

21. CST/SBAC Compare and Contrast Activity
Task: Compare and contrast the California Standards Test and the Smarter Balanced Assessment.
Dublin Unified School District Presentation

22. CST/SBAC Compare and Contrast Activity
CST Review (20 minutes)
As a pair, select 4 CST questions across different strands.
As a pair, review the first CST question together, assigning a Depth of Knowledge level and citing evidence for your answer.
Individually review the 3 remaining CST questions.
Now, share your results and reasoning with your partner.
Dublin Unified School District Presentation

23. CST/SBAC Compare and Contrast Activity
SBAC Review (20 minutes)
As a pair, select 4 SBAC questions across different strands.
As a pair, review the first SBAC question together, assigning a Depth of Knowledge level and citing evidence for your answer.
Individually review the 3 remaining SBAC questions.
Now, share your results and reasoning with your partner.
Dublin Unified School District Presentation

24. CST/SBAC Compare and Contrast Activity
Conclusion (10 minutes)
As a pair, decide what is the most important difference between the CST and the SBAC? Discuss the effects of this difference for your instruction.
Dublin Unified School District Presentation

25. Unpacking the Smarter Balanced Assessments
Ingrid Roberson
Professional Expert
Assessment
Dublin Unified School District Presentation

26. Dublin Unified School District Presentation
Learning Targets
I can describe main features of the SMARTER Balanced Assessment Consortium (SBAC).
TIME: 10 minutes (CONTINUED for Slides 58 to 63)
INTENT: To provide an overview of the broad features of SMARTER Balanced Assessment Consortium’s (SBAC’s) assessment plan
__________________________________________________________________________________ _
There are still many unanswered questions about what the 2014 assessments will look like. Here is what we know:
Although originally associated with PAARC, California has recently signed on with SBAC and has opted to take a governing role. This means California educators will take a strong role in developing the assessments.
Unlike current CST assessments, the new CCSS assessments will be computer-based and be adaptive. Adaptive means that the questions will adjust in difficulty level as the student answers. This will prevent frustration and burnout as students take the test—and thus give a more accurate read of their proficiency levels.
In addition to end of the year summative assessment, SBAC will provide item banks and performance assessments for districts to locally administer benchmarks.
Performance assessments will be both single period and extended—over several days and incorporating speaking and listening tasks.
Dublin Unified School District Presentation

32. Ingrid Roberson, Professional Expert Assessment iroberson@acoe.org
Thank you!
Ingrid Roberson,
Professional Expert
Assessment
Jamie Marantz,
Director,
Core Learning
Dublin Unified School District Presentation