1. from Infusing Rigor & Research into Instruction and Assessment presentation USOE, Salt Lake City, UT February 25-26, 2014 Karin K. Hess, Ed.D. Center for Assessment khess@nciea.org DEPTH OF KNOWLEDGE

2. What is “rigor”? Take a few minutes to write your personal definition of cognitive rigor, as it relates to instruction, learning & assessment.

3. Let’s apply your rigor definition Your class has just read some version of Little Red Riding Hood. What is a basic comprehension question that you might ask? What is a more rigorous question you might ask?

4. Cognitive Rigor Matrix Benjamin Bloom Norm Webb Types of Thinking Depth of Thinking (Bloom’s Modified Taxonomy) or Depth of Knowledge Karin Hess Different Types of Thinking At Different Levels of Depth

5. Bloom’s Taxonomy [1956 ] & Bloom’s Cognitive Process Dimensions [2001]

6. Webb’s Depth of Knowledge Levels DOK-1 – Recall & Reproduction - Recall of a fact, term, principle, concept; perform a routine procedure; locate details DOK-2 - Basic Application of Skills/Concepts - Use of information; conceptual knowledge; select appropriate procedures for a given task; two or more steps with decision points along the way; routine problems; organize/display data; interpret/use simple graphs; summarize; identify main idea; explain relationships; make predictions DOK-3 - Strategic Thinking - Requires reasoning, or developing a plan or sequence of steps to approach problem; requires decision making or justification; abstract, complex, or non-routine; often more than one possible answer; support solutions or judgments with text evidence DOK-4 - Extended Thinking - An investigation or application to real world; requires time to research, problem solve, and process multiple conditions of the problem or task; non-routine manipulations; synthesize information across disciplines/content areas/multiple sources

7. DOK is about complexity---not difficulty! Describe the information contained in graphics or data tables in the text; or the rule for rounding a number Describe how the two characters are alike and different. Describe the data or text evidence that supports your solution, reasoning, or conclusions. Describe varying perspectives on global climate change using supporting scientific evidence, and identify the most significant effect it might have on the planet in 100 years.

8. The Hess Cognitive Rigor Matrix integrates Bloom + Webb Different states/schools/teachers use different models to describe cognitive rigor. Each addresses something different. Bloom – What type of thinking (verbs) is needed to complete a task? Webb – How deeply do you have to understand the content to successfully interact with it? How complex is the content?

10. Let’s practice using the Cognitive Rigor Matrix… Where would you place your LRRH questions in the CRM- basic and more rigorous questions?

12. SOME GENERAL RULES OF THUMB… If there is one correct answer, it is probably level DOK 1 or DOK 2 DOK 1: you either know it (can recall it, locate it, do it) or you don’t know it DOK 2 (conceptual): apply one concept, then make a decision before going on applying a second concept; express relationship (if-then; cause-effect) If more than one answer/approach, requiring evidence, it is DOK 3 or 4 DOK 3: Must interpret, provide supporting evidence and reasoning (not just HOW solved, but WHY it works– explain reasoning for each step/decision made) DOK 4: all of “3” + use of multiple sources/data/ texts; initiate & complete an investigation

13. COMMON CORE READING & WRITING

14. Let’s look at the CRM in regards to writing

15. Quick Tips for Differentiation through DOK Here are 3 ways to approach differentiating curriculum without creating completely different lessons – Focus on one aspect (content/concepts, process, or product) at a time, or use combinations of the three components through an assessment menu Content (texts/ concepts) – different students get different – but related – content/texts (e.g., different texts on same topic or by same author, with different levels of complexity; different texts that relate to topic, concept, theme, or same genre – different origins of myths, different subtopics or perspectives related to a study of cultures or time periods) Process (Depth of Knowledge/DOK) – vary the DOK/process skills for the same text/content (e.g., work independently or with others; analyze within one text or across multiple texts; compare & contrast versus deeper analysis). This differentiation can provide scaffolding for students who may need it before completing a more complex assignment. Product – same content (text/concepts) and application of same process skills (DOK), but products may vary by choice/strengths/interest (e.g., presentation, pamphlet, poster, letter, model, use of technology, illustration, etc.). Assess products with a “common” rubric.

16. “Hard” vs. “Rigorous” from So What do They Really Know?, Chris Tovani Students avoid it The goal is the same for everyone, no matter their starting point Learner becomes discouraged Someone other than the learner has timetable Learner need extrinsic reward Students embrace it Individualized goal for learner—what’s the next step for him/her? Success comes in little steps—not an end Learner has more influence on timetable Learner responds to intrinsic reward HardRigorous