1. DOK and HOTS Presented By: Cisely Scott Julian Davenport

2. DATE: June 23, 2011 BENCHMARK: DOK, HOTS, and Questioning Strategies OBJECTIVE: Today we will examine the role Higher Order Thinking, Depth of Knowledge, and Questioning Strategies on student achievement. ESSENTIAL QUESTION: How does increasing the rigor in your questioning enhance the performance of your students in the classroom? What are the long term implications of using these strategies? VOCABULARY: Depth of Knowledge, Cognitive Complexity (low, medium, high), Difficulty vs. Complexity, HOT Verbs, Questioning Strategies, Inquiry Based Learning, AGENDA: What the Research Says…. What is Depth of Knowledge? Application to FCAT Elements of Inquiry Activity-Developing a HOT Question How Do We Ensure Use in the Classroom? Journal Reflection: How does the use of HOTs questions, DOK, and Questioning Strategies affect teacher performance and student achievement? EXIT SLIP: IN YOUR JOURNAL Connect: How does the use of HOTs questions, DOK, and Questioning Strategies affect teacher performance and student achievement? Connect: 1 question you still have about FCAT 2.0. HOME LEARNING: Review today’s lesson and develop your next steps to share this information with teachers at your school Common Board Configuration (CBC)

3. ESSENTIAL QUESTION (EQ) How does increasing the rigor in teacher questioning enhance the performance of your students in the classroom? What are the long term implications of using these strategies?

4. REACHING YOUR VISION This presentation will help you to reach your vision as a coach by providing you with an overview DOK, HOT, and Questioning Strategies so that you’re able to monitor for and model these strategies within the classroom. This presentation will help you to reach your vision as a coach by providing you with an overview DOK, HOT, and Questioning Strategies so that you’re able to monitor for and model these strategies within the classroom.

5. What the Research Says……

6. Teachers are the Key “Teachers must be the primary driving force behind change. They are best positioned to understand the problems that students face and to generate possible solutions.” James Stigler and James Hiebert, The Teaching Gap

7. Quality Instruction Makes A Difference “Good teaching can make a significant difference in student achievement, equal to one effect size (a standard deviation), which is also equivalent to the affect that demographic classifications can have on achievement.” Paraphrase Dr. Heather Hill, University of Michigan

8. Differences in Instruction “Our research indicates that there is a 15% variability difference in student achievement between teachers within the same schools.” Deborah Loewenberg Ball, Dean of Education, University of Michigan

9. “What Matters Very Much is Which Classroom?” “If a student is in one of the most effective classrooms he or she will learn in 6 months what those in an average classroom will take a year to learn. And if a student is in one of the least effective classrooms in that school, the same amount of learning take 2 years.”

10. Research has indicated that... “teacher quality trumps virtually all other influences on student achievement.” (e.g., Darling-Hammond, 1999; Hamre and Pianta, 2005; Hanushek, Kain, O'Brien and Rivken, 2005; Wright, Horn and Sanders, 1997)

11. So what are effective teachers doing differently? What should you be looking for as a coach? Discuss with your neighbor the answer to this question. (5-10 mins). Come up with a list of 10 things to share.

12. Rigorous Classroom Standards-based teaching Look for Lots of high level activity- - -  High Levels of Questioning  Reflecting  Analyzing  Doing experiments (science)  Doing exploratory activities  Discussing/Debates  Writing  Working in groups And a Scoring Guide available to all students for all major assignments  Sets expectations 12

13. Rigorous Classroom Only by creating a culture of high expectations and providing support so students can truly succeed do you have a rigorous classroom. Barbara Blackburn 2008 Barbara Blackburn 2008 13

14. What is Depth Of Knowledge?

16. Cognitive Demand The kind and level of thinking required of students to successfully engage with and solve a task Ways in which students interact with content

17. Why Depth of Knowledge? F o c u s e s o n c o m p l e x i t y o f c o n t e n t s t a n d a r d s i n o r d e r t o s u c c e s s f u l l y c o m p l e t e a n a s s e s s m e n t o r t a s k. T h e o u t c o m e ( p r o d u c t ) i s t h e f o c u s o f t h e d e p t h o f u n d e r s t a n d i n g.

18. Why Use a Depth of Knowledge? Used to determine the level of the expected outcomes of the Next Generation Sunshine State Standards and benchmarks Used to determine the level of the expected outcomes of the Next Generation Sunshine State Standards and benchmarks Determines the complexity of FCAT items (success with items leads to AYP) Determines the complexity of FCAT items (success with items leads to AYP)

19. Verbs are not always used appropriately... Words like explain or analyze have to be considered in context. “Explain to me where you live” does not raise the DOK of a simple rote response. Even if the student has to use addresses or landmarks, the student is doing nothing more than recalling and reciting.

20. DOK is about what follows the verb... What comes after the verb is more important than the verb itself. “Analyze this sentence to decide if the commas have been used correctly” does not meet the criteria for high cognitive processing.” The student who has been taught the rule for using commas is merely using the rule.

21. Same Verb—Three Different DOK Levels DOK 1- Describe three characteristics of metamorphic rocks. (Requires simple recall) DOK 2- Describe the difference between metamorphic and igneous rocks. (Requires cognitive processing to determine the differences in the two rock types) DOK 3- Describe a model that you might use to represent the relationships that exist within the rock cycle. (Requires deep understanding of rock cycle and a determination of how best to represent it)

22. Activity   In your groups, use the word explain, analyze, or any of the other words on the ladder to come up with a low complexity question and a higher complexity question using the verb. (5 minutes)

23. DOK is about intended outcome, not difficulty DOK is a reference to the complexity of mental processing that must occur to answer a question, perform a task, or generate a product. Adding is a mental process. Knowing the rule for adding is the intended outcome that influences the DOK. Once someone learns the “rule” of how to add, 4 + 4 is DOK 1 and is also easy. Adding 4,678,895 + 9,578,885 is still a DOK 1 but may be more “difficult.”

24. DOK is not about difficulty... Difficulty is a reference to how many students answer a question correctly. “How many of you know the definition of exaggerate?” DOK 1 – recall If all of you know the definition, this question is an easy question. “How many of you know the definition of vociferous?” DOK 1 – recall If most of you do not know the definition, this question is a difficult question.

25. DOK is about complexity The intended student learning outcome determines the DOK level. Every objective in the science and mathematics frameworks has been assigned a DOK level. Instruction and classroom assessments must reflect the DOK level of the objective or intended learning outcome.

26. Quick Quiz 1) Give an example of a statement that uses a verb that “sounds” like a high DOK but is used inappropriately. 2) Fill in the blanks: What _____ the verb is more _____ than the verb itself when deciding the DOK level. 3) What is the difference between difficulty and complexity? 4) What really determines the DOK level? Copy this down in your notebook?

27. Quick Quiz Answers 1) Give an example of a statement that uses a verb that “sounds” like a high DOK but is used inappropriately. answers vary 2) Fill in the blanks: What follows the verb is more important than the verb itself when deciding the DOK level. 3) What is the difference between difficulty and complexity? answers vary, but do not rely on the verb 4) What really determines the DOK level? the intended learning outcomes

28. How does this apply to FCAT?

29. Correlation to FCAT

30. Percentage of Points by Cognitive Complexity Level for FCAT SSS Reading

31. FCAT Science 2.0 Here is the new breakdown, based on Webb’s DOK: Note: A high complexity question in 5 th grade may not be a high complexity question in 8 th grade. The complexity is ultimately based on the cognitive demand placed on the student. Cognitive complexity also may depend on the level of the distractor.

32. Cognition Classification Schemes Classification schemes for cognition allow communication about the nature and characteristics of thinking and learning. Examples: Bloom’s Taxonomy and Webb’s Depth of Knowledge

33. What is Webb’s Depth of Knowledge? Webb’s Levels of Cognitive Complexity  Low Complexity relies on the recall and recognition of previously learned concepts and principles  Moderate Complexity involves more flexible thinking and choice among alternatives  High Complexity requires students to engage in more abstract reasoning, planning, analysis, judgment, and creative thinking

34. Complexity v. Difficulty The degree of challenge of FCAT items is currently categorized in two ways:  Cognitive Complexity  Determined by what the item requires the student to recall, understand, analyze, and do.  Assume student is familiar with basic concepts of the task and focuses on the task.  Depends on the task not the student. Item DifficultyItem Difficulty  Depends on the percentage of students likely to answer correctly. Easy – More than 70%Easy – More than 70% Average – 40% to 70%Average – 40% to 70% Challenging – Less than 40%Challenging – Less than 40%

35. Low Complexity DOK Level 1  Low complexity items rely heavily on the recall and recognition of previously learned concepts and principles.  Items typically specify what the student is to do, which is often to carry out some procedure that can be performed mechanically.  It is NOT left to the student to come up with an original method or solution.  Skills required to respond to low complexity items include: solving a one-step problem;solving a one-step problem; computing a sum, difference, product, or quotient;computing a sum, difference, product, or quotient; evaluating a variable expression, given specific values for the variables;evaluating a variable expression, given specific values for the variables; recognizing or constructing an equivalent representation;recognizing or constructing an equivalent representation; recalling or recognizing a fact, term, or property;recalling or recognizing a fact, term, or property; retrieving information from a graph, table, or figure;retrieving information from a graph, table, or figure; identifying appropriate units or tools for common measurements;identifying appropriate units or tools for common measurements; performing a single-unit conversion;performing a single-unit conversion; reproducing a diagram or standard representation;reproducing a diagram or standard representation; completing a routine procedure, such as measure temperature; andcompleting a routine procedure, such as measure temperature; and calculating using a common formula.calculating using a common formula.

36. Moderate Complexity DOK Level 2  FCAT Mathematics moderate complexity items involve more flexible and choice among alternatives.  Items require a response that goes beyond the habitual, is not explicitly specified in the text, and ordinarily has more than a single step.  The student is expected to decide what to do using informal methods of reasoning and problem solving strategies and to bring together skill and knowledge from various domains.  Skills required to respond to moderate complexity items include: solving a problem requiring multiple operations;solving a problem requiring multiple operations; solving a problem involving spatial visualization and/or reasoning;solving a problem involving spatial visualization and/or reasoning; selecting and/or using different representations, depending on situation and purpose;selecting and/or using different representations, depending on situation and purpose; retrieving information from a graph, table, or figure and using it to solve a problem;retrieving information from a graph, table, or figure and using it to solve a problem; determining a reasonable estimate;determining a reasonable estimate; extending an algebraic or geometric pattern;extending an algebraic or geometric pattern; providing a justification for steps in a solution process;providing a justification for steps in a solution process; comparing figures or statements;comparing figures or statements; representing a situation in more than one way;representing a situation in more than one way; formulating a routine problem, given data and conditions;formulating a routine problem, given data and conditions; explaining relationships between facts, terms, or properties;explaining relationships between facts, terms, or properties; describing and explain examples and non-examples of science concepts;describing and explain examples and non-examples of science concepts; making decisions and comparisons;making decisions and comparisons; selecting a procedure according to criteria and perform procedure;selecting a procedure according to criteria and perform procedure; organizing, representing, and interpreting data.organizing, representing, and interpreting data.

37. High Complexity DOK Level 3  High complexity items make heavy demands on student thinking.  Students must engage in abstract reasoning, planning, analysis, judgment, and creative thought.  Items require that the student think in a more abstract and sophisticated way.  Skills required to respond correctly to high complexity items include: performing a procedure having multiple steps and multiple decision points;performing a procedure having multiple steps and multiple decision points; solving a non-routine problem (as determined by grade-level appropriateness);solving a non-routine problem (as determined by grade-level appropriateness); solving a problem in more than one way;solving a problem in more than one way; describing how different representations can be used for different purposes;describing how different representations can be used for different purposes; generalizing an algebraic or geometric pattern;generalizing an algebraic or geometric pattern; explaining and justifying a solution to a problem;explaining and justifying a solution to a problem; describing, comparing, and contrasting solution methods;describing, comparing, and contrasting solution methods; providing a mathematical justification;providing a mathematical justification; analyzing similarities and differences between procedures and concepts;analyzing similarities and differences between procedures and concepts; formulating an original problem, given a situation;formulating an original problem, given a situation; formulating a mathematical model for a complex situation;formulating a mathematical model for a complex situation; analyzing or producing a deductive argument;analyzing or producing a deductive argument; identifying research questions and design experimentsidentifying research questions and design experiments developing a scientific model for a complex situation; anddeveloping a scientific model for a complex situation; and forming conclusions and generalizations from experimental data.forming conclusions and generalizations from experimental data.

38. How can I REALLY understand the “depth” of the benchmarks?  Resources to help teachers gain a better understanding of the benchmarks are available.  Professional development, PLCs, Lesson Study, common team planning, and instructional coaching will support teachers as they transition to the NGSSS.  What role should my textbook play as a resource?

39. Elements of Inquiry: Checks for Understanding and Questioning Strategies

40. Top 10 Reasons to Ask Questions 1.Increases student retention of knowledge, skills, and understandings. 2.Fosters an active learning environment to maintain student engagement. 3.Stimulates students’ interest and curiosity in content. 4.Encourages both individual and group accountability. 5.Assesses what students already know. 6.Diagnoses difficulties or road blocks to learning. 7.Helps students monitor their own understandings. 8.Supports students in making conjectures and arguing validity. 9.Allows students to hear different explanations by their peers. 10.Develops student communication skills and social development.

41. Quality Questioning  Quality questioning frames, delivers, times, solicits, and responds to questions in a way that reveals as much information as possible about what students know or do not know.  It’s not, necessarily, about asking more questions.  In fact, research shows that teachers ask a lot of questions in the course of a day…approximately one every 72 seconds, on average.  However, almost half of these questions are actually answered by the teacher, not the students.

42. Ask Questions Better  Pose a question to the whole class, then call a student by name to respond, “Question First, Name Last.”  Make and maintain eye contact with students.  Consider a “no hands up” approach.  Use a random technique for selecting students to avoid always calling on the same students.  Limit if not eliminate unison response questions.  Provide students with “wait time” to think before responding.  Redirect student questions to other students to answer whenever possible.  Ask all students regardless of gender, ethnicity, and/or socioeconomic status equally challenging questions.  Vary pace of questioning as necessary to ensure lesson momentum.

44. Activity 1. Each group will select a benchmark from either the Math or Science Test Item Specifications. 2. Groups will develop a Higher Order Thinking Question. 3. Each group will present. 4. Using Think-Pair-Share, groups will discuss whether it is low, moderate, or high complexity and why or why not 5. Using the whiteboards, you will vote L, M, or H, and be asked to explain why.

45. How do we ensure that teachers utilize Higher Order Thinking in the Classroom?  Plan higher order questions in your common planning sessions.  Include HOTS questions in promethean/smartboards to make sure that teachers address the questions.

46. Journal Reflection How does the use of HOTs questions, DOK, and Questioning Strategies affect teacher performance and student achievement?