1. Facilitators of School Improvement Lisa Guzzardo Asaro Deane Spencer February 2014

2. Rigor, DOK and Monkeys Connector Activity

3. Why Rigor Matters Good teaching can make a significant difference in student achievement, equal to one effect size (a standard deviation), which is also equivalent to the effect that demographic classifications can have on achievement. Dr. Heather Hill, University of Michigan

4. Why Rigor Matters If a student is in one of the most effective (rigorous) classrooms he or she will learn in 6 months what those in an average classroom will take a year to learn. And if a student is in one of the least effective classrooms in that school, the same amount of learning take 2 years. Dr. Heather Hill, University of Michigan

5. Task Predicts Performance If between 10 and 20% of the CCSS and questions on most standardized tests are at a low level of complexity, how much instructional time should we devote to DOK Level 1 and Level 2 questions in a lesson or an assessment? Brenda Matthews, 2010

6. Task Predicts Performance If as much as 80% of the CCSS and questions on most standardized tests (and in life) require moderate to high levels of complexity, how much of instructional time should we devote to DOK Level 3 or Level 4 questions in a lesson or on an assessment? Brenda Matthews, 2010

7. Elbow Talk Think about what you do in the classroom. What is it that makes a lesson rigorous? Is it difficulty? Is it length of the assignment? What contributes to rigor in the classroom?

8. Cognitive Demand Levels of Complexity, Not Difficulty Cognitive Demand The kind and level of thinking required of students to successfully engage with and solve a task. The way in which students interact with content.

9. Webb’s Depth of Knowledge Levels of Cognitive Complexity Level 1: Recall and Reproduction Level 2: Skills & Concepts Level 3: Strategic Thinking Level 4: Extended Thinking

10. Webb’s Depth of Knowledge The Depth of Knowledge framework was developed by Norman Webb at the University of Wisconsin to help align standards with assessments. Webb’s Depth of Knowledge was used by the Council of Chief State School Officers to align the assessments and as the basis for the Common Core State Standards.

11. Webb’s Depth of Knowledge Levels of Cognitive Complexity TaskThinking Collecting data samples over several monthsRecall Organizing the data in a chartSkills/Concepts Using the chart to make and justify predictionsStrategic Thinking Developing a generalized model from this data and applying it to a new situation Extended Thinking

12. Webb’s Depth of Knowledge Levels of Cognitive Complexity The Depth of Knowledge (DOK) is NOT determined by the way verbs are used, but instead by the context in which the verb is used and the depth of thinking required.

13. Webb’s Depth of Knowledge Levels of Cognitive Complexity Words like explain or analyze have to be considered in context. “Explain to me where you live” is recall and recite.

14. Webb’s Depth of Knowledge It is about what follows the verb “Analyze this sentence to decide if the commas have been used correctly” Does NOT meet criteria for high cognitive processing. The student has been taught a rule for using commas and is using the rule to complete a task.

15. Webb’s Depth of Knowledge Three uses of a verb = Three DOK DOK 1 – Describe three characteristics of metamorphic rocks. DOK 2 – Describe the difference between metamorphic and igneous rocks. DOK 3 – Describe a model that you might use to represent the relationships that exist within the rock cycle.

16. The DOK is NOT determined by the verb, but the context in which the verb is used and the depth of thinking required.

17. Webb’s Depth of Knowledge DOK is about intended outcome, not difficulty DOK is a reference to the complexity of mental processing that occurs when students answer a question, perform a task, or generate products. Adding is a mental process. Knowing the rule for adding is the intended outcome that influences the DOK. Once someone learns the “rule” of how to add, 4 + 4 is DOK Level 1 Adding 4,658,985 + 9,993,274 is still DOK Level 1

18. Webb’s Depth of Knowledge DOK is about complexity The intended student learning outcome determines the DOK level. Every objective is assigned a DOK level. Classroom instruction and assessments reflect the DOK level of the objective or intended learning outcome. Extending the length of an activity alone does not necessarily create rigor!

19. The DOK is NOT determined by how difficult the task may be, but by the complexity of the processing that is associated with the task.

20. Webb’s Depth of Knowledge Key Points DOK is about what follows the verb. DOK is different from task or item difficulty. (DOK 1 + DOK 1 + DOK 1 = DOK 1) DOK is based on the cognitive complexity of the task students are being asked to do.

21. Webb’s Depth of Knowledge Levels of Cognitive Complexity Level 1: Recall and Reproduction Level 2: Skills & Concepts Level 3: Strategic Thinking Level 4: Extended Thinking

22. Webb’s Depth of Knowledge DOK 1 – Recall & Reproduction Basic tasks that require students to use simple skills or abilities to recall or locate facts from the text. Identify who, what, where, when and why. Recall facts, terms, trends, or concepts. Define a word or concept. Identify cause and effect. Identify purposes.

23. Webb’s Depth of Knowledge DOK 1 – Sample Read the sentence: Undertaking the development of a nonprofit is a major responsibility, but one that allows people to make a significant difference for a cause they truly care about. What is the meaning of the work undertaking as it is used in this sentence? a. to give upc. to try harder b. to managed. overachievement

24. Webb’s Depth of Knowledge DOK 2 – Working with Skills and Concepts Requires both the initial comprehension and subsequent processing of text or portions of the text. Describe or explain how or why. Apply learned concepts to new situations. Provide an example. Compare/contrast people, places, events, purposes, and concepts. Classify or sort items into categories. Convert information from one form to another.

25. Webb’s Depth of Knowledge DOK 2 – Sample What purpose does repeating the phrase, “and Brutus is an honorable man,” serve in Marc Anthony’s first speech? a. to raise doubt about whether Brutus is honorable b. to emphasize that Marc Anthony agrees with Brutus c. to express Marc Anthony’s grief over Caesar’s death d. to praise Caesar’s life and accomplishments

26. Webb’s Depth of Knowledge DOK 3 – Short-Term Strategic Thinking Demands higher order thinking processes, such as analysis and evaluation. Requires knowledge and skill from multiple subject-matter areas. Explain Reasoning Generate solutions in project-based settings Support ideas with evidence Recognize and explain misconceptions Solve real-world problems with predictable outcomes

27. Webb’s Depth of Knowledge DOK 3 – Sample Read “A Dream Deferred” by Langston Hughes. Write 2 paragraphs to explain how the author’s use of imagery and figurative language contributes to the central theme of the poem. Cite at least 2 examples from the poem that support your reasoning and explain how they contribute to the central message.

28. Webb’s Depth of Knowledge DOK 4 – Extended Strategic Thinking Requires student to use complex reasoning and strategic thinking over a long period of time in order to solve a problem. Conduct investigations. Gather and analyze data to support an idea or argument. Solve real-world problems with unpredictable outcomes. Apply and adapt information to real world situations. Develop a logical argument. Make predictions with evidence as support. Design a plan of implementation.

29. Webb’s Depth of Knowledge DOK 4 – Sample Gather data in order to: Define and describe the problems that acid rain could potentially cause in a specific farming community. Include a proposal of alternate solutions to the problem. Select the solution that you feel provides the best alternative and support your reasoning with evidence. Design a plan of implementation for the solution you have proposed.

30. Rigor, DOK, and Monkeys – Using DOK Independently read the article Monkey Helpers by Tamra Orr As a table, generate and record on the handout: One DOK level 1 question Three DOK Level 2 questions One DOK Level 3 question One DOK Level 4 question

31. Rigor, DOK, and Monkeys – Sharing items Designate a scribe at your table. Use the chart paper at your table to record your DOK questions. (Recreate your handout on the chart paper) Include your school name(s) Record your questions with the intended DOK noted.

32. Today’s Outcomes 32 Engage in understanding Rigor and DOK as it relates to instruction Explore the School Systems Review, Interim Self- Assessment and Answer: Now What? Explore M-Step Updates Review the NEW School Data Analysis Program Evaluation Tool Explore MI School Data, Data Director, and Macombfsi.net

33. 33 Today’s Roadmap Welcome Connector: Rigor, DOK, and Monkeys School Systems Review or Interim Self- Assessment M-Step and Principal’s Packet School Data Analysis Program Evaluation Tool MI School Data, Data Director, and Macombfsi.net Team Work

34. 34 Key Working Agreements A Facilitation Tool Respect all Points of View Be Present and Engaged Honor Time Agreements Get All Voices in the Room These breathe life into our Core Values

35. 35 Parking Lot A Facilitation Tool Rest questions that do not benefit the whole group Place questions that do not pertain to content at this time Place questions that pertain, but participants do not want to ask at this time

36. Action Required Chart Any request by you that I need to respond to must be placed on the Action Required Chart You need to PRINT your complete name, school, and email address 36

37. 37

38. 38 GATHER Get Ready Collect Data Build Profile Stages One and Two STUDY Analyze Data Set Goals Set Measurable Objectives Study Research Process Data

39. DUE DATE:

40. 40

41. AdvancED ISA MDE SSR One or the Other; NOT BOTH

42. MDE: School Systems Review (SSR) AdvancED: Interim Self Assessment (ISA) TASKS: 1.) Those schools that have had an EXTERNAL REVIEW this year, NOW WHAT? Use the ppt. and reports from AdvancED to identify action steps. 2.) All schools address the NOW WHAT? 3.) Complete either the School Systems Review, Interim Self Assessment or Self Assessment 42

43. 43 GATHER Get Ready Collect Data Build Profile Focus Stage One Gather Step 2 Collect Data Achievement Data

44. Principal Resource Packet Division of Accountability Services (DAS) Spotlight News, January 15,2015 M-Step Spring 2015 Transition Readiness Timeline M-STEP Resources

45. Division of Accountability Services (DAS) Assessment and Accountability http://www.michigan.gov/mde/0,1607,7-140-22709---,00.html http://www.michigan.gov/mde/0,1607,7-140-22709---,00.html 45 Sign up to receive

46. Stage Two Study Step 4 Analyze Data (4 data points) 46 STUDY Analyze Data Set Goals Set Measurable Objectives Study Research

47. MDE School Data Analysis: ALL SCHOOLS 47

48. MDE School Data Analysis: ALL SCHOOLS Not in ASSIST, yet! FOUR SECTIONS in this ORDER Demographic Data (questions 1-9) Process Data (questions 10-18) Achievement/Outcome Data (questions 19-23) Perception Data (questions 24-35) 48

49. MDE School Data Analysis: ALL SCHOOLS Not in ASSIST, yet! SUMMARY SECTION The intent of this summary is to synthesize the results of your data analyses and to drive the construction of your school’s improvement plan or reform/redesign plan. Questions 36-38 *Question 38 is not included, yet. 49

50. 50 MDE School Data Analysis: ALL SCHOOLS Not in ASSIST, yet!

51. 51 Stage Four Do Step 11 Evaluate Plan DO Implement Plan Monitor Plan Evaluate Plan

52. Deconstruct a STRATEGY at the ACTIVITY Level Created a Strategy Addendum 52

53. Program Evaluation Addendum 53

54. STRATEGY IMPLEMENTATION GUIDE 54

55. AdvancED and MDE RESOURCES http://advanc-ed.org/partnership/mde 55 Located Here

56. PURPOSE ONE Evaluating the Fidelity of Implementation  implemented as intended?  implemented consistently and with fidelity?  enough time and resources available and used for implementation? Did the strategy work?

57. PURPOSE TWO Evaluating Impact (Student Achievement)  increased student achievement?  unintended consequences (good and bad)?  be continued? discontinued? modified? Demographics Student Outcomes Perception School Processes

58. 58

59. 59 Program Evaluation Diagnostic: Due June 30 OCTOBER Work During FSI Deconstructed a Strategy at the Activity Level Revised the Strategy as needed Created a PE Strategy Addendum Complete Strategy/Program/Initiative Description(page 2 in ASSIST) PAGE 2 What is the name of the strategy/program/initiative being evaluated? Example: Classroom Instruction that Works: Non Negotiables Provide a brief description of the strategy/program/initiative being evaluated? Example: Robert Marzano’s 2 nd Edition elaborates on the 9 Strategies that work in a Classroom to include all effective instructional pedagological tools. What is the need being addressed by the strategy/program/initiative? Describe what data analysis surfaced a student learning need that contributed to the selection of the strategy/program/initiative. Provide the reason for selecting the strategy/program/initiative, include intended results. Describe the student learning results you expect to achieve due to implementing the strategy/program/initiative. Cite the research supporting the strategy/program/initiative along with a brief summary. Include a brief summary explaining the strategy/program/initiative.

60. 60 Program Evaluation Tool January – February Work Complete pages 5-6 on Readiness Complete pages 7-8 on Knowledge and Skills Complete pages 9-10 Opportunity

61. Program Evaluation Tool Implementation with Fidelity Was the Strategy/Program/Initiative Implemented as Intended 61 IN AN IDEAL STRATEGY/PROGRAM/INITIATIVE, all personnel involved in the program implement the strategies with fidelity according to the research, carrying out responsibilities by their proposed timelines. They use clearly defined protocols to collect and review formative implementation data to identify unintended consequences. Program leaders consider adjustments guided by implementation data while maintaining the integrity of results. FOCUS

62. ASSIST Program Evaluation Tool Section Screen Shot 62

63. ASSIST Program Evaluation Tool Response Screen Shot 63

64. ASSIST Program Evaluation Tool Evidence Screen Shot 64

65. ASSIST Program Evaluation Tool What Does the Evidence Show Screen Shot 65

66. 66 Presenter: Dr. Jennifer Parker-Moore MISchooldata.org Data Director GATHER Get Ready Collect Data Build Profile STUDY Analyze Data Set Goals Set Measurable Objectives Research Best Practice

67. Team Work Time Create an PE Strategy Addendum Complete page one- of the Program Evaluation Diagnostic in ASSIST Network with Colleagues 67