1. ASC 607 Seminar in Program Evaluation

2. Sometimes, you get it right.

3. Analysis Action Change The big three… Evaluations analyze evidence about a program’s effectiveness by examining strengths and weaknesses. Using the weaknesses as a framework to guide future work, evaluations develop concrete action plans to remedy weaknesses. Action steps lead to new learnings in the organization…all change is learning.

4. Session Outcomes

5. Frame every decision with evidence. 1

6. Self grading Piagetian programs Providing formative evaluation Acceleration Microteaching Classroom behavior Comprehensive interventions for learning disabled students Teacher clarity Reciprocal teaching Feedback Teacher-student relationship Spaced vs. massed practice Metacognitive strategies Prior achievement Vocabulary programs Repeated reading Creativity programs Self-verbalization/self- questioning Professional development Not labeling students Problem-solving teaching Phonics instruction Teaching strategies Study skills Direct Instruction Cooperative vs. individualistic learning Comprehension programs Tactile stimulation programs Mastery learning What do these things have in common?

7. Develop or adopt 2 a process for monitoring and evaluating decisions.

8. Continuous Improvement

9. Logical Reasoning Approach

10. Bicycle Safety and Helmets 1.What would you do to make sure students wore helmets (use yellow stickies)? 2.What would you need to do the things you listed in question one (use green stickies)? 3.What would happen as a result of each thing you did (use blue stickies)? 4.How would you know it happened and how might you capture information along the way (use orange stickies)? 5.What would be your standard for success that would allow you to say that the results were important (use white stickies)?

11. Why would you spend time with a group of stakeholders from your school answering these questions? How would the answers to these questions help you know if you got it (i.e., helmet safety) right? How would they help you know if you got any decision right? Reflect and Share

13. Know what the problem 3 looks and sounds like before you attempt to solve it.

14. Evaluations and Problem Identification What was the ideal state for the helmet safety situation? What was the current reality? Why was action needed? Note: The answer to this question is the problem and we identify the problem once we have analyzed why a gap exists between the ideal state and the current reality. In education, this gap often exists because of issues with cultural, learning, or economic systems.

15. Do gaps exist at your site? What would be the ideal state of this issue in my school? What is the current reality of this issue in my school? What is action needed? What needs to happen to close the gap between the ideal state and the current reality?

16. Think and Reflect What are your concerns about evaluation and continuous improvement? How would you convince your staff that evaluation is important?

17. Facilitate a process 4 that clearly articulates the way the program works and its intended outcomes.

18. Keeping the Logic Straight

19. Case Study Logic Model Read the case study and discuss two questions. – Discuss the pros and cons of each the faculty members’ proposals to reach chronically failing students. – Which proposal might be most successful in reaching chronically failing students? Use flip charts to complete the steps.

20. Nine Step Planning Process Step 1: Determine the ideal state. For example, if we got this right, what would it look and sound like? Step 2: Describe the current reality. What does the situation look and sound like now. Step 3: Assess the gap. Why does a gap exist between the current reality and ideal state? Step 4: Determine intended outcomes. What long term changes would we expect to see and hear? (These are changes in the school environment.) Step 4a: What intermediate changes would we expect? How would teachers’/students’ actions look and sound? Step 4b: What initial changes would we expect to see and hear? How would teachers’/students’ attitudes and knowledge change?

21. Step 5: Indicators. How will we monitor these changes and provide feedback? Step 6: Success Criteria. What will constitute success? Step 7: Identify high leverage actions. What must we do to create the change? Step 8: Outputs. What tangible evidence will our actions produce that is countable? Step 9: Resources. What do we need to act? Nine Step Planning Process

22. Field Work