1. Preparation vs. Production Program personnel initiate the evaluation Central admin or funding require the evaluation Internal team suggest an evaluation Be familiar with the nature of the program Be familiar with the people served Be familiar with the goals and structure of the program Why an evaluation is being considered Meet with stakeholders Decide if useful Consider the timing, the manner, and the cost

2. Overview of Evaluation Models Rule: First ask the questions, then choose the appropriate model The range of questions being addressed by the evaluation or program setting often indicate the best model Use approaches from several models to best meet the goals of the evaluation (back to the questions)

3. The Traditional Model Overview of Evaluation Models

4. The Social Science Research Model Overview of Evaluation Models

5. The Industrial Inspection Model Overview of Evaluation Models

6. The Black Box Evaluation Overview of Evaluation Models

7. The Objectives-Based Evaluation Overview of Evaluation Models

8. The Goal-Free Evaluation Overview of Evaluation Models

9. The Fiscal Evaluation Overview of Evaluation Models

10. The Accountability Model Overview of Evaluation Models

11. The Expert Opinion Model Overview of Evaluation Models

12. The Naturalistic Model Overview of Evaluation Models

13. The Empowerment Evaluation Overview of Evaluation Models

14. The Theory-Driven Evaluation Overview of Evaluation Models

15. An Improvement-Focused Model Overview of Evaluation Models

16. Steps in Preparing to Conduct an Evaluation

17. If you were to conduct an evaluation of the Teacher Performance Appraisal….

18. Figure 6. The axial coding model of the use of the Ontario teacher performance appraisal and its contribution towards reflection and refinement of professional practice. (Barnett, 2006, p.181) Barnett, J. (2006, November). Considering Growth in Teaching: Teacher Perspectives on the Ontario Teacher Performance Appraisal, Ontario Institute for Studies in Education of the University of Toronto, 2006.

19. Causal Conditions (Authentic) “the schema they come in with, the background they come in with is going to dictate their interpretation.” Section 20 teacher (Barnett, 2006) “is very much like going to high school or even university where you have your profs. You write an essay for one they give you an A. If you wrote the same type of essay for another they give you a C, because it is not what they expect.” Second Year teacher (Barnett, 2006) “Do you judge them by what you see going on in the classroom in 20 minutes or do you judge him because he’s gone to the Sears’ Festival four years in a row and won?” Experience Secondary School Teacher (Barnett, 2006)

20. Central Phenomenon Barnett, J. (2006, November). Considering Growth in Teaching: Teacher Perspectives on the Ontario Teacher Performance Appraisal, Ontario Institute for Studies in Education of the University of Toronto, 2006.

21. Content and Intervening Conditions Past Experience Barnett, J. (2006, November). Considering Growth in Teaching: Teacher Perspectives on the Ontario Teacher Performance Appraisal, Ontario Institute for Studies in Education of the University of Toronto, 2006.

22. Strategies – Perceived Value Barnett, J. (2006, November). Considering Growth in Teaching: Teacher Perspectives on the Ontario Teacher Performance Appraisal, Ontario Institute for Studies in Education of the University of Toronto, 2006.

23. Result just jumping through the hoop. A big hoop. A big hoop. It was a lot of work. I don’t think I grew from it because I don’t think I had any sort of constructive feedback….I feel like it was sort of like a Seinfeld episode – a show about nothing. 6 th year junior teacher I really do think it has the potential to be a helpful tool….we could improve by trying some of their ideas or something like that. I just think maybe there are better ways of going about it and I don’t think this is necessarily, this government imposed option is perhaps not the best choice. And end of story right there. 4 th year intermediate teacher it wasn’t a judgemental process. It was more like a discussion. What are you interested in doing?...Are you happy with what you’re teaching? Is there anything we can do to help you? Do you have any concerns? 35 th year French teacher Barnett, J. (2006, November). Considering Growth in Teaching: Teacher Perspectives on the Ontario Teacher Performance Appraisal, Ontario Institute for Studies in Education of the University of Toronto, 2006.

24. Criteria and Standards Criteria  Reflect program intent  Staff can influence  Can be measured reliably  Stakeholders participate in selecting Goals  How much agreement on goals is needed  Different types of goals - implementation - intermediate - outcome * people are treated with respect, no discrimination, no negative side effects

25. Program Theory Is the program accepted? The way we think about the organization in which we work influences what we consider to be possible within it and the actions we take…. In schools, novel approaches to leadership in general, and supervision in particular, can fly in the face of tradition and existing structure, and beget substantial resistance: “that is not the way we do things here.” However, such novel approaches can also lead to new ways of thinking about schooling and the professional growth of teachers. (Mackinnon and Pynch-Worthylake, 2001, p. 6)

26. Is the program accepted? Risk-taking is fundamental to teacher learning (Beerens, 2000). Teachers need to feel there is a freedom in order to move into action (Zepeda & Ponticell, 1998).

27. The value of resisters, on the other hand, has been unappreciated. Resistant acts “often embody a form of good sense” (Gitlin and Margonis 1995). Finding a way to reconcile positive and negative emotion is the key to releasing energy for change. (Fullan, 1997, p. 223)

28. Cunningham and Gresso (1993) contend: There are a number of personal and organizational philosophies that work against employee innovation. One is the natural fear of newness…. There is a natural fear of trying something new even when there is ample evidence it works. This is part of the fear of making changes or even taking small risks. One becomes enamoured with security without realizing how insecure he or she is. (p. 151)

29. Schmoker (1999) warns “goal-setting that uses data to monitor progress can be a threatening endeavour” (p. 40).

30. Glickman (2003): If I push too hard, alienate, and harden other people’s position, am I then part of the solution? If I clarify, help others understand my feelings – but then back off – am I then still part of the solution? If I take a walk because I’m opposed to the decision being made, am I now part of the problem? These issues of value stay with me long after meetings and workdays are over. (p. 339)