1. Overview of Chapters 11 – 13, & 17

2. Clarifying the evaluation request and responsibilities
Chapter 11
Clarifying the evaluation request and responsibilities
An organization has requested that you to complete an evaluation. What questions do you feel are important to ask before beginning the evaluation?
Are there times when you would decline a request for evaluation? Why?
What are some of the advantages and disadvantages in having an evaluation conduct by an external evaluator? By an internal evaluator?
(take 5 minutes with a partner to discuss)

3. Clarifying the Evaluation Request and Responsibilities
Identify the stakeholders
Key questions:
Who are the sponsors and clients?
Who are the managers and staff?
Are there other interest groups, such as agencies, elected officials, the public at large?
2. Identify the purpose for the evaluation
Why is the evaluation being requested?
What question will it answer?
How will the results be used? By whom?

4. Clarifying the Evaluation Request and Responsibilities
3. Determine if the Evaluation is Appropriate
Key questions:
How will the results be used?
Will the results produce trivial information?
Is the program too new?
Is the evaluation purpose ethical?

5. Clarifying the Evaluation Request and Responsibilities
Steps to follow:
Clarify the intended program model or theory
Examine the program in implementation to determine whether it matches the model and could achieve the goals or objectives
Explore different approaches to determine if they meet the stakeholders information needs
Agree on evaluation priorities and intended uses of the study

6. Who will evaluate the program?
Internal vs. External evaluator Internal: know the organization, history, and decision-making style and would be around to encourage use of evaluation results External: bring a greater objectivity and sometimes specialized skill for a particular project or evaluation

7. Chapter 12
Setting Boundaries and Analyzing the Evaluation Context 1. After determining who the stakeholders are, how do you determine who should be involved in the evaluation process? 2. What should the evaluator consider in analyzing the political context in which an evaluation will occur? (take 5 minutes to discuss with a partner)

8. Setting Boundaries and Analyzing the Evaluation Context
The evaluator should identify and communicate with each stakeholder group or its representative: 1. To learn about the groups concerns 2. To understand how they will use the evaluation results This will help the evaluator weigh their input during the evaluation

9. Setting Boundaries and Analyzing the Evaluation Context
Do evaluators need to be aware of Squeaking wheels &Powerful Stakeholders? You need to bring about a “democratic dialogue” if possible, with a balance in the number of people involved and diversity within the group.

10. Setting Boundaries and Analyzing the Evaluation Context
Greene (2005) identified four groups:
People who have decision authority over the program
People who have direct responsibility for the program
People who are the intended beneficiaries of the program

11. Setting Boundaries and Analyzing the Evaluation Context
Scriven (2007) adds to the list: Political supporters or opponents Community Leaders Public in General

12. Setting Boundaries and Analyzing the Evaluation Context
Political Context: 1. Who will gain to gain/lose? 2. Which individuals and groups have power in the setting? 3. How should the evaluator relate/communicate to the groups? 4. Will all groups cooperate? 5. Which groups have a vested interest in the outcomes? 6. What safeguards should be incorporated into the evaluation agreement?

13. Chapter 13
Identifying and selecting the evaluation questions and criteria
What is the focus of evaluation questions?
What are good sources for evaluation questions?
Who should be involved in the divergent and convergent phases? Why?

14. Identifying and selecting the evaluation questions and criteria
Evaluation Questions: Provide focus to the evaluation Specify the information the evaluation will provide Guide choices for data collection, analysis, and interpretation

15. Identifying and selecting the evaluation questions and criteria
Good Sources: Existing Standards in the Field Research Literature Content Experts Evaluators Own Experience

16. Identifying and selecting the evaluation questions and criteria
Divergent: Cronbach (1982), “Opening one’s mind to be entertained at least briefly as prospects for investigation” (p. 210). Fitzpatrick, et al. (2011), “throw a broad net and learn from many possible sources” (p. 316).

17. Identifying and selecting the evaluation questions and criteria
Convergent: Three reasons for reducing the range of variables.
Budget
Manageability/Complexity
Attention Span of Audience
The evaluator aims for the maximum bandwith.
Cronbach (1982)
*Typically completed by an advisory group with representatives from different stakeholder groups.

18. Chapter 17: Reporting Evaluation Results: Maximizing Use and Understanding
What considerations are important in tailoring the reporting of evaluation results to audiences?
What are some of the ways results can be communicated to stakeholders?
What should be included in an evaluation report?
(take 5 minutes to discuss with a partner)

19. Chapter 17: Reporting Evaluation Results: Maximizing Use and Understanding
Different audiences have different information needs and knowledge levels. Reading ability Familiarity with the program Attitude toward the program Role in decision making Familiarity with evaluation practices/methods Attitude and interest in evaluation Experience in using evaluation results Torres, Preskill, and Pointek (2005)

20. Chapter 17: Reporting Evaluation Results: Maximizing Use and Understanding
If possible (Patton, 2008) points out that evaluation data are sued more if the evaluator discusses and negotiates the format, style, and organization of reports with primary users. Other Key Points: Avoid Jargon, Use simple direct language, Use examples, anecdotes, be interesting…

21. Chapter 17: Reporting Evaluation Results: Maximizing Use and Understanding
Key Elements of an Evaluation Report (p. 470)
Executive Summary
Introduction to the Report
Focus of the Evaluation
Brief Overview of Evaluation Plan & Procedures
Presentation of Results
Conclusion and Recommendations
Minority Reports or Rejoinders
Appendices