Program Evaluation Alternative Approaches and Practical Guidelines Chapter 1 Program Evaluation Alternative Approaches and Practical Guidelines

Question 1 What is Evaluation? “to determine or fix the value of: to examine and judge” Authors’ definition: “the identification, clarification, and application of defensible criteria to determine an evaluation object’s value (worth or merit) in relation to those criteria” (p. 7)

Question 2 What are the characteristics of a “program”? A set of planned systematic activities Using managed resources To achieve specified goals Related to specific needs Participants are specific individuals or groups Results in documentable outcomes and impacts Investigable costs and benefits

Question 3 What is the definition of a program? Authors’ definition: An ongoing, planned intervention that seeks to achieve some particular outcome(s), in response to some perceived educational, social, or commercial problem”. (p. 8)

Question 4 How does Evaluation differ from research? Research: primary purpose is to add to knowledge in a field, to contribute to the growth of theory (p. 9) Evaluation: primary purpose is to provide useful information to those who hold a stake in whatever is being evaluated (stakeholders), often helping them to make a judgment or decision. (p. 9)

Research vs. Evaluation Factor Research Evaluation Purpose Add to knowledge in a field, develop laws and theories Make judgments, provide information for decision making Who sets the agenda or focus? Researchers Stakeholders and evaluator jointly Generalizability of results Important to add to theory Less important, focus is on particulars of program or policy and context Intended use of results Not important An important standard Criteria to judge adequacy Internal and external validity Accuracy, utility, feasibility, propriety, evaluation accountability Preparation of those who work in area Depth in subject matter, fewer methodological tools and approaches Interdisciplinary, many methodological tools, interpersonal skills

Stakeholders Who is considered upstream, who is midstream, and who is downstream in regard to Talmage’s (1982) model of evaluation? Upstream: make decision on whether to continue, initiate, or change funding for programs Midstream: program managers (principals), program staff (teachers, counselors, nurse, etc.) Downstream: parents, students, community members

Key Terms Formative vs. Summative (p. 22) How do they differ? Formative: focus on program improvement Summative: more concerned with providing information to make decisions about program adoption, continuation, or expansion

Formative vs. Summative (Upstream, Midstream & Downstream) If you were conducting a formative evaluation, which of the “streams” do you think would be your audience? Construct support for your answer. If you were conducting a summative evaluation, which of the “streams” do you think would be your audience? Construct support for your answer.

Page 36 Question #1 Consider a program in your organization. If it were to be evaluated, what might be the purpose of the evaluation at this point in time? Consider the stage of the program and the information needs of different stakeholder groups. What might be the role of the evaluators in conducting the evaluation? (i.e. facilitator, collaborator, management consultant, social change agent, program planning, etc.) (pp. 16 - 18)

Page 35 #2 What kind of evaluation do you think is most useful – formative or summative? What kind of evaluation would be most useful to you in your work? What kind of evaluation would be most useful to your school or school board?