1. Review: Introduction Define Evaluation
How do formal/informal evaluation differ?
What are two uses of evaluation in education?
What are the pros/cons of using an external evaluator?

2. Alternative Approaches to Evaluation
Dr. Suzan Ayers
Western Michigan University
(courtesy of Dr. Mary Schutten)

3. Alternative Approaches
Stakeholders: individuals and groups who have a direct interest in, and may be affected by, evaluation; should be involved early, actively & continuously
Program: activities that are provided on a continuing basis; typically what is evaluated
There are a variety of alternative, often conflicting, views of what evaluation is and how it should be carried out

4. Why so many alternatives?
The way one views evaluation directly impacts the type of activities/methods used
Origins of alternative models stem from differences in:
Philosophical & ideological beliefs
Methodological preferences
Practical choices

5. Philosophical & Ideological Beliefs
Epistemologies (philosophies of knowing)
Objectivism (social science base of empiricism; replicate)
Subjectivism (experientially-based; tacit knowledge)
Pros/Cons of each?
Principles for assigning value (parallel obj/subj)
Utilitarian: focus on group gains (avg scores); greatest good for the greatest number
Intuitionist-pluralist: value is individually-determined
Room for both or are these dichotomous?
Philosophical purists are rare (impractical?)
Choose the methods right for THAT evaluation
Understand assumptions/limitations of different approaches

6. Methodological Preferences
Quantitative (numerical)
Qualitative (non-numerical)
Evaluation is a transdiscipline; crosses paradigms
“Law of the instrument” fallacy
With hammer/nails, all appears to need hammering
Identify what is useful in each evaluation approach, use it wisely & avoid being distracted by approaches designed to deal w/ different needs

7. Practical Considerations
Evaluators disagree whether/not intent of evaluation is to render a value judgment
Decision-makers or evaluator render judgment?
Evaluators differ in views of evaluation’s political role
Authority? Responsibility? These dictate eval style
Influence of evaluators’ prior experience
Who should conduct the evaluation and nature of expertise needed to do so
Desirability (?) of having a wide variety of evaluation approaches

8. Classification Schema for Evaluation Approaches
Conceptual approaches to evaluation, NOT techniques
Objectives-oriented: focus on goals/objectives & degree to which they are achieved
Management-oriented: identifying and meeting informational needs of decision makers
Consumer-oriented: generate information to guide product/service use by consumers
Expertise-oriented: use of professional expertise to judge quality of evaluation object
Participant-oriented: stakeholders centrally involved in process
See figure 3.1 (p. 68)

9. Objectives-oriented Approach
Purposes of some activity are specified and then evaluation focuses on the extent to which these purposes are achieved
Ralph W. Tyler popularized this approach in education (criterion ref test)
Tylerian models
Metfessel & Michael’s paradigm (enlarged vision of alternative instruments to collect evaluation data)
Provus’s Discrepancy Evaluation Model (agree on stds, det if discrepancy exists btwn perf/std, use discrepancy info to decide to improve, maintain, terminate program)
Logic models
Determine long-term outcomes & backtrack to today

10. Objectives-oriented Steps
Establish broad goals or objectives tied to mission statement
Classify the goals or objectives
Define objectives in behavioral terms
Find situations where achievement of objectives can be shown
Select/develop measurement techniques
Collect performance data
Compare data with behaviorally stated objectives

11. Objectives-oriented Pros/Cons
Strengths: simplicity, easy to understand, follow and implement; produces information relevant to the mission
Weakness: can lead to tunnel vision
Ignores outcomes not covered by objectives
Neglects the value of the objectives themselves
Neglects the context in which evaluation takes place

12. REVIEW/Qs Why are there so many alternative approaches to evaluation?
What conceptual approach to evaluation did we discuss tonight? What is its +/-?
Why do you think the NASPE STARS program has been considered this type of evaluation?
Identify your most likely evaluation object