1. Using Evaluation Theory to Improve Priority Setting for Healthcare
Neale Smith & Craig Mitton
Centre for Clinical Epidemiology & Evaluation, Vancouver Coastal Health Research Institute, University of British Columbia
Presentation to the 8th Biennial Conference of the International Society on Priorities in Health Care
Waltham, MA April 23, 2010 1

2. Evaluation and Priority Setting: Common Roots
Increased expectations for government accountability
Evidence-based decision making
Increased demand for efficient use of resources in times of fiscal restraint

3. Objective: From this ...
Priority Setting & Resource Allocation Literature
Evaluation & Evaluation Theory Literature

4. ... to this Priority Setting & Resource Allocation Literature
Evaluation & Evaluation Theory Literature

5. An Evaluation Theory Tree
SOURCE:
Alkin, M.C., & Christie, C.A. (2004). An evaluation theory tree. In M.C. Alkin (Ed.), Evaluation roots: tracing theorists’ views and influences. Thousand Oaks, CA: Sage. Pages

6. Primary Perspective Methods Evaluator Use Decision Maker Valuing
Larger Public or Society

7. Guiding Evaluation Question
Methods
How to measure the impact of PSRA processes on desired outcomes?
Use
What do decision makers in a particular context need in order to improve the usefulness to them of PSRA?
Valuing
How do the results of PSRA processes affect the interests of different social groups (especially the least advantaged)?

8. Common Methods Methods Controlled experiments if possible
Quantitative measures if possible
Focus on measures of PSRA outcomes
Use
Interviews, focus groups with decision makers
Action research
Other qualitative methodologies, e.g., phenomenology
Valuing
Constructivist, critical, participatory action research approaches

9. Evaluation Focus Methods Intended outcomes and objectives
Program theories
External or independent observer
Use
PSRA processes
Intended use
Organizational capacities
Valuing
Unintended outcomes
Power relationships

10. Implications
More evaluation of PSRA is needed – and we should expand its focus to consider outcomes, as well as process and outputs
Evaluation should be considered from the beginning – this can influence process design
Existing PSRA studies fail to draw upon evaluation theory: we can take advantage of such approaches to more clearly identify features we wish to address, and how best to do so

11. Contact information