1. Process Use: Intentional Practice or Just Good Practice? anzea 2013 Conference 22–24 July 2013 Alexandra Park, Epsom, Auckland Michael Blewden Massey University

2. Overview Background Research question/approach Case study of findings Implications for practice

3. Process Use Learning and development from stakeholder participation in evaluation Influence/consequence of evaluation processes Distinct from/independent of findings use

4. e.g. Stakeholder participation… …enhances willingness to use findings …develops evaluative thinking or action …develops shared understanding …influences the evaluand

5. Patton (1997) says: “Evidence of process use is represented by the following kind of statement after an evaluation: “The impact on our program came not just from the findings but also from going through the thinking process that the evaluation required”

6. Shaping this study Process use: - enhances value and utility - often an unintentional side-product - more likely if we purposefully seek it

7. Patton (1997) again “...the possibility and desirability of learning from evaluation processes as well as findings can be made intentional and purposeful…” “…instead of treating process use as an informal offshoot, explicit and up-front attention to the potential impacts of evaluation logic and processes can increase those impacts and make them a planned purpose for undertaking the evaluation…”

8. The question of ‘intent’ Patton infers evaluators may choose to deliberately seek process use adopt specific practices to achieve it increase the value and utility of evaluation

9. The question of ‘intent’ …historically, process use more typically regarded as “...an informal offshoot” (Patton, 2007) …few methodologies intentionally seek process use…rarely an integrated goal of practice (Morabito, 2002) Observations of NZ practice and context

10. The (research) question Why and for what purpose do evaluators seek process use? Why do they choose the practices they do to achieve it?

11. Research Approach Process use as ‘sensitising concept’ Process use as a ‘construction’ Interpretivist explanation Importance of context

12. Interpretivist explanation Meaning + beliefs + desires = behaviour e.g. action X was done because person held belief Y according to which doing X would fulfil desire Z

13. Process use intent and practice Traditions Values Beliefs explaining beliefs Practice setting Beliefs about evaluation Beliefs about role Evaluation theory Evaluation practice Project setting Beliefs about outcomes Journey to practice Cultural context Beliefs about practice

14. Intent and practice is ‘understandable’ Meaning Why important Justifications, reasons Point and purpose Expectancies Embedded ‘rules’

15. General awareness and experience of process use Process use examples considered important and intentional

16. Participants 24 practicing evaluators Eligibility criteria In-depth face to face interviews Auckland and Wellington location

17. Assumptions Desirable for evaluators to seek process use but not necessarily always Pursuing process use may have risks Understanding, use, relevance or appropriateness of the term not assumed

18. Evaluation as process Evaluation as development Evaluation as findings use Evaluation as capacity building

19. Intent and practice Beliefs about evaluation Evaluation as intervention Social betterment Enabling Equality Knowledge is experiential and constructed

20. Intent and practice Beliefs about practice Collaborative, transparent, understandable, trustful Accountable to relational ethics and morals Should address issues of power and inclusion Tools and procedures as learning

21. Intent and practice Beliefs about role Should facilitate mutual learning, development, improvement Responsibility to give back/return value Should act in the interests of those with less power Intent regarding process impacts

22. Intent and practice Beliefs about outcomes Data quality Accept evaluative conclusions/findings Capacity development and learning outcomes Critical engagement Attitudinal/affective change Equality

23. Explanations Findings are ‘ideographic’ - however…. PU integral and inevitable Intent/practice understandable when evaluators are understood as thinking, ‘meaning makers’ Enhancing process use about debating the evaluator’s mandate, role, responsibility

24. Reflections Do these evaluator beliefs have implications – positive or negative? Are there risks to evaluation? Could there be process use misuse? How should we respond?