Presentation is loading. Please wait.

Presentation is loading. Please wait.

Approaches to Public Policymaking, Policy Analysis & Evaluation Research Kathy Luckett University of Cape Town.

Similar presentations

Presentation on theme: "Approaches to Public Policymaking, Policy Analysis & Evaluation Research Kathy Luckett University of Cape Town."— Presentation transcript:

1 Approaches to Public Policymaking, Policy Analysis & Evaluation Research Kathy Luckett University of Cape Town

2 Research Paradigms Not rigid paradigmatic incommensurability A map for navigating choppy waters around policy analysis and evaluation methodological debates Post-positivist – experimental, pragmatic Interpretative – constructionist, post- structuralist Critical – PAR, empowerment evaluation,CSH Critical Realism – theory-based evaluation

3 Post-positivist: Quasi-experimental Popper, Campbell & Stanley (1963, 1966), Lasswell, Rossi, Lipsey & Freeman Based on methods of the natural sciences, statistical measurement techniques Social science can contribute to improved governance or management Establish cause & effect relations bet policy/ programme objectives, inputs & interventions --------- outputs, outcomes & impact Human performance can be objectively measured tv. Efficiency & effectiveness criteria Evaluator: objective, neutral, rational Problem: to secure internal validity of evaluation results By 19702 disillusion set in, shift to quasi-experimental methods (pre- & post tests, times series, comparison group designs)

4 Critique: Quasi-experimental models Uses a model developed for closed systems for open social systems Adopts a flat ontology – reality = regularities bet observable, atomistic objects & events (ignores the non-observable) Causality = regularities bet variables within stat. sig. samples Claims about causation usually unclear and unconvincing Can only provide descriptions (for a few variables on large populations), seldom explanation

5 Post-positivist: Pragmatic (dominant model) Developed from ‘new public management’, the ‘evaluative state’ – wants practical, workable results, useful for decision-making Takes policy/ programme goals as focus of evaluation Methods: a) open-ended case study (improvement) e.g. Patton b) closed-system sets up criteria & performance indicators to measure performance & accountability of individuals & institutions e.g. programme accreditation Sets up criteria and performance indicators to measure performance & accountability by institutions and individuals – a closed system

6 Critique: Pragmatic Models Assumes stable external environment Difficult to set measurable objectives, criteria & indicators for actual performance Difficult to control variables in open soc systems - possibility of rival explanations, difficult to prove cause & effect Ignores context & stakeholder meanings, ‘black box’ evaluation – seldom diagnostic Can be prescriptive, leading to conformity

7 Interpretive: Constructionist 1970s – 80s ‘linguistic turn’, 1980s policy sociology: meaning socially constructed, human action culturally and discursively mediated – rejection of naturalism Vickers (1995) policymaking as communicative activity for institutional regulation, a process of norm- setting Neo-institutional theory emphasises cognitive and normative factors in policy adoption and implementation Guba & Lincoln (1989), 4 th generation evaluation: focus on subjective stakeholder meanings, values & interests, evaluator as facilitator, truth as agreement, evaluation useful to insiders

8 Critique: of Constructionist Models Over-socialised, emphases subjectivity at expense of structure, truth located in subjectivities of respondents Ignores systemic asymmetries of power Inability to rise above context Relativist ontology

9 Interpretive: Post-structuralist Foucault’s ‘geneaology’, Ball (1993), Gale (2000) Discourse is socially constitutive, in dialectical relation to practice – sets up systems of power/ knowledge, norms & values Policy as political artefact – as text & discourse – with unequal material & discursive effects that should be exposed Policy has a normalising & regulatory role, sets up subject positions that constrain ways of speaking & thinking Technologization of language for institutional ends How do certain discourses become dominant? What discourses are at work when those who govern, govern? How do they become institutionalised & supported legislatively, professionally & financially?

10 Critique: Post-structuralist Models Weak on method, selectivity of data, dominance of researcher as interpreter, tendency to jump from data to (preconceived) narrative Quest to successfully link the micro and macro levels of analysis difficult to achieve All of social life gets reduced to discourse,(materiality of the social world gets lost) Knowledge reduced to conditions of its production and interests of its producers (epistemological relativity)

11 Critical: Emancipatory Neo-Marxist insights, Frankfurt School (Habermas empancipatory interest) Critical policy analysis (the ‘argumentative turn’) policy discourses construct social problems & policy solutions, policymaking a form of argument to persuade & manufacture consent Challenge: how do discourses become institutionalised & reflected in institutional practices? Ulrich (1994) Critical systems heuristics: policy to be normatively acceptable to those affected by it, value clarification – diff groups of stakeholders

12 Critical: Emancipatory Developmental evaluation (Patton) PAR Empowerment evaluation (Fetterman 1996) Transformative evaluation (Mertens 2005) Development of evaluees, giving voice to the silenced, inclusion of marginalised groups affected by the results

13 Critique: Emancipatory Models Utopian: the ‘better argument’ is produced through power not rational dialogue - all communication already penetrated by power Why should the involved (the powerful) bother to take into account the views and concerns of the affected (the powerless)? Cannot work under conditions of coercion requires a fully functioning public sphere Needs to hold material conditions and structures as contexts for vlaues & interests Post-structuralists: consensus is neither possible nor desirable

14 Critical Realist: Theory-based Bhaskar (1978, 1998), Sayer (1992, 2000) Reality is stratified – empirical (experiences), actual (events) & real (non-observable structures & causal powers) Holds tog ontological realism + epistemological relativism Both agency & structure have causal powers – attend to both (analytically separate) Openess of the social world, plurality and contingency of causality Key to successful intervention = change of social practice

15 Critical Realist: Theory-based Pawson & Tilley (1997) Realist evaluation: what works, how, for whom and under what conditions? (builds in context & subjectivity) Evaluator to make programme theory explicit & to check it out with stakeholders: C + M = O tests assumptions about causal relations & change Tests goal realisation, but places in context of wider social explanation Evaluation can be cumulative – middle range theories Critique: Demanding to operationalise, time- consuming

16 Conclusion Be aware of tradition & model you’re working in - & of other possibilities Complex nature of policy analysis & evaluation justifies methodological pluralism But don’t use methods opportunistically, select according to values, purpose of evaluation, stage of the policy/programme cycle & practical constraints Think purpose (teleology), ontology, epistemology 1 st – then methodology!

Download ppt "Approaches to Public Policymaking, Policy Analysis & Evaluation Research Kathy Luckett University of Cape Town."

Similar presentations

Ads by Google