Presentation on theme: "Dr Mark Abrahams, University of Cape Town, March 2007"— Presentation transcript:
1 Dr Mark Abrahams, University of Cape Town, March 2007 Historical and emerging theoretical foundations for Monitoring and Evaluation in South Africa: Implications for practice.Dr Mark Abrahams, University of Cape Town, March 2007
2 IntroductionMonitoring and evaluation (M&E) relatively new (Louw, 1998; Potter, 1999; Potter and Kruger, 2001; Babbie and Mouton, 2001).Historical development of programme evaluation in South AfricaThe emerging theoretical foundationsImplications for practice.
3 Analytic Framework Shadish, Cook and Leviton (1991); Shadish (2005) Social programmes and policies aimed at improving the welfare of individuals, organisations and society .Theory - connotes a body of knowledge that organizes, categorizes, describes, predicts, explains, and otherwise aids in understanding and controlling a topic
4 QuestionsWhat good is a fine evaluation of an important attempt to solve a puny problem?What good is a fine evaluation of a puny attempt to solve a serious problem?What good is a fine evaluation of a program that solves a serious problem if the results are not stored and used to ameliorate the problem?
5 Point (s) of departureEvaluation is just one part of a complex, interdependent, non-linear set of problem-solving activitiesEvaluation research is more than the application of methods, more than an isolated academic concern and it operates in the context of the social policy and public administration movements.
6 Five ‘fundamental issues’ Social programming: the ways that social programs and policies develop, improve, and change, especially in regards to social problems.Knowledge construction: the ways researchers learn about social action.Valuing: the ways values can be attached to program descriptions.
7 Cont.Knowledge use: the ways social science information is used to modify programs and policies.Evaluation practice: the tactics and strategies evaluators follow in their professional work, especially given the constraints they face.
8 Three periods in S.A. Pre - 1994 Post – 1994 1999 to date Anti-apartheid, anti-government, liberationPost – 1994Reconstruction and development1999 to dateGovernance, sustainability, accountability
9 The Drivers The Democratic state International and regional agreements The Presidential addressPower and accountabilityBenchmarkingLegislation – Performance managementInternational and regional agreementsMillennium goalsNEPADSustainable DevelopmentInternational Exchange/ globalisation
10 Drivers (cont.) Economic growth Sustainable development Unemployment Infrastructure and spatial developmentHIV/AIDS and other health issuesEquity and redressPoverty alleviationSafety and securitySkills developmentClimate change and environmental concernsCivil and ‘uncivil’ societyFree press
11 Emerging theories of social programmes Programmes are policy-connected interventions underpinned by the Constitution and Bill of RightsThey are constructed with clear objectives, timelines and action plans and resources are allocated to achieve targetsIntegrated approach emphasised, recognising the multi-faceted nature of social problems.Social engineering
12 Theories of knowledge construction Multi-methodological approach with emphasis on reliable dataPreference for quantitative dataAudit and compliance strategiesMonitoring vs evaluation
13 Theories of valuing Stakeholder dilemma Who are the ‘holders’? What is the stake ‘steak’ for each holder?What is at stake for each holder?Who holds?Who holds what?Who suffers?Who gainsWhat are the consequences of non-delivery?
14 Theories of UseCentralising structures (GWM&E) to ensure and enable use.Advocacy purposesEarly warning systemsPerformance measurementPolicy development
15 Theories of practice Objectivity through indicators Elements of the followingManipulable solution theory – reducing uncertainty about programme effectsGeneralisable explanations theory – knowledge of complex interrelationships among multiple causal determinantsStakeholder service theory - tailored to information needs of stakeholders
17 Theory of Social Programmes Anatomy of social programmes (Pawson, 2006)Intervention as theoryInterventions as activeIntervention chains thickly populatedIntervention chains non-linear can go into reverseInterventions embedded in multiple social systemsInterventions leaky (change over time)Interventions are open systems and change the conditions that make them work in the first place.
18 Theory of KnowledgeOntology, Epistemology and methodology essential topicsKnowledge of causation and generalisationAll methods are potentially fallibleThe limitations and potential of the single case study – multiple studiesParameters that influence the choice of methods
19 Theory of Valuing Prescriptive and descriptive valuing Agreed upon principlesStandards of performance and measuring performanceSynthesizing results?Stakeholder interest vs criteria of merit
20 Theory of Use The challenges of instrumental use Promoting use Facilitating useInfluencing useConceptual use of findingsCommunicating resultsAgreement about useMultiple stakeholders
23 Historical overviewIn 1999, Potter reported that since the 1970s, an estimated R6 billion of overseas and local funding had been used by various NGOs to engage in development projects in various sectors of society, and only a small fraction of these programmes had been evaluated.
24 History cont.Potter and Kruger (2001) illustrate with a 1998 PsychLIT database (Silverplatter International N.V.) search for references from 1974 to 1997 that produced only “fifteen articles and one book chapter that were indexed under ‘programme evaluation’ and ‘Africa’, out of a total of articles and books that were indexed under ‘programme evaluation’ (p. 192).” Eleven of the articles, including the book chapter were from South Africa, representing less than half a percent of the total available on that database.
25 History cont.Louw’s (1998) ‘informal’ survey of programme evaluation and the types of evaluations produced in South Africa revealed that qualitative approaches to programme evaluation was the dominant one and experimental, including quasi-experimental designs were present but not in great numbers. He found that most of the work involved formative rather than summative evaluation.
26 Historical Elements of P.E. Pre 1994Project DrivenDonor requirementsMostly formativePersonality drivenVague objectivesLiberation rhetoricDwindling resourcesPost 1994Programme focusedPlanning requirementsClear objectivesLogical frameworksPolicy frameworksAccountability principles – International agencies
27 Post 1999 Policy - programme relationship Integration at conceptual levelAccountability structures and frameworksNeed for sophisticated information management systemsManaging development implementationLarge scale implementationInternational treaties and agreementsSustainable development
28 Historical theoretical foundations Theory of social programmesFocused projects can bring about desired changeDesired change can be brought about within specified timeParticipants involved and eager for the same reasonsClear lines of contestation/ conflictLocal knows best, as long as people are involved
29 Historical theoretical foundations Theory of useInstrumental intent