Presentation on theme: "RESULTS BASED MANAGEMENT"— Presentation transcript:
1 RESULTS BASED MANAGEMENT UNEP Brown Bag LunchSeptember 2006Kabell Konsulting ApSManaging for results
2 Agenda Presentation Dialogue What is RBM? The Evolution of RBM Key ConceptsResults ReportingBehavioural IssuesDialogueRBM in UNEPFeed back
3 Introduction Objectives To provide input to UNEP’s discussions on RBM To share some international lessons with using RBMTo discuss where strengthening is needed in UNEP
4 What is RBM? Definition: “RBM is a management strategy focusing on performance and achievement of outputs, outcomes and impacts” (DAC OECD 2002)Facilitates the systematic thinking about three basic questions:What is our goal: “Are we doing the right thing”?How will we reach that goal: “Are we doing it right?”How do we know whether we have achieved our goal and that we are doing it right: “How do we know?”Men Hvad er Results Based Management mere præcist. Den præcise definition ifølge DAC kan i jo selv læse. Overordnet set, så angiver Results-Based Management en metode til systematisk at vurdere:Gør vi det rigtigeGør vi det på den rigtige mådeOg hvordan ved vi om vi gør det rigtige på den rigtige måde (M&E systemerne)Synonymous with Performance Management or Outcome Based Management -
5 What is RBM?RBM is not:“A management strategy focusing on inputs and activities”It may include, but should not be limited to, thefollowing elements:Performance MeasurementPerformance ReportingPerformance Based BudgetingVigtigt at gøre sig klart at Results-Based Management som helhed er en omfattende tilgang til management som indeholder mere end måling og rapportering af resultater. Disse elementer er nødvendig elementer, men det er en stor fejltagelse at tro at implementeringen af disse elementer isoleret set er udtryk for RBM.
6 The Evolution of RBM In the 1980s: compliance and control. In the 1990s: project based, introduction of log frames, evaluations focus on input and results and is made for accountability.After 2000: more systemic approach with results frameworks, focus on programmes not projects and on outcomes and impact, evaluation for learning, often jointly with partners
7 Focus on performance not compliance The Evolution of RBMFocus on performance not compliance“Did projects spendtheir budget andcomply with rulesand procedures?”“Did projects achieve their objectives and deliver results?”Focus on outcomes and impact not input, activities (and output)“How many treeswere planted andpeople trained?”Overordnet set kan man sige at der gennem de sidste 20 år er sket et paradigmeskift i den offentlige sektor fra et fokus på compliance til resultater og fra fokus på input og outputs til outcomes og impact.Realiteten er dog ofte den at mange organisationer i dag fortsat fokusere på deres output. Det kan vi vende tilbage til senere.“What was the developmentresult e.g. reduction in povertyand improved living conditions”
8 The Evolution of RBM Focus on management, not administration Client-centered and citizen focusedEmphasis upon Outputs versus Inputs; reduction in ex-ante controlsExtensive use of outsourcing, competition and private service providersNext Steps Program in the United KingdomReinventing Government in the United StatesAustralian and New Zealand PEM ReformsAlternative Service Delivery in Canada
9 The Evolution of RBM RBM has gained considerable following because: It improves transparencyIt provides the basis for stronger external and internal accountabilityIt rewards achievement of resultsIt encourages evidence based policymaking and management…and as a result improves the quality of policy and decision makingRBM forventes at bidrage til mere gennemsigtighed .Bestræbelserne på at kunne træffe beslutninger på baggrund af dokumenterede erfaringer kender vi især for uddannelses sektoren. Afhængigt af hvor man stiller sig i den debat kan man til tider stille spørgsmål til validiteten af den erfaringsopsamling der foretages og heraf til de beslutninger der træffes på baggrund af erfaringsopsamling.
10 RBM in Development Cooperation 2000 MDGs2002 Monterrey2003 Rome2004 Marrakesh2005 Paris HLF-2Mutual responsibilityCoherencePartnershipHarmonizationAlignmentResultsfor shared resultsPrecedent sector publique en general, domaine du developpement nous retrouvos ces memes pre-ocupations et principes, mais avec un element de plus - le partenariat2000 – meilleur coherence a travers les MDG – resultats et indicateurs2002 partenariat - Bonne gouvernance dans les pays en vois de developpement, ressources et efficacite dans la gestion de l’aide – transaction costs - aussi First International Roundtable on Better Measuring, Monitoring, and Managing for Development ResultsRome - harmonization - action plans - nouvelle structure mises en place: “l´architecture globale”Marrakesh – manaaging for results –Paris 2005 The second High-Level Forum on Harmonization and Alignment for Aid Effectiveness
11 LessonsIndicators and evaluations should go beyond compliance and accountabilityM&E should contribute to build bridge between individual projects and sectorwide or global programmesIndicators should measure contribution to MDGs and intermediary outcomesEvaluation of the performance of partners should be based on results achieved towards joint objectives
12 Glossary of Key Terms in Evaluation & Results Based Management Key ConceptsGlossary of Key Terms in Evaluation & Results Based Management
13 Key Concepts The Results Chain Impact Outcome Assumptions and Risks IndicatorsOutputsConsider limitations of the Logframe approachAssumes that necessary resources will be providedThe assumed cause-effect relations may not evolve as expectedAbsorbing inputs from other actorsStaticCapturing unintended consequencesMed henblik på at undgå misforståelser i forhold til brugen f begreberne i the results chain har OECD DAC i 2002 lavet en lille ordbog med definitioner på nøgle begreberne. Med udgangspunkt i disse definitioner vil jeg kort gennemgå begreberne i the results chain.ActivitiesInputs
14 Key Concepts Input Activities Output The financial, human and material resources for the development interventionActivitiesThe coordination, technical assistance or training tasks deliveredOutputThe products, capital goods and services which results from a development intervention; may also include changes resulting from the intervention which are relevant to the achievement of outcomes
15 Key Concepts Outcome (- a consequence of outputs) Impact The likely or short term and medium term effect of an intervention’s outputsImpactPositive and negative, primary and secondary long term effects produced by a development intervention, directly or indirectly, intended or unintendedDAC and UNFPA measurable change in behaviour, attitudes or socio-cultural values of groups;measurable change in legal, institutional and society practices
16 Key ConceptsResultThe output, outcome or impact (intended or unintended, positive and/or negative) of a development interventionResults may:Appear within a short time or take years to be fully realizedBe planned or unforseen;Be either positive or negative;Be reflected at the level of individuals, groups, institutions or societyA result is a describable or measurable change resulting from a cause and effect relationship. It is a change that can be observed, described and measured in some way, for which the cause can be identified
17 Key Concepts Indicator Quality-test of indicators: A quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the the changes connected to an intervention or to help asses the performance of a development actorQuality-test of indicators:S SpecificM MeasurableA AvailableR RelevantT TrackableImparative to include stakeholders in this proces: those that are going to collect the data and those that are going to use the data
18 Key Concepts Identify indicators for output, outcome and process: Specify exactly what is to be measuredUse a mix of qualitative and quantiative indicatorsUse project rating systems, where aggregation is neededConsider proxy indicators where direct indicators are not availableMake sure to have baselines and target valueRemember:”An imprecise indicator about an important result is better than a precise indicator of an insignificant result”Provide example of project rating systemQualitative indicators:Peoples judgement and perceptions about a subject e.g. such as the confidence people have in sewingmachines as instruments of financial independenceQuantitative indicators:Objective or idenpendently verifiable numbers or rations. Measures quantity, such as the number of peoplewho own sewing machines in a villagePerhaps there is too much focus on indicatorsMeasuring an indicator only provides an indication of development not evidenceDanger of goal displacement – managing for indicatorsAn indicator is normatively definedLimitations - Not everything is measurableDoes not capture unintended effects
19 Example Indicators in UNDP Strategic Framework Do they say something significant?Mix of qualitative or quantitative?Can they be aggregated?Are there baselines or target values?Use of process indicators?Can you actually find data to support them?AlsoNo change theory: How we move from input to impactNo defined outcome
20 Elements of an RBM System A Results Based Management SystemSituation and problem analysisFormulating objectives, intended results and strategiesIdentifying indicatorsSetting baseline and targetsCollect data on performanceAnalyse and report performanceIntegrating monitoring and evaluation findingsUsing performance information for future planning and decision makingStrategic PlanningPerformance MeasurementPerformance ManagementPerformance measurement = Monitoring
21 Example of results measurement system RBM in UNDP5 Goals30 services lines6 cross-cutting driversReporting on:rate of achievement of annual targets towards expected outcomesextent to which UNDP promote drivers of development effectivenessDifficulties identified in review by Dalberg Do not demonstrate the actual achievements targeted.Indicators are not currently standardized across offices, and thus there is variability in the measurability and ability to attribute output and outcomes.The distinction between output and outcomes is in some cases not clearGoalsAchieving the MDGs and reducing human poverty;Fostering democratic governance;Managing energy and environment for sustainable developmentSupporting crisis prevention and recovery; andResponding to HIV/AIDS Service lines for this strategic goal are: 1)Frameworks and strategies for sustainable development 2) Effective water governance 3) Access to sustainable energy services 4) Sustainable land management to combat desertification and land degradation 5) Conservation and sustainable use of biodiversity 6) National/sectoral policy and planning to controlemissions of ozone-depleting substances and persistent organic pollutants.The six drivers are: (i) developing national capacities; (ii) enhancing national ownership; (iii) advocating and fostering an enabling policy environment; (iv) seeking South-South solutions; (v) promoting gender equality; and (vi) forging partnerships for results. (UNDP/DP/2005/16, 17)
22 Results ReportingHaving determined expected results, identified indicators, and set baselines and targets as described, it is time to:Collect data on performanceAnalyse performanceReport on performance – for external and internal useThree important issues to consider:Purpose of results reportingAggregationAttribution/contributionPurpose:Internal vs External or both?Budgetting purposesAccountability or learning
23 Difficult to aggregate due to diversity in outcomes Results ReportingAggregating results: output or outcome?Project/programme outcomeEasier to establish link between outputs and project outcome than national development changesMore relevant for internal learning purposesbutDifficult to aggregate due to diversity in outcomesOne possibility is to establish standard outcomes for common program approaches (groupings of similar projects)Establish rating system measuring succes in meeting outcomes (Consider issue of validity/reliability)Establish a solid validation process
24 Results Reporting Attribution vs. Contribution The move from projects and outputs to programmes and outcomes/higher order goals makes it difficult to attribute specific outcomes to individual donorsIn Denmark, even National Audit Office has accepted this in terms of performance measurementHowever, in-depth evaluations are still sometimes carried out with the specific objective of demonstrating attribution (counterfactual methodology)
25 Results Reporting Lessons learned in regards to results reporting Qualitative assessments / statements are more interesting for internal learning than quantitative assessmentsRatings provide a good indication of status in projects but should be supplemented with qualitative assessment and validationSupplement performance measurement (monitoring) with evaluationsEstablish a solid validation processExamples of performance monitoring systems at agency level:Ratings: Self-assessments at project/programme level. Needs to be assisted by some sort of validation process e.g. review of assessments and field visits (field-based auditing)
26 Behavioural IssuesDifferent approaches to RBM are likely to effect equally different on behaviour of staffManaging By ResultsPerformance measurement is used for accountability/reporting to internal and external stakeholdersSanctions if targets not met (creates push for staff to focus on outputs)Fixed targetsRessource allocationManaging For ResultsPerformance measurement is used with the aim of achieving better resultsStretch targets and assessments for learning to modify targetsConsider both external and internal demands and needs for accountability and learning when establishing a performance measurement system
27 Behavioural Issues Experiences from implementing RBM ”What gets measured gets done””An imprecise indicator of a significant result is better than a precise indicator of an insignificant result””You become what you measure”Organizations typically focus more attention on the methodological and technical aspects of results-oriented M&E systems than on changing mental models (World Bank 2004)It is pivotal to create an organisation culture conducive to learning and innovation and establishing incentives that promote managing for results (World Bank 2004)Strong emphasis on formal systems of tight specification and measurement. Failure to recognizing the fact that for complex activities (..) a highly formalised approach to management has severe limitations (OECD 2002).The shift to a results-based approach has been largely characterized by a focus on the budgeting and programming aspects of management without realizing or emphasizing at the outset the scope of changes required in other areas of management for an effective implementation of an RBM system (JIU 2004)
28 RBM in UNEP? Conducive Conditions Problematic Conditions Enablers and constraints for introducing RBM:Conducive ConditionsAn organisation has products and products that are simpleProducts are uniform, isolated and the production is autonomousAn organisation is product orientedCausalities are knownQuality definable in performance indicatorsEnvironment is stableProblematic ConditionsAn organisation has obligations and is highly value-orientedProducts are multiple and generated together with othersAn organisation is process orientedProducts are interwoven and causalities are unknownQuality not definable in performance indicatorsEnvironment is dynamicFollow up on exercise
29 Self assessment in UNEP Conditions?Experiences?Incentives?Capacity?Plans?
Your consent to our cookies if you continue to use this website.