2An orientation to CIDA’s approaches to programming with its development partners
3Many inputs – but what are the results? Sustainable HumandevelopmentMany inputs – but what are the results?Why do we talk about RBM?
4Brief History of RBMGenesis of the Logical Framework (LFA, Zopp) – 1968RBM - formally introduced in USAID and DFID 1990CIDA introduces RBM in1994Focus on Results – Millennium Development GoalsGlobal focus on RBM at a string of meetings starting in 2002 (Monterrey, Rome, Marrakech etc.)Paris Declaration 2005 – strengthening aid effectiveness:Harmonizing practicesAlignmentChanging “ownership” of aid activitiesMutual accountabilityAccra 2008 – reinforces the focus on development resultsnew UNDP focus on “managing for development results”
7Overall ObjectivesTo provide partners with an opportunity to deepen their own and CIDA’s understanding of respective project proposals which it will be partnering organizations on andTo provide CIDA with an opportunity to orientate partners on the approaches which CIDA takes to its development programming with its partners.To work with partners to generate the documentation required to process contribution agreements.
8CIDA’s 2008 RBM Policy Update RBM policy update needed to make performance management more (a) rigorous, (b) modern and (c) pragmatic in order to:Standardize and provide better coherence in RBM terms, definitions and methodology used in across the agency,Align RBM at CIDA with Canadian Government approach, andHarmonize RBM at CIDA with the International Donor CommunityCANADIAN CONTEXTRecent Canadian governments have responded to increased public pressure for government accountability and performance through new standards for stronger accountability and reporting obligations.2006 Federal Accountability Act (FAA)Bill C293: Official Development Assistance Accountability ActINTERNATIONAL CONTEXTThe last 10 years have seen the growth of a consensus and commitment to performance in development2000 Millennium Development Goals (MDGs)2002 Monterrey Conference on Financing for Development2005 Paris Declaration on Aid EffectivenessManaging for Development Results (MfDR) Joint Venture at the OECD/DAC provides a world forum to share lessons and advocateCIDA CONTEXTRBM has been used in CIDA for 30 years in various forms – considered model by peersLots of RBM info at project level, clearer PAA contextStaff understand need for monitoring/reporting resultsBUTRBM application became overly individualized, Project info is fragmented, program-level info is fuzzy, Staff are demanding simpler monitoring/reporting formatsRBM definitions had not been updated, which created confusion, misalignment with GoC and international standards
9WHY is Results Based Management a preferred development planning, implementation and reporting approach?It focuses on the impact of interventions in the short, medium and long term
10“Results” in Day to Day Life InputsWaterSalt, condimentsCooking oilVegetablesMeat, fishFirewoodWorking timeOutcome ResultImmediate (short-term),Satisfied (after having eaten this sauce, main dish or meal)Outcome ResultMedium- term effect,improved physical well-being (if I can make sure I have a balanced diet over time)ActivitiesPreparing the ingredients, cooking the meal, serving the mealOutcome ResultLong-term effect,‘quality of life’ (if I can ensure I have a balanced diet for more than 5 years)OutputsThe sauce, the main dish, the meal
11Typically, governments, development agencies and CSOs have focused their attention on: inputs (what they spend),activities (what they do), andoutputs (what they produce).
12What is Missing?The destination►When we simply focus on inputs (spend), activities (do), and outputs (goods and services produced) we are blind - we don`t know our destination so we can’t see when and where we have gone off course
13Modern development management requires planners, programmers, financial officers and M&E specialists to look beyond activities and outputs to focus on:actual results(the impacts and effects of their programs)
14Various Terminology Used Around the World USAIDNORADDFIDOECDTBS andCIDA (2008)CIDA( )ImpactUltimateOutcomeStrategicObjectiveGoalImpactGoal(Impact)OutcomesIntermediateOutcomesIntermediateResultPurposeOutcomes(short andMedium term)Purpose(Outcome)ImmediateOutcomesProject levelIntermediateResultOutputsPoints à souligner : Yvetta (suite)La terminologie de la gestion axée sur les résultats (GAR) varie d’un organisme à l’autre. IL arrive parfois qu’un même terme ait des sens différents.Par exemple, la Banque mondiale appelle objectif ce que l’ACDI considère comme un effet – énoncé au passé.Pourquoi voir cette notion maintenant? Nous voulons que vous vous concentriez sur les concepts plutôt que sur la terminologie.CONSEIL(S):Lorsque l’on collabore avec d’autres organisations, il faut d’abord déterminer si elles utilisent une méthode de GR et, le cas échéant, leur demander comment elles définissent certains termes et quels outils elles utilisent. Grâce à ces renseignements, il sera plus facile d’établir un parallèle entre leur chaîne de résultats et leurs indicateurs et les outils de GR de l’ACDI.Lorsque vous comprenez les concepts qui se cachent derrière la terminologie de l’ACDI en matière de GR, ne vous vous accrochez pas aux définitions. Dégagez-vous-en, concentrez-vous sur les concepts. Imaginez-vous en situation réelle. Les mots exprimés dans différentes langues ont la même signification parce qu’ils représentent un seul et même concept.Une fois que vous saisissez les principaux principes et concepts, vous serez en mesure de naviguer à travers les différents outils et la terminologie de GR utilisés par différents organismes. Les concepts sont les mêmes, peu importe la langue. Par exemple, le mot chien en français, dog en anglais et pes en slovaque désigne tous un animal à quatre pattes qui aime les humains.Documents à remettre :Le CAD a publié un dictionnaire de la GR en français, en espagnol, en anglais et dans une autre langue (je crois)OuputsOutputsOutputsOutputsActivitiesActivitiesActivitiesActivitiesActivitiesActivities(Process)
16What is a Result?A result is a describable or measurable change in state that is derived from a cause and effect relationship.Source : RBM in CIDA - Policy Statement (2008)This definition comprises two important elements:Importance of measuring changeImportance of cause and effectResults are not stagnant – they exist in an organic, cause and effect relationship
17Two Major Elements in “Results” CHANGECAUSALITYMeasurable transformationIn a groupIn an organizationIn a societyIn a countryCause and effect relationship between an action and the results achieved – this can be described as the“If-then” logic.
18Results-Based Management Defining realistic expected results, based on appropriate analyses;Clearly identifying program beneficiaries and designing programs to meet their needs;Monitoring progress towards results and resources consumed, with the use of appropriate indicators;Identifying and managing risks, while bearing in mind expected results and the necessary resources;Increasing knowledge by learning lessons and integrating them into decisions; andReporting on results achieved and the resources involvedThis slide outlines the principles of the RBM management philosophy. RBM is…..1.Analysis – You need proper analysis (political, socio-economic, gender, cultural, ethnic etc..)to choose activities, beneficiaries and intermediaries. Refer to examples used later in the results chains to point out the problems when analysis isn’t conducted (I.e. targeting training at men when it is really women who look after water in the community, or putting in wells but not realizing that the responsible party – the Regional Water Authority – doesn’t have the capacity to maintain them) Often it is useful to start with the Problem Statement – the problem tree analysis.2. Identify beneficiaries (and stakeholders, and intermediaries)3.Monitor – choose sound indicators to measure progress towards results achieved and made adjustments when and where required during the life of the investment (when you realize that the RWA doesn’t have the capacity, add an activity to address that)4.Identify and Manage Risk – Figure out what could impact your investment and either change activities to avoid or deal with or develop risk-response strategies to cope if the event occurs.5. Incorporate lessons learned into your ongoing investment and into future programming6.Report on results achieved – compare expected results to actual results and talk about any variance between the two.Risk Management
20Development results Immediate outcome Inter-mediate outcome Knowledge People Materials resourcesOrganise SRH workshops for women and men in Community XSRH workshops for women and men held in Community XEnhanced awareness of sexual and reproductive health rights in Community XIncreased decision-making capacity of women and girls to negotiate safer sex practices in intimate relationshipsWomen and girls in Community X have enhanced sexual and reproductive autonomyImmediate outcomeInter-mediate outcomeUltimate outcomeDevelopment results
21The Importance of Analysis The results chain should be informed by analysis (gender, environment, social, political etc…)Ensure that gender equality results are identified at outcome levels as required by policyEnsure that there is a logical connection through activities, outputs and outcomes to support the integration of cross-cutting themes (I.e., don`t just stick them in the results)
23The Result StatementA result statement outlines what a policy, program or investment is expected to achieve. At CIDA, result statement = Outcome.A result statement or Outcome describes the change stemming from CIDA’s contribution to a development activity in cooperation with others.Different types of change (and each type has its own place in the LM):Change in system, stateChange in circumstancesChange in behaviour or practicesChange in functioning or performanceChange in knowledge or awarenessChange in skills or abilities
24Language of Change Results are about change ► for the better Change is signified by words such as:Improved (health conditions)Increased (income of local farmers)Strengthened (capacity of local NGOs)Reduced (vulnerability)Enhanced (ability to apply RBM)Promoted (engagement with stakeholders)Result statement is always phrased in the past tense to demonstrate expected changeUse an adjective drawn from a verb in the past tense that will demonstrate the direction of expected change
25LM ExampleLMs usually contain more than one results chain and address a number of different elements in the context (in relation to the problem statement), all of which lead, logically to the ultimate outcome.
26Using the Logic ChainUltimate Outcome: Change in State for target populationIntermediate Outcomes: Change in practice and/or behaviour of target populationImmediate Outcomes: Change in knowledge, awareness or access (etc)Outputs: Complete activities (good and services)Activities: What CIDA/partners do to mobilize inputs and create outputs.???This slide provides a reminder of the LM structure and the kinds of change seen at each level. Put it up for reference during Debrief.??
27Integrating Rights into RBM HUMAN RIGHTSPOLITICALSOCIALCULTURALECONOMICGenderEnvironmentLabourDisabilityChildrenOlder PeopleLandFood SecurityDevelopmentBuilt into the Results Framework
28Performance Measurement Framework (PMF) Used to systemically plan the collection of relevant data to assess and demonstrate progress made in achieving expected resultsEnsures performance information is collected on a regular basisAllows for real-time, evidence-based management decision makingDeveloped and monitored through consultation with partners, other donors, local stakeholders and sometimes beneficiariesA performance measurement framework (PMF) is used to systematically plan the collection of relevant data over the lifetime of an investment to assess and demonstrate progress made in achieving expected results.Best defined through multi-stakeholder workshops.It is critical to include the direct beneficiaries/target populationA Performance Measurement Framework will help structure the answers to who will do what, when and how.It will document the major elements of the monitoring system and ensure that comparable performance information is collected on a regular and timely basis
29THE PERFORMANCE MEASUREMENT FRAMEWORK A Performance Measurement Framework (PMF) is used to systematically plan the collection of relevant data over the lifetime of an investment to assess and demonstrate the progress toward achieving expected results.A PMF:Is best defined through multi-stakeholder participatory workshops;Will help structure the answers to who will do what, when and how;Will document the major elements of the monitoring system and ensure that comparable performance information is collected on a regular and timely basis;Allows for real-time, evidence-based management decision making; andShould be developed and monitored through consultation with partners, other donors, local stakeholders and beneficiaries.Each outcome should aim for three indicatorsMixture of qualitative and quantitative; andMixture of country-based and project-based.
30Performance Indicators A performance indicator is a unit of measure-ment that specifies what is to be measured along a scale or dimension but does not indicate the direction or change.Performance indicators are a qualitative or quantitative means of measuring an output or outcome, with the intention of gauging the performance of a program or investment.Performance indicators are neutral & do not indicate directionality & do not embed a target.Quantitative performance indicators are discrete measures such as number, frequency, percentile, ratios, variance, etc. In other words, they can be identified as measures of quantity. Based on information gathered from census, surveys, enumerations and administrative records and are often referred to as hard, objective indicators.Qualitative performance indicators can be defined as people’s judgment and perception about a subject. Where quantitative measures are not possible, objective measurement of qualitative changes is possible by defining indicator in behavioral terms.Example:Quantitative performance indicatorsNumber of: Number of women that gain employment – Reported as: 300 women gain employment by June 2001.Frequency of: Frequency of feeding programs carried out in three schools - Reported as: Weekly feeding programs carried out in September 2000.% of: % of counseled persons - Reported as: 35% increase in persons being counseled by July 2001.Ratios: Ratio of adults that can read and write - Reported as: 8 out every 10 adults can read and write by December 2001.Qualitative performance indicators can be expressed as:Extent to: Extent of involvement of religious leaders -Degree of: Degree of client satisfaction as defined by the speed of laboratory testing processes-Level of: Level of participation of women in decision-making -
31Examples of Performance Indicators Qualitative(experiential or perception):attitudes,skills,perceptions of,quality oflevel of (understanding),beneficiary opinion,extent of,congruence withpresence ofetc.:Quantitative (discrete measures):number of,frequency of,ratio of,variance with,% of,etc.:Examples:QuantitativeNumber of women in the decision-making positionsNumber of women and men capable of reading basic material.Ratio of men and women in decision-making positions in the GovernmentNumber of human rights violationsQualitativeLevel of satisfaction of the beneficiaries (as defined by…).Quality of the service provided.Perception of men on women’s participation in the local Committee.Coherence between the management tools developed and the absorptive capacity of the beneficiaries.Perception of staff about their skills in revenue generation
32Gender Sensitive Indicators Measure the gender gaps and inequalities you are seeking to addressPoints to changes is the status and roles of women and men over timeRequires the collection of data disaggregated by sex, ageWomen and men should be actively involved in the planning process and selection of indicators that make sense
33Performance Indicator vs. Target vs. Result Often there is confusion between results, performance indicators, and targetsResults: are precise statements of what is to be accomplished:Increased literacy among groups X and Y.Performance indicators: specify exactly what is to be measured along a scale or dimension, but do not indicate the direction of changeLiteracy rate, level of knowledgeTargets: specify a particular value for an indicator to be accomplished by a specific date in the futureTotal literacy rate to reach 85% among groups X and Y by the year 2010Often there is confusion between results, indicators, and targets
34Results Based Budgeting RBB is an integral part of the RBM cycleREPORTINGBUDGETING
35Results-based implementation Results based monitoring, evaluation and reportingResults-based costing and budgetingResults-based planning
36Project and Programme Results Finance and resourcing results through RBBOperations Planning Programming and Implemen-tationCosting & BudgetingIT systemsCapacity and Organisational DevelopmentKnowledge ManagementPerformance ManagementMonitoring & EvaluationHuman and Organisational ResultsPerformance & Sustainability Results
37Results Based Budgeting (RBB) A results-driven budgeting process in which:Programme formulation and resources justification involve a set of predefined objectives, expected results, outputs, inputs and performance indicators which constitute a ‘logical framework’;Expected results justify resource requirements, which are derived from — and linked to - outputs to be delivered, with a view to achieving such results; andActual performance in achieving results is measured by predefined performance indicators
38Results Based Budgeting “Results-based budgeting (RBB) is about formulating budgets that are driven by a number of desired results which are articulated at the outset of the budgetary process, and against which actual performance is measured at the end of the period.”
39Traditional Budgeting Framework Results directly linked to the budgeting approachOutcomesInputsActivitiesOutputsWhat we investWhat we doBudgeting approach focuses on costing activities / deliverables
40Costing Results Costed Results Costed Implementation PLANING FOR RESULTSAttribution GapLong-term Goal (Impact)OutcomesOutputsActivitiesInputsCan we cost for the effects or behavior changes resulting from program outputs?Can we cost for long-term, widespread improvement in society?Costing the products and services used to simulate the achievement of resultsCosting the utilization of resources to generate products and servicesCosting resources committed to program activitiesCosted ResultsCosted ImplementationTraditional BudgetsResults-based budgets
41Intermediate Outcomes Results Based Budgeting ApproachInputsOutputsActivitiesIntermediate OutcomesChange in behaviourPerformance indicatorsChanges in practiceUltimate OutcomesChanges in stateOld Budgeting FocusNew Budgeting Focus
42Risk Management Process 5. EvaluationImproving the processOrganizational learningPerformance reporting4. MonitoringDecision-makingAdjusting3. Risk ResponseDetermining optionsMitigation optionsImplementing2. Risk AssessmentImpact and likelihoodRisk rankingRisk tolerance1. Risk IdentificationEnvironment scanningPartner collaborationSelecting key risk areasUnderstandingContextCommunications strategyGender considerationsStakeholder consultationAccountabilitiesFeedbackThis chart highlights the risk management process at CIDA. Now we only really begin to have risk management once we finish the 3rd step. Only after this is completed can we say we are effectively managing risks and managing for risks.Major criticisms of CIDA risk management practices are cleverly embedded in the model. They can be found at EDRMS #309640
43Reputation / Public Confidence CIDA’s Key Risk AreasOperationalRisksFinancialDevelopmentOp1: Human resourcesOp2: Performance mgtOp3: Information systemsDev1: StrategicDev2: Socio-politicalDev3: Inst capacityDev4: ModalityDev5: Disasters, Environment, diseaseReputation / Public ConfidenceFin1: FundingFin2: FiduciaryFin3: InstrumentThis model similar to several other international organizations. The 12 key risks were developed by process including document searches, focus groups, rating sessions, management consultations, etc. In a study of 640 risks found in recent RRMAFs, all risk fitted the model except two.It is important that this model evolve continuously to reflect the changing risk environment.Operational : events that could impact on CIDA’s ability to operate effectively and efficientlyFinancial/Legal/Contract : events that could impact on CIDA’s ability to properly protect public fundsDevelopment : events that could impact on CIDA’s ability to meet expected resultsReputation :reduction in stakeholder confidence or reputation
44Typical Risk MatrixWhen risks are too high, use additional risk response strategies to reduce the overall risk level.Inherent risk: Risk level considered with current risk response.Residual risk: Risk level after additional risk response.
45Risk Register Purpose of the Risk Register: A risk register lists all the identified risks, the results of their analysis and a summary of risk response strategies. The risk register should be continuously updated and reviewed over a regular reporting schedule for the length of the project.
46Elements of a Risk Register Risk definitionInitial level of riskRisk levels through the project life cycleRisk response strategiesRisk owner
47Example Risk definition Start Date 2 Date 3 Risk response Owner For Program level onlyRisk definitionStartDate 2Date 3Risk responseOwnerOperational1. There is a risk that …2. … performance mgt..Very LowImprove data collection …Introduce new UN indicatorsDOHighFinancial-Risk 3-Risk 4Very HighExtended use of FRAU …None needed at the moment.D.DirExternal factors-Risk 5-Risk 6LowOther outcome risks-Risk 7